Job Description

Description
We are looking for an experienced Lead Software Engineer (Big Data) to be placed in our Office in Bangsar South MY. As a Lead Engineer will be roll-up the functional and technical aspect of Software Development with a big data technology in Project space to cloud services.
EPAM Systems, Inc. (EPAM) is a global product development, digital platform engineering, and digital and product design agency headquartered in the US. EPAM has been named 5 times in Forbes’ 25 Fastest Growing Public Tech Companies. Kuala Lumpur is the most recent location in which EPAM has acquired a new business. This is your opportunity to join an organization in start-up mode that plans to (at least) double in size every 2-3 years and the team that you are joining will be responsible for that.
We Value
• Value the individuals – We encourage and motivate people to grow. We perceive our people as a source of our success.
• Strive for excellence – We strive for the highest standards of excellence and continuously learn. We take pride in our engineering and accomplishments.
• Act as a team – We treat one another with respect and encourage the best ideas to come from anywhere within the organization. We value our diversity.
• Focus on Customers – We build long term customer relationships, as we strive to always understand our customers' business and needs.
• Act with integrity – We operate legally, honestly and ethically. We take responsibility for our actions.
 
We Offer
• Friendly team and enjoyable working environment
• Work-life balance and flexible schedule
• Online training library, mentoring, career development and potential partial grant of certification
• Unlimited access to LinkedIn learning solutions
• Referral bonuses
• Compensation for sick leave and paid time off
• Opportunities for self-realization
 
Responsibilities
• Lead, design and implement innovative analytical solution using Hadoop, NoSQL and other Big Data related technologies, evaluating new features and architecture in Cloud/ on premise/ Hybrid solutions.
• Work with product and engineering teams to understand requirements, evaluate new features and architecture to help drive decisions.
• Build collaborative partnerships with architects and key individuals within other functional groups.
• Perform detailed analysis of business problems and technical environments and use this in designing quality technical solution.
• Actively participate in code review and test solutions to ensure it meets best practice specifications.
• Build and foster a high-performance engineering culture, mentor team members and provide team with the tools and motivation.
• Write project documentation.
 
Requirements
• Candidate must possess at least Bachelor’s Degree in preferably Computer Science/ Information Technology or equivalent.
• Candidates with a minimum of 6 years solid experience in Big Data technologies and Enterprise Software Development.
• Engineering experience and practice in Data Management, Data Storage, Data Visualization, Disaster Recovery, Integration, Operation, Security.
• Strong experience building data ingestion pipelines (simulating Extract, Transform, Load workload), Data Warehouse or Database architecture.
• Strong experience with data modelling; hands-on development experience with modern Big Data components.
• Cloud: experience in designing, automation, provisioning, deploying and administering scalable, available and fault tolerant systems.
• Good understanding of CI/CD principles and best practices.
• Analytical approach to problem-solving with an ability to work at an abstract level and gain consensus; excellent interpersonal, leadership and communication skills.
• Motivated, independent, efficient and able to handle several projects: work under pressure with a solid sense for setting priorities.
• Ability to work in a fast-paced (start-up like) agile development environment.
• Strong experience in high load and IoT Data Platform architectures and infrastructures
• Vast experience with Containers and Resource Management systems: Docker, Kubernetes, Yarn.
• Experience in direct customer communications.
• Experience in technology/team leading of data-oriented projects.
• Solid skills in infrastructure troubleshooting, support and practical experience in performance tuning and optimization, bottleneck problem analysis.
• Experienced in different business domains.
• English proficiency.
 
Technology Stack
• Programming Languages: Java/ Scala; Python; SQL; Bash.
• Big Data stack: Hadoop, Yarn, HDFS, MapReduce, Hive, Spark, Kafka, Flume, Sqoop, Zookeeper.
• NoSQL: Cassandra/ Hbase; MongoDB.
• Queues and Stream processing: Kafka Streams; Spark Streaming; Event Hub; IOT Hub; Storage Queues; Service Bus; Stream Analytics.
• Data Visualization: Tableau, QlikView.
• ETL & Streaming Pipelines: Pentaho; Talend; Apache Oozie, Airflow, NiFi; Streamsets.
• Operation: Cluster operation, Cluster planning
• Search: Solr, Elasticsearch/ELK
• InMemory: Ignite, Redis.
• Solid Cloud experience with more than 2 leading Cloud providers (AWS/Azure/GCP): Storage; Compute; Networking; Identity and Security; NoSQL; RDBMS and Cubes; Big Data Processing; Queues and Stream Processing; Serverless; Data Analysis and Visualization; ML as a service (SageMaker; TensorFlow).
• Enterprise Design Patterns (ORM, Inversion of Control etc.).
• Development Methods (TDD, BDD, DDD).
• Version Control Systems (Git, SVN).
• Testing: Component/ Integration Testing, Unit testing (JUnit).
• Deep understanding of SQL queries, joins, stored procedures, relational schemas; SQL optimization.
• Experience in various messaging systems, such as Kafka, RabbitMQ.
• Rest, Thrift, GRPC, SOAP.
• Build Systems: Maven, SBT, Ant, Gradle.
• Docker, Kubernetes, Yarn.

Skills Required