Data Engineer

Kuala Lumpur, Malaysia

Job Description


As part of Renoir\'s Data CoE, the Data Engineer (DE) will serve to develop databases and data processing pipelines to enable predictive analytics, data science, and business intelligence for our clients. The Data Engineer will also have strong analytical skills and the ability to combine data from different sources. To succeed in this role, the DE should know how to work with key stakeholders, the BA/Data Analyst, and the development team to analyse and define technical solutions to meet business and non-functional requirements. Design, development, and testing of: ETL and data processing pipelines. Data Lakes, Data Warehouses, and Delta Lakes. Analytical Data modelling. Aggregating data using SQL for BI tools. Database schemas, indexes, views, etc. SQL queries and stored procedures. Cloud infrastructure and services (AWS, Azure) Data security (encryption, masking, etc.) Assist in the creation and support of analytic, BI, and reporting processes. Responsibilities: Responsible for analysing, building, and optimising the data infrastructure and data pipeline along with Renoir\'s clients. Support the client to, potentially, migrate data from legacy systems to new solutions to align the system architecture with business requirements while enabling the delivery of Renoir\'s data solutions. Support the design of conceptual and logical data models and flowcharts.Identify ways to improve data reliability, efficiency, and quality. Connect all clients\'s data sources into Renoir\'s data solution (third-party vendor). Building end-to-end data pipeline infrastructure for optimal extraction, transformation, and loading of data from various data sources using AWS and SQL technologies - Glue, Lambda, Redshift, Athena, S3, Step Functions, PySpark, and Data Lake. Analyse the client\'s complete system landscape and specific system requirements and implement migration methods for existing data and/or consolidation of data into data warehouse cubes with the support of direct interfaces or APIs (where applicable). Support the client to improve system performance by conducting tests, troubleshooting, and integrating new elements. Optimise new and current database systems. Define security and backup procedures. Develop database solutions to ensure clients\'s information is stored effectively and securely. Provide the foundations and support the client to set in place the tools for continuous monitoring and testing of the system interfaces to ensure optimised performance and security. Analyse structural requirements for new solutions and applications that might be required for Renoir\'s clients. Quantify the benefit potential of Data Analytics solutions using ROI frameworks to aid in the prioritisation of development and definition with the client. Requirements: Bachelor\'s degree in Computer Science, Computer Engineering or a relevant field. At least 3-5 years of hands-on experience in a similar role. Technical Skills: Experience in writing complex SQL queries. Experience in AWS services such as Glue, Lambda, Redshift, Athena, S3, Step Functions, Event Bridge, Cloud Trail, and Cloud Watch. Experience working with ETL and ELT tools: Talend, SSIS, Glue, and Data Factory. Experience with scripting languages: Python, Spark, PySpark, etc. Experience with Microsoft Azure, AWS, GCP, and other cloud tools and services. Familiar with big data tools: Hadoop, Spark, Kafka, etc. Familiar with data pipeline and workflow management tools. Familiar with Familiar with analytics software tools such as Alteryx, Power BI, and Tableau. Good working knowledge of productivity tools such as Git, Jira, and Confluence. Familiar with DevOps and DataOps tools. Personal Attributes: Leadership skills. Innovative yet grounded in operational reality. Excellent oral and written communication skills. Excellent organisational and analytical abilities. Critical thinker with good logical and creative problem-solving skills. Work collaboratively. Fast learner and a self-starter.

foundit

Beware of fraud agents! do not pay money to get a job

MNCJobz.com will not be responsible for any payment made to a third-party. All Terms of Use are applicable.


Related Jobs

Job Detail

  • Job Id
    JD1022346
  • Industry
    Not mentioned
  • Total Positions
    1
  • Job Type:
    Full Time
  • Salary:
    Not mentioned
  • Employment Status
    Permanent
  • Job Location
    Kuala Lumpur, Malaysia
  • Education
    Not mentioned