Job Description

Data Engineer

We are looking for a skilled

Data Engineer

to build a scalable big data processing platform that supports real-time decision-making and AI-powered automation. In this role, you will design and operate systems that process high-volume data, ensure data quality, optimize resource usage, and provide reliable insights to cross-functional teams. Your work will directly contribute to improved energy efficiency, enhanced data visibility, and seamless delivery of real-time analytics.

Key Responsibilities

Build and maintain scalable big data pipelines capable of handling large volumes of sensor and control data. Design and operate real-time data processing systems to support timely, data-driven decision-making. Integrate technologies such as

Kafka

and

Spark Streaming

to enable low-latency, event-driven data streaming. Implement cloud-based analytics solutions across

AWS

,

Azure

, or

GCP

for advanced data processing. Develop automated data quality checks and validation frameworks to ensure integrity across data pipelines. Collaborate closely with AI engineers, IoT engineers, and software teams for seamless ingestion, model development, and analytics workflows. Optimize compute and storage resources to reduce data processing and operational costs. Continuously monitor, test, and tune data systems for improved performance, reliability, and cost-efficiency. Implement and maintain metadata management processes and tools.

Who We're Looking For

3-6+ years of experience in big data engineering or data infrastructure. Hands-on expertise in large-scale data processing, ETL pipelines, and cloud-based storage. Strong proficiency in

Spark, Python, and SQL

. Experience with

Spark Streaming

,

Kafka

,

Delta Lake/Lakehouse

,

Databricks (including DLT)

, and

Azure Synapse

. Experience with data visualization tools such as

Apache Superset

,

Tableau

, or

Power BI

. Proficiency with version control systems such as

Git

. Solid background in cloud computing (

AWS, GCP, Azure

) and scalable infrastructure design. Strong understanding of data quality, automated validation, and schema management. Proven track record of improving system or pipeline efficiency (e.g., 40%+ improvement). Ability to collaborate with engineering and AI teams to deliver clean, production-ready datasets.
Job Type: Full-time

Pay: RM8,000.00 - RM14,000.00 per month

Benefits:

Health insurance Maternity leave Opportunities for promotion Professional development
Work Location: In person