Job Description

xf0x9fx8cx9f We're Hiring: Data Migration Engineer! xf0x9fx8cx9f
Role Overview:
Seeking a skilled Data Migration Engineer with 4-8 years of experience in data migration, data setup, and data systems development. The ideal candidate should have expertise in Spark, SQL, and Java(Scala) for data processing, reporting, and development. Strong knowledge of data architecture and semantic layer development is required, along with experience in regression testing and cutover activities in large-scale migrations.
Key Responsibilities:
Spark:

• Develop and optimize Spark-based data processing for large-scale ETL and analytics.
• Implement Spark SQL, DataFrames, RDDs, and Streaming for efficient data transformations.
• Optimize Spark job performance, tuning memory, partitioning, and execution plans.
• Handle real-time and batch data processing using Spark Streaming or structured streaming.

• Write and optimize complex SQL queries for data extraction, transformation, and aggregation.
• Work on query performance tuning, indexing, and partitioning for optimized execution.
• Develop and manage stored procedures, functions, and views for data operations.
• Ensure data consistency, integrity, and security in relational database systems.

• Java is essential as Scala runs on the JVM, making JVM tuning critical for Spark-based workloads.
• Develop data processing applications using Scala (running on JVM) and Java-based backend services.
• Optimize JVM performance, memory management, and garbage collection to enhance Spark job execution.
• Utilize Scalas functional programming features for efficient data transformations in Spark.
• Implement multithreading, concurrency, and parallel processing in Java for high-performance applications.