Scala Data Engineer

Company Overview

(Company details not specified)

Job Summary

We are seeking a highly skilled Scala Data Engineer to join our team. The successful candidate will be responsible for building and optimizing scalable data pipelines and developing streaming pipelines using Scala and Apache Spark. This role is crucial in enabling our organization's data strategy and improving the reliability and performance of our data systems.

Responsibilities

• Build and optimize scalable data pipelines using Spark and Scala.
• Develop streaming pipelines with Kafka/Event Hubs.
• Own the design and architecture of data systems, focusing on performance and reliability.
• Mentor junior engineers and enforce high-quality coding standards.
• Collaborate across teams to contribute to product delivery.

Qualifications

• Mandatory Skills:

  • Proficient in Scala (preferred over Java).
  • Strong knowledge of SQL.
  • Expertise in Apache Spark.
  • Strong understanding of data structures and algorithms.
  • Hands-on experience with data engineering on large-scale systems.
  • Exposure to cloud-native environments (preferably Azure).

• Preferred Skills:

  • Experience with Hadoop, Kafka, and streaming technologies (Structured Streaming/Event Hubs).
  • Proficiency in CI/CD tools such as Git, Docker, and Jenkins.
  • Familiarity with Medallion Architecture, Parquet, and Apache Iceberg.
  • Knowledge of orchestration tools like Airflow or Oozie, and NoSQL databases.
  • Experience with data governance tools such as Alation, Collibra, and lineage/metadata tracking.

Preferred Skills

• Experience with streaming and batch data processing.
• Knowledge of data governance practices and tools.

Experience

• 8 to 15 years of relevant experience in data engineering, with a strong focus on Scala and Spark.

Environment

• The position is based in Bangalore (Whitefield).
• Hybrid work mode: 3 days work from office per week.

Salary

• Budget: 1.8 LPM + GST.

Growth Opportunities

(Opportunities for career advancement not specified)

Benefits

(Benefits offered not specified)