Data Engineer

Company Overview

[Company information not provided]

Job Summary

The Data Engineer will be responsible for designing, developing, and maintaining robust data pipelines that ensure optimal performance, scalability, and maintainability. This role plays a crucial part in supporting the organization's data-driven initiatives by collaborating with cross-functional teams and providing mentorship to other team members.

Responsibilities

• Design, develop, and own robust data pipelines, ensuring optimal performance, scalability, and maintainability.
• Design and implement Data Lake, Data Warehouse, and Lakehouse solutions with various architecture patterns.
• Ensure data quality, integrity, and governance across all stages of the data lifecycle.
• Monitor and optimize the performance of data engineering pipelines.
• Contribute to design principles, best practices, and documentation.
• Collaborate closely with cross-functional teams to understand business requirements and translate them into effective technical designs and implementations.
• Provide mentorship and guidance to team members of the data engineering team, promoting knowledge transfer and a culture of continuous learning.

Qualifications

• Bachelor's degree in Computer Science, Information Systems, or a related field; Master’s degree is a plus.
• Minimum of 8+ years of experience as a Data Engineer.
• Deep experience in designing and building robust, scalable data pipelines – both batch and real-time using modern data engineering tools and frameworks.
• Proficiency in AWS Data Services (S3, Glue, Athena, EMR, Kinesis, etc.).
• Strong understanding of SQL queries and various file formats such as Apache Parquet, Delta Lake, Apache Iceberg, or Hudi and Change Data Capture (CDC) patterns.
• Experience in stream processing frameworks like Apache Flink or Kafka Streams, or any other distributed data processing frameworks like PySpark.
• Expertise in workflow orchestration using Apache Airflow.
• Strong analytical and problem-solving skills, with the ability to work independently in a fast-paced environment.
• In-depth knowledge of database systems (both relational and NoSQL) and experience with data warehousing concepts.
• Hands-on experience with data integration tools and familiarity with cloud-based data warehousing and processing is highly desirable.
• Excellent communication and interpersonal skills, facilitating effective collaboration with both technical and non-technical stakeholders.
• A strong desire to stay current with emerging technologies and industry best practices in the data landscape.

Preferred Skills

• Familiarity with additional cloud providers or data processing tools.
• Experience in implementing data governance frameworks.

Experience

• A minimum of 8+ years of relevant experience in data engineering.

Environment

• Work from office (WFO) in Gurugram.

Salary

• ₹1.3 Lakh Per Month + GST.

Growth Opportunities

[Company growth opportunities not provided]

Benefits

[Company benefits not provided]