Lead - Data Engineer

Company Overview

Company Overview: Not specified.

Job Summary

We are seeking a Lead Data Engineer to oversee and manage our data engineering efforts. This role is critical in designing, implementing, and maintaining scalable data pipelines and architectures that support our data analysis and business intelligence initiatives. The Lead Data Engineer will play a significant role in ensuring data integrity, efficiency, and accessibility across the organization.

Responsibilities

• Design and implement robust data pipelines and architecture using Azure or AWS services.
• Utilize PySpark and Databricks for processing large datasets efficiently.
• Manage data integration and transformation using Azure Data Factory (ADF) and implement Medallion Architecture principles.
• Develop and maintain a Lakehouse and Data Warehouse infrastructure to facilitate data analytics.
• Ensure data quality and reliability through automated testing and monitoring strategies.
• Collaborate with cross-functional teams to understand data needs and provide insights effectively.

Qualifications

• Strong experience with Azure and/or AWS cloud environments.
• Proficient in using Synapse, Databricks, PySpark, and Azure Data Factory (ADF).
• Solid understanding of Medallion Architecture, Lakehouse concepts, and Data Warehousing principles.
• Bachelor’s degree in Computer Science, Information Technology, or a related field.
• Strong analytical, problem-solving, and organizational skills.
• Proven ability to work in a fast-paced, team-oriented environment.

Preferred Skills

• Familiarity with big data technologies and frameworks.
• Experience with data governance and security best practices.
• Knowledge of machine learning concepts and tools.

Experience

11 to 16 years of relevant experience in data engineering, focusing on cloud-based environments and data architecture.

Environment

This position offers a Hybrid work model, with opportunities to work from our offices in Chennai, Coimbatore, Bangalore, or Pune.

Salary

Salary details are not specified.

Growth Opportunities

Opportunities for advancement within the company may include transitioning to senior leadership roles within the data engineering team or moving into related areas such as data science or analytics.

Benefits

Benefits are not specified.