Lead Data Engineer

Company Overview

Details about the company, its industry, and culture are not provided.

Job Summary

We are looking for a highly skilled Senior Data Engineer with deep experience in modern cloud data platforms, data engineering best practices, and scalable architectures. This role will focus on building and maintaining robust data pipelines, implementing Medallion architecture, and enabling enterprise-grade analytics using Microsoft Fabric, Azure Synapse, Databricks, and Lakehouse paradigms.

Responsibilities

• Design and implement scalable data pipelines using Azure Data Factory (ADF) and Databricks.
• Build and manage Lakehouse architecture leveraging Delta Lake, OneLake, and Medallion architecture (Bronze, Silver, Gold layers).
• Develop and optimize big data processing jobs using PySpark on Azure Databricks.
• Architect and manage data warehouses and data lakes on Azure or AWS cloud environments.
• Integrate and orchestrate data across multiple sources using Microsoft Fabric, Synapse Pipelines, and ADF.
• Collaborate with data scientists and analysts to deliver high-quality, production-ready data assets.
• Ensure data quality, governance, and security compliance across pipelines and storage layers.
• Optimize query performance and cost across cloud compute and storage layers.

Qualifications

• Strong experience with Azure or AWS cloud services.
• Hands-on expertise in Azure Synapse Analytics, Microsoft Fabric, and Databricks.
• Proficiency in PySpark, SQL, and distributed data processing frameworks.
• Good understanding of Medallion architecture, Delta Lake, and Lakehouse principles.
• Deep experience with Data Warehouse concepts and modeling (Kimball/Inmon).
• Experience with ADF and Synapse pipelines for orchestration.
• Familiarity with CI/CD for data (Azure DevOps, Git integration).
• Understanding of data security, access control, and compliance in cloud platforms.

Preferred Skills

• Experience with Microsoft OneLake or Data Activator in Fabric.
• Exposure to AWS Glue, Redshift, or Snowflake.
• Knowledge of ML pipeline integration in Databricks.
• Certification in Azure Data Engineer or AWS Big Data.

Experience

Details regarding years of experience and specific relevant experience are not provided.

Environment

Information about the typical work setting, location (remote, in-office, hybrid), and any physical or environmental conditions relevant to the job are not provided.

Salary

Salary information is not provided.

Growth Opportunities

Opportunities for career advancement within the company are not provided.

Benefits

Details about offered benefits such as insurance, paid leave, and work policies are not provided.