Please refer to this job description.

Roles and Responsibilities

• Work closely with the Architect to understand the high-level design and translate it into actionable tasks.

• Implement data pipelines, workflows, and storage structures as per the design.

• Ensure to setup ADLS Gen 2 with proper folder structures and ensure data is stored in optimized file formats.

• Develop robust ETL pipelines using Azure Data Factory(ADF) and/or Synapse Pipelines

• Implement incremental data loads to minimize processing costs and improve efficiency.

• Write optimized SQL queries for Serverless SQL pool, leveraging Predicate Pushdown and partition pruning.

• Implement robust error handling mechanisms in pipelines to handle failures gracefully

• Track and optimize costs by reducing unnecessary data scans

Required Skills

• Must have a minimum of 5 years’ experience with deep understanding and expertise in building scalable data solutions in Azure using ADLS Gen2, Synapse Analytics, ADF, Azure Functions etc

• Strong proven experience in SQL, data modeling and query optimization

• Proficiency in developing and optimizing ETL pipelines

• Experience with automation tools(Azure CLI, PowerShell, etc)

• Ensure adherence to best practices.

• Good communication and stakeholder management skills

• Azure certifications preferred