Senior ADF Developer / Lead (Azure Data Factory) with Data Modeling Expertise

Company Overview

Company details are not specified.

Job Summary

The Senior Azure Data Factory (ADF) Developer/Lead will play a crucial role in designing scalable, secure, and high-performance data solutions to support both operational and analytical workloads. This position involves extensive collaboration with various stakeholders to gather requirements and deliver reliable data solutions through advanced data modeling and integration techniques.

Responsibilities

• Lead data architecture and modeling efforts for OLTP (Online Transaction Processing) and OLAP (Online Analytical Processing) systems using advanced techniques.
• Collaborate with business stakeholders, data architects, and product owners to gather requirements and deliver scalable data solutions.
• Design, develop, and optimize ADF pipelines, including parameterized pipelines and Mapping Data Flows for complex data transformations.
• Create and maintain PostgreSQL schemas, tables, views, materialized views, and stored procedures.
• Perform advanced data modeling including 3NF (Third Normal Form), Star/Snowflake schemas, and Slowly Changing Dimensions (SCD Type 1/2).
• Optimize database performance through query tuning, indexing, and partitioning.
• Implement robust data quality checks, security best practices, and CI/CD (Continuous Integration/Continuous Deployment) pipelines for ADF and PostgreSQL changes.
• Develop and maintain logging and monitoring mechanisms in ADF to ensure pipeline health and troubleshoot issues.
• Manage file handling in ADF, including reading, writing, and transforming data from multiple formats.
• Ensure data integrity and validation through comprehensive quality checks.
• Review pull requests, enforce coding standards, and take ownership of code quality and maintainability.
• Handle complex SQL queries and optimize performance for large-scale data scenarios.

Qualifications

• Educational Qualification: A Bachelor's or Master's degree in Computer Science, Information Technology, or a related field.
• Extensive expertise in Azure Data Factory, especially in designing and implementing parameterized pipelines, Mapping Data Flows, and performance optimization.
• Advanced proficiency in PostgreSQL covering indexing, partitioning, query tuning, transaction management, and performance optimization.
• Proven experience in data modeling, including dimensional modeling (Star/Snowflake schemas, SCD Type 1/2) and OLTP modeling (3NF).
• Strong understanding of ETL (Extract, Transform, Load)/ELT (Extract, Load, Transform) patterns, including Change Data Capture (CDC), late-arriving data handling, and data reconciliation.
• Hands-on experience with Azure services such as ADLS Gen2 (Azure Data Lake Storage Gen2), Key Vault, Azure Monitor, and Log Analytics.
• Expert-level SQL skills for handling complex business logic, data transformations, and performance tuning.
• Experience with Git-based workflows, including branching strategies, and building CI/CD pipelines using Azure DevOps, GitHub Actions, ARM (Azure Resource Manager)/Bicep templates, and Flyway.

Preferred Skills

• Familiarity with Microsoft Purview, Azure Synapse, Databricks, or Python scripting for data engineering tasks.

Experience

• Minimum experience requirements are not specified; however, prior experience in a similar role is expected.

Environment

• Work setting, location, and physical or environmental conditions are not specified; details may vary based on company norms and practices.

Salary

• Salary details are not specified.

Growth Opportunities

• Opportunities for career advancement are not specified, may be available based on performance and company growth.

Benefits

• Benefits details are not specified.