Data Engineer

Company Overview

Company overview is not specified.

Job Summary

The Data Engineer will be responsible for designing, building, and maintaining scalable data pipelines. This role is crucial for processing large volumes of data efficiently and developing enterprise-grade data solutions on the Azure cloud platform, contributing to the organization's data-driven decision-making processes.

Responsibilities

• Design and implement complex data pipelines using Azure Data Factory and Databricks.
• Write and optimize complex SQL queries to manage and manipulate data effectively.
• Architect and build enterprise-grade data platforms leveraging Azure Data Services, including ADLS Gen2 and Synapse Serverless.
• Gather requirements and translate them into technical architectures while assessing the pros and cons of different architectural choices.
• Employ streaming technologies such as Kafka and Spark Structured Streaming for real-time data processing.
• Implement Infrastructure as Code (IaC), Continuous Integration/Continuous Deployment (CI/CD), and DevOps practices in data analytics projects.

Qualifications

• Bachelor’s degree in Computer Science, Information Technology, or a related field.
• In-depth knowledge of Data Engineering principles and practices, particularly on Azure Databricks.
• Strong programming skills with proficiency in Python.
• Extensive hands-on experience in building complex data pipelines within Azure Data Factory.
• Well-versed in Azure Data Services such as ADLS Gen2 and Synapse.
• Multi-year experience (over 6 years) in a data engineering role, including expertise in data processing and transformation.
• Familiarity with cloud computing concepts, particularly those relevant to the Azure platform, including networking and security practices.
• Experience with IaC tools and practices, preferably using Azure DevOps and Terraform.

Preferred Skills

• Knowledge of Microsoft Fabric would be considered an advantage.
• Familiarity with Big Data solution architectures and best practices in cloud services selection (e.g., ADF, Databricks, Synapse).

Experience

• A minimum of 6 years in a data engineering role with a strong track record of designing and maintaining large-scale data solutions.

Environment

Typical work setting and location are not specified.

Salary

Salary details are not provided.

Growth Opportunities

Opportunities for career advancement within the organization are not specified.

Benefits

Details on offered benefits are not provided.