Azure Databricks Data Engineer

Company Overview

[Company description not provided.]

Job Summary

We are seeking a skilled Azure Databricks Data Engineer with 2-4 years of experience to join our dynamic and fast-paced team. The ideal candidate will have hands-on experience working with Databricks Unity Catalog, Delta Lake, PySpark, SQL, Azure Data Factory (ADF), and other core Azure cloud services. You will play a pivotal role in building and maintaining scalable data pipelines, optimizing data workflows, and ensuring data security and governance.

Responsibilities

• Build and maintain scalable data pipelines using Azure Databricks and other Azure services.
• Optimize data workflows for performance and reliability.
• Ensure data security and governance practices are adhered to in all data engineering solutions.
• Collaborate with cross-functional teams to meet business requirements.
• Troubleshoot and resolve data-related issues to ensure reliable and consistent data delivery.

Qualifications

• Education: Bachelor's degree in Computer Science, Information Technology, or a related field.
• Experience: 2-4 years of hands-on experience in Azure Cloud-based data engineering solutions.
• Technical Skills:
  • Strong knowledge of Azure Databricks, including architecture, Delta Lake, Databricks Unity Catalog, workflow, compute, Azure Synapse Analytics, Azure SQL Managed Instance.
  • Proficiency in PySpark, with the ability to process large-scale datasets in distributed environments.
  • Hands-on experience with SQL, including query writing, optimization, and performance tuning.
  • Experience with Azure Data Factory (ADF).
  • Solid understanding of data governance, data lineage, access controls, and compliance frameworks.

Preferred Skills

• Strong problem-solving skills and ability to troubleshoot complex data pipelines.
• Experience working with Azure Data Lake Storage and other Azure services.
• Familiarity with CI/CD pipelines for deploying data solutions.

Experience

• 2-4 years of experience in data engineering with a focus on Azure technologies.

Environment

• [Work setting and location information not provided.]

Salary

• [Salary information not provided.]

Growth Opportunities

• [Career advancement opportunities not provided.]

Benefits

• [Offered benefits information not provided.]