Azure Databricks

Job Summary

We are seeking a skilled and experienced Azure Databricks professional to join our team on a full-time basis in Chennai. The ideal candidate will be responsible for designing, developing, and maintaining big data solutions using Azure Databricks, PySpark, and SQL programming. This role will play a vital part in ensuring the reliability, efficiency, and scalability of our data engineering infrastructure, contributing directly to the organization’s data-driven decision-making and analytics initiatives.

Responsibilities

• Design, develop, and optimize large-scale data processing solutions using Azure Databricks.
• Implement data pipelines and ETL processes utilizing PySpark and SQL programming.
• Collaborate with data scientists, analysts, and other stakeholders to understand data requirements and deliver solutions.
• Monitor, troubleshoot, and enhance existing Databricks jobs and workflows for performance and reliability.
• Ensure best practices for data security, integrity, and compliance within the Azure ecosystem.
• Document technical designs, code, and processes for ongoing maintenance and knowledge sharing.

Qualifications

• Bachelor’s degree in Computer Science, Information Technology, Engineering, or a related field.
• Minimum of 5 years of hands-on experience in data engineering or related roles.
• Proficient in Azure Databricks and its ecosystem.
• Strong experience with PySpark for large-scale data transformations.
• Advanced SQL programming skills for data manipulation and querying.
• Solid understanding of data modeling, ETL concepts, and data warehousing principles.
• Strong analytical, problem-solving, and communication skills.

Preferred Skills

• Experience with other Azure data services (e.g., Azure Data Factory, Azure Synapse Analytics).
• Familiarity with CI/CD pipelines and DevOps practices for data engineering.
• Knowledge of data governance, data quality, and compliance standards.
• Experience with version control systems such as Git.
• Exposure to cloud security best practices and monitoring tools.

Experience

• Minimum 5 years of relevant experience in data engineering, with a focus on Azure Databricks, PySpark, and SQL programming.
• Demonstrated track record of building and optimizing data pipelines in a production environment.

Environment

• Work Location: Chennai (in-office or hybrid arrangement, as per company policy).
• Work Type: Full-time.
• Fast-paced, collaborative, and technology-driven environment.
• Immediate start date available.

Deadline

• Applications accepted until 2025-06-22.