Azure Databricks (ADB) + PySpark Developer

Company Overview

Hexaware Technologies is a prominent player in the IT services industry, focused on delivering innovative solutions and services. The company's culture encourages growth, collaboration, and excellence in technology.

Job Summary

Hexaware Technologies is seeking a skilled ADB + PySpark Developer to join our growing Data Engineering team in Chennai. The ideal candidate will be responsible for building scalable data solutions using Azure Databricks, PySpark, and related Azure services, ensuring smooth, secure, and optimized data flows across systems.

Responsibilities

• Develop and maintain large-scale data pipelines using Azure Databricks and PySpark.
• Perform advanced data transformation, cleansing, and enrichment using Spark SQL and Python.
• Collaborate with cross-functional teams to translate business requirements into technical data solutions.
• Work on end-to-end implementation of data ingestion, processing, and consumption pipelines.
• Optimize PySpark jobs for performance and manage resource configurations on Azure Databricks.
• Integrate Azure Databricks with other Azure services such as Data Lake, Blob Storage, Azure Data Factory (ADF), and Key Vault.
• Ensure data quality, security, and compliance throughout the data lifecycle.
• Troubleshoot production issues and monitor workflows using monitoring tools or logs.

Qualifications

• Bachelor’s or Master’s Degree in Computer Science, IT, or a related field.
• Strong hands-on experience in Azure Databricks, PySpark, and Spark SQL.
• Proficient in Python programming for data processing and automation.
• Good understanding of Azure Data Lake Gen1/Gen2, Azure Data Factory, and Azure Blob Storage.
• Experience working with Delta Lake, Parquet, or other optimized storage formats.
• Strong SQL skills and experience in working with relational and non-relational databases.
• Familiarity with Git, DevOps practices, and code versioning.
• Experience in performance tuning and debugging Spark jobs.
• Azure Certifications like DP-203 or Azure Data Engineer Associate will be an added advantage.

Preferred Skills

• Exposure to Terraform, CI/CD pipelines, or Azure DevOps.
• Familiarity with dbt, MLflow, or real-time streaming with Spark.
• Understanding of data governance and role-based access in Azure.

Experience

• 4–8 Years in relevant fields, ideally in data engineering or cloud technologies.

Environment

• Location: Chennai (Work from Office).
• This is a Full-Time position within a collaborative and fast-paced work setting.

Salary

• Salary details are not specified.

Growth Opportunities

• Opportunities for career advancement may arise as the company expands and the Data Engineering team grows.

Benefits

• Details regarding benefits are not specified.