Python Databricks Engineer

Company Overview

Not specified

Job Summary

The Python Databricks Engineer will play a crucial role in data engineering, utilizing Python and Databricks to process and analyze healthcare data. This position requires a candidate with a strong background in data engineering who can work independently and efficiently within a hybrid work environment.

Responsibilities

• Design, develop, and maintain data pipelines using Python and Databricks for efficient data processing.
• Collaborate with cross-functional teams to gather requirements and deliver scalable data solutions.
• Utilize Apache Spark for large-scale data processing tasks and ensure optimal performance.
• Integrate AWS technologies such as S3 and Lambda into data processing workflows.
• Develop and manage ETL processes using DataStage or similar ETL tools.
• Schedule and monitor workflows with Apache Airflow to ensure timely data delivery.

Qualifications

• Education: Bachelor’s degree in Computer Science, Engineering, or related field.
• Experience:
  • 5+ years of Data Engineering experience.
  • 2+ years of Python development experience.
• Technical Skills:
  • Proficiency in Databricks, Apache Spark, and SQL.
  • Experience with AWS services such as S3 and Lambda.
  • Familiarity with healthcare data processing systems.
  • Practical knowledge of batch processing frameworks.

Preferred Skills

• Strong analytical and problem-solving skills.
• Excellent communication and teamwork abilities.
• Familiarity with additional data pipeline tools and technologies.

Experience

• 7-10 years of relevant experience in Data Engineering, with demonstrated expertise in Python and Databricks.

Environment

• Work Setting: Hybrid model; candidates are expected to work 2-3 days from the office located in Bangalore, Pune, Mumbai, Hyderabad, or Noida.
• Working Hours: 11 AM to 8 PM Indian Standard Time, with a requirement for 4 hours overlap with US business hours.

Salary

• Estimated starting salary of 2.4 LPM + Tax. Note: The final rate may be negotiable based on feedback and experience.

Growth Opportunities

Not specified

Benefits

Not specified