Data Engineer - Data Bricks

Company Overview

Our client, a leading Insurance company, is at the forefront of providing innovative solutions in the insurance industry. They are dedicated to harnessing the power of data to enhance operations and deliver exceptional service to their clients.

Job Summary

The Data Engineer will play a crucial role in designing and implementing data acquisition and integration processes. This position aims to support the organization’s goals by ensuring efficient data processing and transformation, critical for providing insights and improving decision-making in the P&C Insurance sector.

Responsibilities

• Define data acquisition and integration designs for the atomic layer in data architecture.
• Configure Pyspark data pipelines to move and transform data across various data layers.
• Create and maintain HBase physical tables and Phoenix SQL tables to support data storage needs.
• Collaborate with relevant teams to ensure performance meets Service Level Agreements (SLAs).
• Confirm readiness of data for feeds and conversions based on target system and business process requirements.
• Apply Data Vault methodology with prior experience in insurance data model on Azure with Pyspark coding.

Qualifications

• 8+ years of experience in data engineering or related roles.
• Proficiency in Data Vault, Pyspark, Azure Data Factory (ADF), and Python.
• Experience with Databricks as part of the data engineering process.
• Strong understanding of the P&C Insurance industry and its data requirements.
• Familiarity with HBase and Phoenix SQL for data storage solutions.
• Ability to work independently and in a team-oriented environment.
• Excellent analytical and problem-solving skills.
• Degree in Computer Science, Information Technology, or a related field.

Preferred Skills

• Knowledge of additional programming languages or data processing tools.
• Experience with cloud computing platforms and scalable data architecture.
• Familiarity with data governance and compliance practices.

Experience

• Minimum of 8 years of experience in data engineering, particularly in environments involving large datasets and complex integrations, especially within the insurance sector.

Environment

• Work will be conducted primarily during day shifts.
• Locations available: Pune and Ahmedabad.
• The position may be offered as a subcontractor or a permanent role depending on preferences.

Salary

Salary will be determined based on market standards and the candidate's experience.

Growth Opportunities

Potential career advancement opportunities within the company, including senior data engineering roles and leadership positions.

Benefits

• Comprehensive benefits package including health insurance, paid leave, and work-life balance initiatives.