Sr. Data Engineer- H

Company Overview

Company details were not provided.

Job Summary

The Sr. Data Engineer will focus on understanding functional requirements and translating them into technical specifications. This role requires working with various data frameworks and components to handle data ingestion and processing, primarily using Azure Databricks. The engineer will collaborate closely with clients and architecture teams to ensure solutions meet business needs and project plans.

Responsibilities

• Understand the functional requirements and convert them into technical specifications.
• Create frameworks and reusable components for handling data ingestion and processing in Databricks.
• Address complex business problems and manage large volume data sets effectively on the Azure Databricks platform.
• Collaborate directly with the Client Data Engineering Manager to translate business requirements into technical specifications.
• Work with the customer architecture team on code design and perform code reviews.
• Participate in sprint cycles with other team members to deliver quality outputs as per project plans.

Qualifications

• Technical Skills:

  • Hands-on design and development experience on the Databricks platform.
  • Proficient in Scala programming.
  • Experience with Azure ecosystems such as Azure Data Lake Storage (ADLS) Gen2, Azure Key Vault, Azure Data Factory (ADF), etc.
  • Expertise in Spark optimization for managing large data sets and implementing complex business logic in Spark/Scala.
  • Previous experience in designing ETL (Extract, Transform, Load) and ELT (Extract, Load, Transform) pipelines in Databricks.

• Soft Skills:

  • Strong communication skills to effectively interact with team members and stakeholders.

Preferred Skills

• Additional experience with data processing frameworks and cloud-based data solutions would be an asset.

Experience

Years of experience and specific types of relevant experience needed were not specified.

Environment

Typical work setting and location details were not provided, though it is a remote position.

Salary

Salary details were not provided.

Growth Opportunities

Potential career advancement opportunities within the company were not specified.

Benefits

Offered benefits were not provided.