Data Engineer

Company Overview

Company information is not specified.

Job Summary

The Data Engineer will be responsible for designing, developing, and optimizing scalable data pipelines. This role is crucial for ensuring the integrity and security of data throughout its lifecycle while collaborating with cross-functional teams to deliver high-quality data solutions.

Responsibilities

• Design, develop, and optimize scalable data pipelines using PySpark and Databricks.
• Implement and manage data workflows in Azure Data Factory (ADF).
• Work with large-scale Big Data technologies including Hadoop, Hive, and related ecosystems.
• Collaborate with cross-functional teams to understand data requirements and deliver high-quality solutions.
• Ensure data quality, integrity, and security across all stages of the data lifecycle.
• Troubleshoot and resolve data-related issues in a timely manner.

Qualifications

• Strong hands-on experience with PySpark, Databricks, and ADF.
• Proficiency in Big Data technologies: Hadoop, Hive.
• Solid understanding of data architecture, ETL processes, and performance optimization.
• Experience working in hybrid environments and managing distributed data systems.
• Excellent problem-solving and communication skills.
• Relevant educational qualifications such as BSc in Computer Science, Information Technology, or related fields.

Preferred Skills

• Knowledge of additional data processing frameworks or tools.
• Familiarity with cloud services beyond Azure, such as AWS or Google Cloud.

Experience

• Minimum of 3-5 years of relevant experience in data engineering or related fields.

Environment

Work setting and location details are not specified.

Salary

Salary information is not specified.

Growth Opportunities

Career advancement opportunities are not specified.

Benefits

Benefits information is not specified.