Data Engineer

Company Overview

N/A

Job Summary

The Data Engineer will focus on analyzing existing Talend ETL jobs and designing equivalent workflows in Apache Airflow. This role is critical for ensuring seamless data pipeline transitions and maintaining high data quality standards.

Responsibilities

• Analyze existing Talend ETL jobs to understand business logic and data flow.
• Design and implement workflows in Apache Airflow using Python.
• Develop Directed Acyclic Graphs (DAGs) for scheduling and monitoring data pipelines.
• Ensure data quality, implement error handling, and maintain logging in Airflow pipelines.
• Collaborate with data engineers, analysts, and business stakeholders to validate migrated jobs.
• Optimize the performance and scalability of Airflow workflows.
• Document migration processes, pipeline architecture, and operational procedures.
• Provide support during User Acceptance Testing (UAT) and production deployment.

Qualifications

• Strong experience with Apache Airflow (2+ years).
• Hands-on experience with Talend ETL tools (Open Studio or Talend DI).
• Proficiency in Python for scripting and pipeline development.
• Experience with SQL and working with relational databases (e.g., PostgreSQL, MySQL, Oracle).
• Familiarity with cloud platforms (AWS, GCP, or Azure) and cloud-native data services.
• Understanding of CI/CD pipelines, version control (Git), and DevOps practices.
• Knowledge of data warehousing concepts, ETL best practices, and data governance.

Preferred Skills

N/A

Experience

N/A

Environment

N/A

Salary

N/A

Growth Opportunities

N/A

Benefits

N/A