Data Engineer

Job Summary

The Data Engineer will play a crucial role in designing, developing, and maintaining ETL processes. This position aims to ensure the efficient ingestion, transformation, and storage of data, facilitating insights that drive business decisions.

Responsibilities

• Design, develop, and maintain ETL processes using Azure Data Factory (ADF), SQL Server Integration Services (SSIS), and Databricks.
• Collaborate with data architects, analysts, and other stakeholders to understand data requirements and translate them into technical solutions.
• Build and optimize data pipelines to ensure efficient data ingestion, transformation, and storage.
• Monitor and troubleshoot data integration processes to ensure data quality and performance.
• Implement data governance and security best practices within Azure and ETL frameworks.
• Conduct data profiling and analysis to identify trends and anomalies.
• Document processes, data flows, and technical specifications for reference and compliance.
• Stay current with Azure updates and emerging technologies to recommend enhancements.

Qualifications

• Education: Bachelor's degree in Computer Science, Information Technology, or a related field.
• Experience: 5+ years of experience in data engineering or related roles, with a focus on Azure and ETL technologies.
• Proficiency in Azure Data Factory, Azure Databricks, and SQL Server Integration Services (SSIS).
• Strong understanding of ETL concepts and best practices.
• Experience with data modeling, data warehousing, and big data technologies.
• Knowledge of programming languages such as SQL, Python, or Spark.
• Familiarity with Azure cloud services (e.g., Azure SQL Database, Azure Blob Storage).
• Excellent problem-solving skills and attention to detail.
• Strong communication and collaboration skills.

Preferred Skills

• Experience with data visualization tools (e.g., Power BI, Tableau).
• Familiarity with data governance frameworks and tools.
• Knowledge of machine learning and AI principles in data processing.
• Azure certifications (e.g., Azure Data Engineer Associate) are a plus.

Experience

5+ years of experience in data engineering or related roles, with a focus on Azure and ETL technologies.