Senior Data Engineer

Company Overview

Not specified.

Job Summary

The Senior Data Engineer plays a critical role in designing, building, and maintaining scalable data architectures and pipelines. This position is essential for transforming raw data into actionable insights, enabling data-driven decisions across the organization. The engineer will leverage technologies such as Medallion Architecture, Microsoft Azure Synapse Analytics, and Azure Fabric to optimize data processing and integration.

Responsibilities

• Design and implement data architectures using Medallion Architecture to facilitate data ingestion, processing, and storage.
• Develop and maintain ETL processes to ensure high-quality data flow and management within the Azure Synapse environment.
• Collaborate with data scientists and analysts to understand data needs and create efficient pipelines for data retrieval and analysis.
• Optimize SQL queries and data processing workflows for performance improvements and cost reductions in Azure environments.
• Implement and monitor data governance practices to ensure compliance and integrity across datasets.
• Troubleshoot and resolve data-related issues and implement relevant solutions swiftly.

Qualifications

• Bachelor's degree in Computer Science, Data Engineering, Information Technology, or a related field.
• Proficiency in data modeling and database design, with hands-on experience in Azure Synapse Analytics.
• Familiarity with Medallion Architecture principles and implementation strategies.
• Strong programming skills in languages such as Python, SQL, or Scala.
• Experience with data pipeline orchestration tools such as Azure Data Factory or similar platforms.
• Knowledge of big data technologies including Apache Spark, Databricks, or related frameworks.
• Excellent problem-solving abilities and a passion for working with large datasets.
• Familiarity with Azure Fabric for efficient data management and integration.

Preferred Skills

• Experience with machine learning workflows or frameworks.
• Familiarity with cloud computing practices and services beyond Azure.
• Knowledge of data visualization tools like Power BI or Tableau.
• Familiarity with data security and privacy practices.

Experience

Minimum of 5 years of experience in data engineering or related fields, with a focus on building and maintaining data solutions in cloud environments.