Data Engineer (Microsoft Fabric & PySpark)

Job Summary

The Data Engineer will play a critical role in designing, building, and managing scalable data pipelines. This position requires a strong foundation in Microsoft Fabric and PySpark to ensure the development and maintenance of reliable data solutions that meet business needs while maintaining data quality and security.

Responsibilities

• Build and manage scalable data pipelines using Microsoft Fabric and PySpark.
• Work collaboratively with business teams to deliver reliable data solutions tailored to meet their requirements.
• Ensure data quality, performance, and security across all workflows.
• Develop and implement ETL (Extract, Transform, Load) and ELT (Extract, Load, Transform) processes for efficient data integration.

Qualifications

• 5+ years of experience in Data Engineering or related fields.
• Strong hands-on experience in Microsoft Fabric.
• Expertise in PySpark for data processing and analytics.
• Proficient in SQL for querying and managing databases.
• Good knowledge of data pipelines and workflow orchestration.
• Experience with ETL/ELT processes and data integration methodologies.
• Strong analytical and problem-solving skills.
• Excellent communication and collaboration abilities.

Preferred Skills

• Familiarity with cloud technologies and data warehousing solutions.
• Knowledge of data visualization tools and techniques.
• Experience with big data technologies, such as Hadoop or Spark.

Experience

• Minimum of 5 years of relevant experience in data engineering roles, focusing on Microsoft Fabric and PySpark.