AWS Glue Data Engineer

Job Summary

We are seeking an experienced AWS Glue Data Engineer who will utilize strong hands-on expertise in PySpark, Python, and SQL to design, develop, and optimize data pipelines and ETL (Extract, Transform, Load) workflows in a cloud-based environment. The ideal candidate will work with large datasets and AWS services to ensure efficient data transformation, integration, and delivery that aligns with the organization’s goals.

Responsibilities

• Design and implement robust data pipelines and ETL workflows using AWS Glue and related services.
• Optimize data transformations for high performance and scalability in a cloud environment.
• Collaborate with data scientists and business stakeholders to understand data requirements and deliver solutions.
• Monitor and troubleshoot data pipelines, ensuring seamless data flow and efficiency.
• Work with large datasets to perform data cleaning, integration, and transformation using PySpark and SQL.

Qualifications

• Strong experience with AWS Glue, PySpark, Python, and SQL.
• Proven ability to design and optimize data pipelines in a cloud-based environment.
• Familiarity with AWS services such as S3, Redshift, Lambda, and Glue Data Catalog.
• BS in Computer Science, Information Technology, or a related field; relevant certifications are a plus.
• Strong analytical and problem-solving skills.

Preferred Skills

• Experience with additional AWS services and tools, such as Athena and EMR.
• Familiarity with data warehousing concepts and tools.
• Knowledge of data governance and data quality principles.

Experience

• 5+ years of experience in data engineering or a related field, with a focus on AWS technologies and data pipeline development.