Python Developer

Company Overview

Not specified.

Job Summary

The Python Developer will be responsible for designing, developing, and maintaining data pipelines and Extract, Transform, Load (ETL) processes using AWS services, primarily AWS Glue and Amazon S3. This role requires a focus on data management, analysis, and collaboration with various stakeholders to ensure efficient data handling and integrity.

Responsibilities

• Data Pipeline Design and Development: Design, build, and maintain scalable and efficient data pipelines using AWS Glue, integrating with services like Lambda, EMR, Redshift, Kinesis, and Athena.
• ETL Job Creation and Management: Develop and optimize AWS Glue ETL jobs using Spark, Python, or Scala to clean, transform, and enrich raw data.
• Data Cataloging and Discovery: Utilize AWS Glue Data Catalog and crawlers to discover, catalog, and manage metadata for data sources stored in Amazon S3.
• Amazon S3 Data Management: Manage data storage within Amazon S3, implement storage classes, lifecycle policies, and ensure security and accessibility.
• Data Quality and Validation: Implement quality checks and validation rules in the ETL process to ensure data reliability and accuracy.
• Monitoring and Troubleshooting: Monitor the performance and health of Glue jobs and data pipelines, troubleshoot issues, and optimize for efficiency and cost-effectiveness.

Qualifications

• Required Skills:

  • Proficient in Python, SQL, and knowledge of AWS services, especially AWS Glue and Amazon S3.
  • Understanding of data warehousing concepts and data lake architectures.
  • Strong SQL skills for data manipulation and analysis.
  • Analytical and problem-solving capabilities.
  • Effective communication skills for collaboration and technical explanations.

• Educational Qualifications:

  • A degree in Computer Science, Information Technology, or a related field is preferred.

Preferred Skills

• Experience with additional AWS services such as AWS Glue, Scala, and Spark would be advantageous.

Experience

• Total Experience Required: 6-8 years in relevant fields, focusing on Python development, AWS technologies, and data engineering.

Environment

• Work Setting: Hybrid model - Work From Office/Home.
• Working Days: Monday to Friday.
• Location Preference: Any Wipro location.

Salary

• Max Working Order Rate: 230,000 per month.

Growth Opportunities

Not specified.

Benefits

Not specified.