Python PySpark Developer

Company Overview

Company: Not specified
Industry: Not specified
Culture: Not specified

Job Summary

The Python PySpark Developer will be responsible for designing and implementing data processing applications using Python and PySpark. This role involves collaborating with data engineers and analysts to optimize data workflows and ensure efficient data processing capabilities, contributing towards the organization’s overarching goals of data management and analytics.

Responsibilities

• Develop data pipelines and workflows using Python and PySpark for large-scale data processing.
• Collaborate with data engineers to ensure data is processed efficiently and is accessible for analytics purposes.
• Perform data analysis and transformation tasks to meet project requirements.
• Optimize existing PySpark jobs for performance enhancements and cost reduction.
• Troubleshoot and resolve issues related to data processing jobs.
• Document all development and processes for future reference and team collaboration.

Qualifications

• Educational Background: Bachelor's degree in Computer Science, Information Technology, or a related field.
• Experience: A minimum of 5 years of experience in Python and PySpark development.
• Technical Skills: Proficiency in Python, experience with PySpark, knowledge of Hadoop ecosystems.
• Soft Skills: Strong analytical thinking, problem-solving abilities, and effective communication skills.
• Certifications: Relevant certifications in data engineering or cloud technologies are a plus.

Preferred Skills

• Familiarity with data visualization tools such as Tableau or Power BI.
• Experience with cloud services like AWS or Azure.
• Knowledge of SQL and relational databases.

Experience

• At least 5 years of relevant experience in Python and PySpark in a data engineering role.

Environment

• Typical work setting is in an office environment located in Hyderabad. Opportunities for remote working may be available.

Salary

• Not specified.

Growth Opportunities

• Career advancement opportunities may arise as the company expands, including leadership roles in data engineering and analytics.

Benefits

• Not specified.