Data Engineer with Python

Job Summary

The Data Engineer will be responsible for designing, implementing, and maintaining scalable data pipelines and architectures. This role is crucial in ensuring the availability and accuracy of data for analytics and decision-making processes within the organization.

Responsibilities

• Design and develop robust data pipelines to ingest, process, and store large datasets.
• Collaborate with data scientists and analysts to understand data requirements and provide necessary data solutions.
• Optimize data storage and processing for efficient performance using best practices.
• Implement data quality checks and monitoring solutions to ensure data integrity.
• Document data engineering processes and create workflow diagrams for clarity.

Qualifications

• Bachelor's degree in Computer Science, Engineering, or a related field.
• 5+ years of experience in data engineering or similar roles.
• Proficiency in Python programming for data manipulation and analysis.
• Experience with data warehousing solutions (e.g., Amazon Redshift, Google BigQuery).
• Strong knowledge of SQL and NoSQL databases.
• Familiarity with cloud platforms (e.g., AWS, Azure, Google Cloud).
• Understanding of ETL processes and data integration techniques.
• Excellent problem-solving skills and attention to detail.

Preferred Skills

• Experience with big data technologies such as Apache Spark or Hadoop.
• Knowledge of data visualization tools (e.g., Tableau, Power BI) is a plus.
• Familiarity with machine learning principles and frameworks.

Experience

A minimum of 5 years of relevant experience in data engineering, data analysis, or a related field is preferred.

Environment

The typical work setting is in an office environment located in Bangalore, but the details regarding remote work or hybrid options are not specified.