Data Engineer

Company Overview

(Company information is not specified.)

Job Summary

The Data Engineer role is essential for developing, managing, and optimizing data pipelines and databases to ensure efficient collection, storage, and accessibility of data for analysis. This position plays a critical role in supporting data-driven decision-making and contributing to the organization's goals.

Responsibilities

• Design, build, and maintain robust ETL pipelines using Airflow and Databricks on AWS.
• Create and manage data systems and architectures using Snowflake, AWS S3, and GitLab.
• Develop data models and perform data transformations with PySpark and SQL.
• Collaborate with data analysts and stakeholders to understand data needs and ensure data accessibility.
• Monitor and optimize performance of data processes with Jenkins and troubleshoot data-related issues.
• Implement data visualization solutions using Looker to present data insights effectively.

Qualifications

• Proficiency in ETL processes and data engineering concepts.
• Strong experience with Databricks on AWS.
• Extensive knowledge of Airflow for scheduling and managing workflows.
• In-depth understanding of Snowflake for data warehousing.
• Proficient in programming and scripting with PySpark and SQL.
• Familiarity with version control systems, particularly GitLab.
• Knowledge of cloud storage solutions, specifically AWS S3.
• Strong analytical skills with the ability to interpret and leverage data effectively.

Preferred Skills

• Experience with data visualization tools, particularly Looker.
• Familiarity with continuous integration and deployment tools such as Jenkins.
• Understanding of big data technologies and frameworks.

Experience

(Years of experience and types of relevant experience needed are not specified.)

Environment

(Typical work setting, location, and any physical or environmental conditions relevant to the job are not specified.)

Salary

(Estimated salary range is not specified.)

Growth Opportunities

(Potential career advancement opportunities within the company are not specified.)

Benefits

(Offered benefits, such as insurance, paid leave, work policies, etc. are not specified.)