Senior Data Engineer

Company Overview

Company details are not specified.

Job Summary

The Senior Data Engineer will design and implement high-performance, scalable, and optimized data solutions for large enterprise-wide data mining and processing. This role focuses on the development of data flows and the deployment and implementation of Big Data Platforms, contributing significantly to the organization’s data strategy.

Responsibilities

• Design and propose end-to-end data pipelines for data projects, covering ingestion, storage, and consumption, as well as databases to support web-based applications.
• Develop and implement data warehouses and data marts to serve data consumers efficiently.
• Execute implementations from design stages to operationalization, ensuring effective maintenance post-implementation.
• Perform database design and modeling to organize data at macro and micro levels and provide logical data models for consumers.
• Conduct database performance tuning and manage data lifecycles to enhance efficiency.
• Assist in support and enhancements of existing data pipelines and databases.

Qualifications

• 6-10 years of total experience working with data integration teams.
• 3+ years of experience developing data pipelines within an Apache Spark environment, preferably using Databricks.
• 2+ years of active work specifically within Databricks, showcasing in-depth experience.
• 2+ years of experience in working with data warehouses and familiarity with data warehouse modeling techniques.
• Strong knowledge of PySpark, Python, and SQL, along with distributed computing principles.
• Proficient in data modeling, database technologies, and data warehousing concepts.
• Experience with ETL/ELT processes, including design and implementation via tools like SSIS or similar.
• Familiar with cloud platforms such as AWS or Azure and big data technologies like Hadoop and Spark.
• Expertise in complex SQL query performance tuning and overall database performance tuning.

Preferred Skills

• Experience developing data solutions utilizing native IaaS and PaaS solutions on AWS (e.g., Redshift, RDS, S3) is considered advantageous.

Experience

• Required years of experience: 6-10 years of relevant experience in data engineering and integration.

Environment

Work setting, location (remote, in-office, hybrid), and physical conditions are not specified.

Salary

Salary details are not specified.

Growth Opportunities

Opportunities for career advancement within the company are not specified.

Benefits

Details regarding offered benefits are not specified.