Data Engineer

Company Overview

Not specified

Job Summary

The Data Engineer will be responsible for designing, constructing, and maintaining large-scale data processing systems. This role will work closely with data architects and data scientists to ensure efficient data flow and access throughout the organization, enabling data-driven decision-making.

Responsibilities

• Design, implement, and maintain scalable and robust data pipelines using technologies such as Hadoop, MapReduce, Spark, and Kafka or Flink.
• Develop containerized applications using Docker and manage orchestration with Elastic Kubernetes Service (EKS).
• Collaborate with cross-functional teams to understand data requirements and translate them into technical specifications.
• Monitor and optimize the performance of data systems and pipelines.
• Ensure data quality and integrity through comprehensive testing and validation processes.
• Troubleshoot and resolve issues in production data environments.

Qualifications

• Bachelor's degree in Computer Science, Engineering, or a related field.
• 6+ years of experience as a Data Engineer or in a similar role.
• Proficiency in big data technologies such as Hadoop, Spark, and data streaming platforms like Kafka or Flink.
• Experience with containerization technologies, particularly Docker.
• Familiarity with cloud services, especially EKS for Kubernetes management.
• Strong problem-solving skills and ability to work effectively in a remote environment.
• Understanding of data modeling, ETL processes, and data warehousing concepts.

Preferred Skills

• Experience with additional programming languages such as Python or Java.
• Knowledge of data visualization tools and techniques.
• Familiarity with machine learning frameworks and libraries.

Experience

• Minimum of 6 years of relevant experience in data engineering or a related field.

Environment

• Remote work setting, with flexible hours and a collaborative digital workspace.