Data Engineer

Company Overview

Company details are not provided, please include if available.

Job Summary

As a member of the Data team, the Data Engineer will tackle cutting-edge data engineering and distributed computing challenges. This role involves improving throughput, reducing latency, and maintaining uptime of data pipelines and web services. The engineer will focus on writing test-driven code to process terabytes of data in multi-region distributed systems.

Responsibilities

• Scale the current data pipeline to handle 5X the present capacity within one year.
• Transition from batch-oriented processing to a near-real-time processing engine.
• Build performance monitoring systems for databases, web services, and processing engines.
• Think big and execute with a milestone-based approach rather than a big-bang strategy.
• Design and code with considerations for scale, high availability, and cost efficiency.
• Mentor and review the code of fellow team members.
• Lead a micro team within the Data team and implement effective tech processes and tools.
• Own problem statements and devise solutions to address them.
• Be open to working with a polyglot tech stack.

Qualifications

• 5-7 years of proven experience in developing scalable REST/gRPC services or streaming pipelines and data-intensive applications.
• Expertise in Java programming language and frameworks.
• Hands-on experience with data modeling, database design, and performance optimization.
• Familiarity with web frameworks such as Vert.X, Spring Boot, and Quarkus is a plus.
• Experience with data processing technologies such as Kafka, Flink, and Spark is a plus.
• Knowledge of containerization tools such as Docker and Kubernetes is a plus.

Preferred Skills

• Understanding of cloud services and architectures.
• Experience in automated testing frameworks and CI/CD practices.
• Knowledge of big data technologies and frameworks.

Experience

• 5-7 years relevant experience in data engineering or related fields.

Environment

Typical work setting and location details are not provided, please include if available.

Salary

Salary details are not provided, please include if available.

Growth Opportunities

Opportunities for career advancement are not specified, please include if available.

Benefits

Benefits details are not provided, please include if available.