Data Engineer (Python, FastAPI, Kubernetes, Azure)

Company Overview

Not specified.

Job Summary

The Data Engineer will be responsible for designing, building, and optimizing scalable data pipelines and microservices. This role is crucial in enhancing the organization's data infrastructure and delivering high-performance systems that handle high concurrency workloads.

Responsibilities

• Design, build, and optimize scalable data pipelines and microservices.
• Deploy and manage APIs in Kubernetes (K8s) ensuring high availability.
• Orchestrate I/O-heavy API calls for high concurrency workloads.
• Work with queues, databases, and event-driven systems for real-time data processing.
• Implement and manage Azure cloud data solutions including Data Factory, Event Hub, and Key Vault.
• Collaborate with cross-functional teams to ensure high performance and reliability of systems.

Qualifications

• 4-6 years of experience in Data Engineering, preferably 3-5 years in relevant roles.
• Strong expertise in FastAPI, Socket.IO, Python, and Kubernetes.
• Experience in production deployments with high concurrency (10k+ users).
• Hands-on experience with queues such as Celery, RabbitMQ, or AWS SQS.
• Strong knowledge of databases, including PostgreSQL, MongoDB, Databricks, and Iceberg.
• Proficient in working with web sockets and async programming using async.io.
• Strong hands-on experience with Azure Cloud services, including:
  • Azure Data Factory
  • SQL/NoSQL Databases
  • Event Hub
  • Storage
  • Key Vault
• Strong experience in production-grade Python coding.
• Familiar with Microservices and Containers.
• Proficient in SQL.

Preferred Skills

• Exposure to GoLang (optional).
• Knowledge of API and automation tools like Logic Apps.
• Familiarity with CI/CD pipelines, DevOps practices, and Azure monitoring tools.

Experience

• 4–6 years of relevant experience required, with a preference for candidates having 3–5 years specifically in Data Engineering.

Environment

• Hybrid work setting, requiring 3 days from the office located in Gurgaon.

Salary

Not specified.

Growth Opportunities

Not specified.

Benefits

Not specified.