Founding Inference Engineer

Job Summary

We are seeking a Founding Inference Engineer to take full ownership of the infrastructure and performance of our model-serving stack. This role is critical in transforming open-weight models into reliable, cost-efficient production services tailored for enterprise customers. Positioned at the intersection of GPU infrastructure, model serving, systems performance, and production reliability, you will be responsible for uptime, service-level agreements (SLAs), GPU orchestration, serving efficiency, and optimizing the cost-per-token economics of every GPU deployed. You will lead the scaling efforts, capacity planning, reliability practices, and performance systems necessary to consistently and efficiently serve real enterprise workloads.

Responsibilities

• Own the end-to-end production inference stack, including GPU orchestration, model deployment, serving reliability, and performance.
• Build, operate, and optimize model serving using vLLM and related inference technologies.
• Drive cost-per-token optimization by maximizing the number of useful tokens delivered per unit of compute cost.
• Profile and improve throughput, latency, concurrency, memory utilization, and GPU efficiency across production workloads.
• Develop capacity plans based on traffic patterns, workload characteristics, latency requirements, utilization, growth projections, and failure scenarios.
• Ensure model uptime, meet SLAs, maintain service reliability, and implement graceful failure handling for enterprise workloads.
• Identify and resolve bottlenecks at the systems level across GPU compute, memory, networking, and serving infrastructure.
• Build benchmarking and performance-testing systems that reflect real customer workloads rather than isolated peak-throughput metrics.
• Establish comprehensive observability across the inference layer, including metrics, logging, monitoring, alerting, and cost-per-token tracking.
• Collaborate closely with the AI/ML team to efficiently bring open-weight and post-trained models into production.
• Partner with the Platform Engineering team to integrate inference services with the model gateway, evaluations, usage tracking, and customer-facing systems.
• Lead production follow-through by diagnosing incidents, conducting performance investigations, deploying improvements safely, and continuously raising efficiency standards.

Qualifications

• Deep hands-on experience in GPU inference, model serving, systems performance, or infrastructure engineering.
• Strong expertise with vLLM or comparable large language model (LLM) serving and inference frameworks.
• In-depth understanding of GPU architecture, memory behavior, concurrency, scheduling, and performance optimization.
• Proven experience operating production systems with stringent uptime, latency, throughput, and reliability requirements.
• Ability to translate benchmark results into realistic capacity plans that consider workload variation, demand growth, and failure modes.
• Skilled in profiling and optimizing performance across the full stack, including models, runtimes, GPUs, networking, and infrastructure.
• Strong knowledge of distributed systems, cloud infrastructure, observability, and incident response.
• Experience designing GPU orchestration, scheduling, autoscaling, or resource-utilization systems is highly valuable.
• Ability to balance performance, reliability, engineering complexity, and cost when making infrastructure decisions.
• Comfortable working closely with AI/ML, platform, and product teams to solve ambiguous, cross-functional problems.
• Strong ownership mindset with a willingness to remain hands-on in a fast-moving, early-stage environment.

Preferred Skills

• Experience building, optimizing, or operating GPU-based infrastructure, large-scale inference systems, machine learning platforms, high-throughput distributed services, or performance-critical backend systems.
• Familiarity with capacity planning that accounts for utilization, concurrency, workload mix, reliability requirements, scaling needs, and failure scenarios.
• Ability to contribute beyond individual optimizations by owning foundational layers of the company’s product infrastructure.

Experience

• Experience working on systems where performance, latency, availability, and infrastructure economics directly impact product and customer experience.
• Prior involvement in production environments requiring high reliability and cost-effective GPU resource management.

Environment

• This role involves working in a fast-paced, early-stage startup environment.
• Collaboration with cross-functional teams including AI/ML, platform engineering, and product teams.
• The position requires a hands-on approach to infrastructure ownership and continuous improvement.

Why This Role

• Own the infrastructure that directly defines the company’s cost-per-token advantage.
• Work on the core serving layer powering real enterprise AI workloads in India.
• Solve challenging problems spanning GPU systems, vLLM, throughput, reliability, capacity planning, and infrastructure economics.
• Work directly with the company founders and be part of the founding team.
• Shape the company’s inference architecture, operating standards, and performance culture from the ground up.
• Build the foundational infrastructure layer for the company’s ambition to become India’s largest AI infrastructure company.