LLMOps / ML Infrastructure Software Engineer

Quick Facts

Role: LLMOps / ML Infrastructure Software Engineer

Description

Build and optimize LLM serving infrastructure, including GPU orchestration, scaling, and performance/cost optimization. Own monitoring and observability for model serving, plus model governance infrastructure and automated deployments. Troubleshoot production issues and collaborate with Applied/Foundational AI and SRE/Platform teams.

Responsibilities

• Build and optimize LLM serving infrastructure using vLLM, SGLang, or equivalent inference engines.
• Manage GPU infrastructure: orchestration, scaling, load balancing, capacity planning, and high availability.
• Optimize inference performance and costs via batching, quantization, caching, and improved GPU utilization.
• Implement monitoring and observability for serving: infrastructure health, latency, errors, and output degradation.
• Support enterprise deployments into on-premise, private, and air-gapped environments with security and reliability.
• Develop model governance infrastructure: model registries, versioning, LLM gateways, logging, and auditing.
• Build CI/CD pipelines and infrastructure-as-code for automated, reliable deployments.
• Troubleshoot production issues and collaborate with Applied AI, Foundational AI, and SRE/Platform teams.

Requirements

• Strong background in Infrastructure Engineering, Platform Engineering, MLOps, or LLMOps; experience running ML/LLM workloads in production.
• Hands-on experience with LLM serving technologies such as vLLM, SGLang, or similar.
• Practical knowledge of GPU infrastructure orchestration, scaling, and inference optimization.
• Production experience with Kubernetes, Docker, Terraform, and CI/CD.
• Proficiency in Python and/or Go.
• Experience with monitoring, observability, and troubleshooting production systems.
• Understanding of inference cost, latency, throughput, and performance trade-offs.
• Strong ownership mindset and ability to collaborate across engineering teams.

Benefits

• Work on cutting-edge enterprise AI with real-world applications and major enterprise customers.
• Solve challenging engineering problems across GPU infrastructure, inference performance, and secure deployments.
• Take ownership and impact architecture, reliability, and cost-efficiency of the AI platform.
• Collaborate with AI and engineering teams in a fast-growing, ambitious environment.
• Remote flexibility.