Software Engineer, Inference AI/ML

Quick Facts

Join the Inference team to ship production features that improve latency, reliability, and cost for model serving on a GPU platform.

Base salary range: $92,000 to $135,000 (starting salary based on knowledge, skills, experience, and market location). Total rewards may include discretionary bonus, equity awards, and eligibility-based benefits.

Description

Join the Inference team to ship production features that improve latency, reliability, and cost for model serving on a GPU platform.

Implement well-scoped features and fixes in Python/Go/C++ for model-serving services such as Triton, vLLM, TensorRT-LLM, and Ray Serve.

Write tests, code comments, and short design docs; participate in code reviews. Add basic metrics and dashboards; assist with alarms and runbooks. Follow on-call runbooks and learn incident response in a guided rotation. Contribute to performance experiments like request batching, concurrency, and caching with guidance.

Requirements

• BS/MS in CS, EE, or related field, or equivalent practical experience
• Foundations in data structures, algorithms, and networked services
• Experience with Python or Go (C++ a plus)
• Linux fundamentals
• Git/CI basics
• Exposure to containers and Kubernetes (coursework or projects welcome)
• Curiosity about GPU inference concepts (micro-batching, KV cache, streaming)

Preferred

• Internship or project that deployed a microservice or ML inference demo
• Coursework/research with PyTorch or TensorFlow (simple CUDA projects a plus)
• Familiarity with Grafana/Prometheus/OpenTelemetry or similar tooling

Benefits

• Medical, dental, and vision insurance (100% paid)
• Company-paid life insurance and voluntary supplemental life insurance
• Short and long-term disability insurance
• Flexible Spending Account; Health Savings Account
• Tuition Reimbursement
• Employee Stock Purchase Program (ESPP)
• Mental wellness benefits
• Family-forming support; paid parental leave
• Flexible, full-service childcare support
• 401(k) with generous employer match
• Flexible PTO
• Catered lunch each day; casual work environment
• Work culture focused on innovative disruption

Qualifications

• Export control compliance: position requires access to export controlled information; applicant must be a U.S. person, eligible to access without required export authorization, or eligible and reasonably likely to obtain required authorization.