Senior Staff Software Engineer, AI/ML Inference

Quick Facts

• Role: Senior Staff Software Engineer, AI/ML Inference

Description

You will lead the architecture, optimization, and operational excellence of an AI inference platform, owning the end-to-end inference lifecycle—from onboarding cutting-edge models to deploying optimized inference engines in production. You will architect an intelligent gateway layer using Envoy and ext_proc to enforce access control, provide observability, and enable advanced request routing across multiple backends on Kubernetes.

Responsibilities

• Own end-to-end inference lifecycle: model onboarding to production deployment of optimized inference engines
• Deploy and run inference on Kubernetes across multiple backends
• Architect an intelligent gateway layer with Envoy and ext_proc for access control, observability, and request routing
• Improve performance through pragmatic engineering and measurable impact
• Contribute to foundations, research/experimentation, and de-risking of AI technologies for future work experiences

Requirements

• 8+ years of backend software engineering experience
• 4+ years of designing and operating distributed systems
• Bachelor’s degree in computer science, software engineering, or a closely related field (a master’s may offset up to one year; equivalent practical experience considered)
• Expertise in Go or Python, object-oriented programming, design patterns, and time/space-efficient algorithms
• Model-serving/runtime experience such as vLLM, TensorRT-LLM, or Triton Inference Server
• Exposure to LLM APIs or ML-serving systems (or ability to ramp up quickly)
• Expertise in efficient, modular, quality software engineering
• Knowledge of core AI/ML techniques and algorithms
• Knowledge of unit testing, profiling, and code tuning
• Proficiency using AI coding assistants with rigorous review (security anti-patterns and edge cases)
• AI-assisted debugging and trace analysis with understanding of prompt data privacy rules

Benefits

• Health plans, including flexible spending accounts
• 401(k) Plan with company match
• ESPP
• Matching donations
• Flexible time away plan and family leave programs
• Base pay guideline: $201,300 - $352,300 plus equity (when applicable) and variable/incentive compensation