Senior Staff Software Engineer, AI/ML Inference Engineering

Quick Facts

• Role: Senior Staff Software Engineer, AI/ML Inference Engineering
• Focus: AI inference platform architecture, optimization, and production operations

Description

Lead the architecture, optimization, and operational excellence of an AI inference platform. Own the end-to-end inference lifecycle—from onboarding cutting-edge models to deploying optimized inference engines in production across multiple backends on Kubernetes. Architect an intelligent gateway layer using Envoy and ext_proc for access control, observability, and advanced request routing, with measurable performance impact.

Responsibilities

• Own end-to-end inference lifecycle (model onboarding through production deployment)
• Design and optimize inference engines for multiple backends on Kubernetes
• Architect an intelligent gateway layer for access control, observability, and request routing
• Deliver pragmatic engineering with performance measurement and operational excellence

Requirements

• 8+ years of backend software engineering
• 4+ years designing and operating distributed systems
• Expertise in Go or Python, OOP, design patterns, and time/space-efficient algorithms
• Model-serving and runtime experience (vLLM, TensorRT-LLM, Triton Inference Server)
• Exposure to LLM APIs or ML-serving systems (or ability to ramp up quickly)
• Knowledge of core AI/ML techniques and algorithms
• Knowledge of unit testing, profiling, and code tuning
• Bachelor’s degree in CS/software engineering (or closely related field); master’s can offset up to 1 year; equivalent practical experience accepted

Benefits

• Base pay range: $181,200–$317,100 (plus equity when applicable, variable/incentive compensation)
• Health plans, flexible spending accounts, 401(k) with match, ESPP, matching donations, flexible time away, and family leave programs
• Compensation varies by geographic location and may change based on work location