Software Engineer, Model Routing & Inference

Quick Facts

• Role: Software Engineer, Model Routing & Inference

Description

Build and operate an inference platform that powers every AI interaction in the product. You will own the full inference path to make AI interactions faster, more reliable, and more cost-effective at scale. The work includes designing an inference gateway across providers, plus failover and routing controls to prevent user-visible degradation during outages and traffic spikes.

Responsibilities

• Build and evolve an inference gateway as a single abstraction over provider API semantics, enabling model onboarding as a config change

• Design intelligent cross-provider failover so provider outages don’t cause user-visible degradation

• Design routing backpressure and admission control so traffic spikes don’t cascade into providers

Requirements

• Experience building high-throughput, low-latency distributed systems (inference serving, traffic routing, or real-time data pipelines)

• Bachelor’s degree in computer science, engineering, math, or scientific discipline OR 2+ years of professional software engineering experience in lieu of a degree

• If there is a fit: participate in 2–3 short technical interviews, then an onsite interview with a small project and team meeting

Benefits

• Annual salary: $140,000–$225,000 USD (level and base salary determined case-by-case)

• Equity

• Comprehensive medical, vision, and dental coverage

• 401(k)

• Short- and long-term disability insurance

• Life insurance

• Various discounts and perks