Staff Machine Learning Engineer

Quick Facts

• Role: Staff Machine Learning Engineer (AI/Agentic Engineering)

Description

You will own significant parts of an agent execution “harness” that enables AI agents to reason over real enterprise data, use tools, and run safely at Fortune 500 scale.

Responsibilities span orchestration, reliability at scale, observability, prompt infrastructure, evaluation frameworks, and production LLM integration.

You will also provide technical leadership through architecture decisions, code reviews, and coaching.

Responsibilities

• Design and build the agent execution harness: orchestration layer, context management, tool invocation, retries, and execution state

• Ensure reliability at scale: fault tolerance, latency, and throughput; prevent silent or unpredictable failures

• Add observability: tracing, cost attribution, and latency visibility for production behavior and early failure detection

• Build prompt infrastructure: versioning, templating, and systematic evaluation

• Design and own evaluation frameworks (unit evals, integration evals, production monitors) to measure quality and catch regressions

• Integrate and abstract over frontier LLMs: routing, fallback strategies, and production cost/latency trade-offs

• Provide technical leadership: standards, architecture decisions, code reviews, and coaching on agentic design patterns

• Define system boundaries: tool calls vs sub-agents vs hardcoded paths vs human escalation

Requirements

• 7+ years building production software systems with reliability, performance, and scalability

• Hands-on experience shipping generative AI products used by production customers

• Deep understanding of LLMs: failure modes, context constraints, and prompt design impact

• Python experience

• Prompt engineering experience: systematic design, versioning, and evaluation of prompts across model updates/A-B cycles

• Experience in eval engineering with evaluation suites designed, shipped, and used for production quality decisions

• Cost/efficiency awareness: reasoning about model routing, inference cost, and latency trade-offs

• Strong fundamentals in distributed systems, API design, and testing

• Comfort operating in fast-moving, ambiguous, startup-like AI environments

• Experience with Kubeflow and at least one of SageMaker, Vertex AI, or AzureML

• SQL experience

Benefits

• Base pay guideline: $176,100 - $308,200, plus equity (when applicable), variable/incentive compensation, and benefits

• Health plans including flexible spending accounts

• 401(k) with company match

• ESPP

• Matching donations

• Flexible time away plan and family leave programs