Staff AI Engineer (Agentic AI Harness & Quality)

Quick Facts

• Role: Staff AI Engineer — Agentic AI Harness & Quality

• Focus: agentic AI harness engineering, evaluation, and production quality for enterprise agents

Description

You will help expand an enterprise assistant platform’s agentic AI capabilities by building and maintaining an eval-driven, data-centric agent harness. The work spans tuning and evaluation of agents, improving end-to-end performance (accuracy, latency, reliability, and capability), and reducing agent misbehavior. You’ll also collaborate with web and chat platform teams to turn harness investments into strong user experiences.

Responsibilities

• Apply software engineering and compound AI system engineering to deliver customer value

• Drive agent behavior quality: increase goal success rate and decrease misbehavior in real customer environments

• Build agent harness components such as cognitive architecture, eval benchmark creation, context engineering, generative UI, and dynamic agent orchestration

• Support multimodal I/O (image, voice), multilinguality, and conversational memory management

• Improve grounding and verifiability for generated text and focus on deployment safety

• Use ML/LLM fundamentals to improve harness components; evaluate via small experiments and productionize at scale

• Read and apply latest AI research and open-source models

• Partner with platform teams to convert harness work into production user experiences

Requirements

• Production-quality, fully unit-tested code and rigorously-evaluated updates to models, prompts, context curation, or tunable components

• Strong LLM and agent evaluation fundamentals for action trajectories, generated text, and non-uniform sampling regimes

• High attention to detail and evaluation dataset data quality

• Thoughtfulness about API contracts and implications of exposing features to stakeholders/end users

• Ability to develop on a Mac and code in Python and/or Golang (with help from coding agents)

• Familiarity with industry standard agent architectures and LLMs

• Ability to catch subtle miscommunication bugs in English-language agent instructions and fix them with precise writing and evals

• High ownership, fast execution, curiosity, and a continuous improvement mindset

Benefits

• Base pay guideline: $176,100–$308,200 plus equity (when applicable) and variable/incentive compensation

• Health plans, flexible spending accounts, 401(k) with company match, ESPP

• Matching donations, flexible time away plan, and family leave programs