Senior Staff Software Engineer (AI Agents & Evaluation Architecture)

Quick Facts

• Role: Senior Staff Software Engineer (AI Agents & Evaluation Architecture)

Description

Lead the technical direction for model-driven capability across HR domain agentic and conversational experiences. Own the architecture, standards, and evaluation infrastructure that help multiple teams ship these experiences safely.

This role is not ML research. Focus on engineering load-bearing natural-language logic, enforcing probabilistic correctness via large-scale evaluation, and delivering production-grade quality and safety.

Responsibilities

• Define AI architecture: agent decomposition/composition, where reasoning happens, context assembly and bounds, tool exposure, and autonomy delegation.

• Own multi-release model selection and migration, orchestration strategy, build-versus-adopt decisions, and cost/latency/quality tradeoffs.

• Set the autonomy boundary for HR actions, including mechanisms that require human decision points for sensitive changes (pay, employment status, restricted records, employee relations).

• Design a shared instruction and tool description surface as a versioned, upgrade-safe contract with extension points, deprecation paths, and compatibility guarantees.

• Build evaluation infrastructure: golden datasets, multi-turn suites, judge calibration, CI gates, drift detection, and HR-specific failure-class coverage (access violations, cross-scope leakage, jurisdiction/policy variants).

• Ensure production quality and safety via observability for containment, hallucination rate, tool-selection error, unsafe actions, and injection vectors.

• Establish AI-assisted engineering standards: convert ambiguous problems into testable specs; define accountable delivery expectations with review/verification harnesses.

• Perform hands-on integration and risky migration/prototyping to resolve architectural gaps.

• Partner across product, engineering, and design to translate business needs into robust technical designs.

Requirements

• 7+ years of software engineering with a record of technical direction adopted beyond your immediate team.

• Direct experience authoring agentic instructions/prompts, designing autonomous workflows, and building evaluation to verify them.

• Production experience with LLM APIs, retrieval-grounded features, agent orchestration, tool/function calling, and structured output enforcement, including at least one production migration.

• Entitlement-aware data handling: per-user access enforcement in retrieval and tool layers, role-based scope separation, and audit-aware handling of restricted records.

• Evaluation experience used by other engineers: dataset curation, judge calibration, CI gating, drift detection.

• Strong system design, API, data modeling, and testing skills across front-end, server-side, and relational data.

• Accountable use of AI coding assistants and agents.

• On-call and incident-command experience for customer-facing systems with ownership of systemic fixes.

• Experience with Java, JavaScript/TypeScript, and React; ability to learn new tools and frameworks quickly.

• Bachelor’s degree in CS/software engineering or related field, or equivalent practical experience.

Benefits

• Base pay guideline (location-dependent) plus equity (when applicable), variable/incentive compensation, and benefits.

• Health plans (including flexible spending accounts), 401(k) with company match, ESPP, matching donations, flexible time away, and family leave programs.