Senior Director, Technical Product Management — AI Quality & Evaluation

Quick Facts

• Role: Hands-on technical product leadership for AI quality and evaluation (Agentforce)
• Focus: Define evaluation/quality standards and lead customer-impact investigations and experiments

Description

Lead product strategy and hands-on execution for AI quality and evaluation, helping turn frontier agent capabilities into customer-reliable products. You will establish how AI quality is measured, run customer investigations and experiments to validate improvements, and deliver focused feature capabilities from early learning through delivery. Work closely with AI researchers, applied scientists, engineers, and customers to design credible evaluations and drive measurable outcomes.

Responsibilities

• Set a coherent product strategy and roadmap for AI quality and evaluation tied to customer needs and measurable business outcomes
• Establish shared quality definitions, evaluation requirements, coverage priorities, and release criteria across research, engineering, and product partners
• Make explicit tradeoffs across customer quality improvements, evaluation capabilities, and feature investments
• Lead customer investigations and experiments (e.g., diagnose incorrect answers or tool workflow failures; define expected behavior; validate customer impact)
• Build an operating approach connecting customer feedback, evaluation evidence, experiments, releases, and learning across teams
• Partner to design credible evaluations using AI/ML fundamentals (training/test separation, overfitting control, representative coverage, independent validation)
• Prototype customer-facing experiences that make quality gaps clear and help customers verify changes
• Coach junior designers and technical program managers in problem framing, experimentation, and delivery
• Communicate competitive insight: differentiation, limitations, and investment choices

Requirements

• Record of building AI/ML products and delivering measurable customer outcomes with increasing ownership and impact
• Deep hands-on experience building AI agents and evaluation capabilities with AI researchers, applied scientists, or AI engineers
• Strong applied AI/ML judgment across model behavior, prompting, retrieval, and agent/tool workflows and related tradeoffs
• Hands-on ability to inspect data and traces, prototype with coding agents, and reason about experiments, overfitting, training/test splits, edge cases, and uncertainty
• Ability to frame customer problems, test assumptions, and change direction based on evidence
• Experience establishing shared standards and resolving cross-team disagreements while maintaining accountability and delivery momentum
• Evidence of coaching and mentoring colleagues and delivering independently in a fast-moving environment
• Clear communication with customers, technical teams, and senior leaders, including honesty about what is known vs. uncertain

Benefits

• Comprehensive benefits and resources supporting work-life balance (time off, medical/dental/vision, mental health support, paid parental leave, life and disability insurance, 401(k), and employee stock purchasing program)
• Equal opportunity employer; assessments based on merit, competence, and qualifications
• Salary range varies by location, with base salary typical ranges provided in the posting (excluding bonus/equity/benefits)