Senior AI Behavioral Scientist & Product Analyst

Quick Facts

• Role: Senior AI Behavioral Scientist & Product Analyst

• Focus: Evaluate and optimize AI agents and autonomous systems for usability and systems performance

Description

You will assess how AI agents perform in real-world contexts by applying HCI and behavioral science frameworks alongside quantitative analytics. You will own the instrumentation and reporting that demonstrates Agent ONE is working, adopted, and valued by customers, and link outcomes to retention and revenue. You will translate behavioral and performance findings into product recommendations and communicate impact through dashboards and narratives.

Responsibilities

• Design and conduct behavioral studies covering usability, task completion, error recovery, and user satisfaction

• Use HCI and behavioral science to identify friction points; analyze reliability, decision transparency, and failure modes

• Build user analytics frameworks and define KPIs for success, trust, task completion, and cognitive load

• Instrument user sessions to collect behavioral and interaction data

• Conduct statistical analysis, A/B testing, and cohort analysis to measure impact and validate improvements

• Evaluate agents across accuracy, latency, cost efficiency, scalability, and robustness; communicate trade-offs

• Identify bottlenecks and systemic issues impacting user experience across the interaction lifecycle

• Translate findings into roadmap priorities; partner with Product, Engineering, and UX Research

• Create dashboards, reports, and narratives for stakeholders and leadership

Requirements

• Prior experience evaluating AI systems, ML models, chatbots, or autonomous agents

• 4+ years of experience in user analytics, product analytics, behavioral research, or HCI research

• Strong HCI and behavioral science fundamentals (cognitive load, mental models, task completion, user trust)

• Proficiency with quantitative analytics: SQL and Python or R; familiarity with A/B testing and experimental design

• Experience designing and instrumenting analytics tracking and interpreting complex datasets

• Systems-level thinking about product performance and interacting variables affecting user outcomes

• Excellent communication skills to translate technical findings for non-technical stakeholders

Benefits

• Equal opportunity employer and fair chance hiring approach with background checks after a conditional offer