Principal UX Researcher

Quick Facts

• Role: Principal UX Researcher

Description

Lead human-centered approaches for evaluating AI model performance with mixed-methods research that reflects real user goals, workflows, and quality standards. Translate user needs into measurable constructs, evaluation criteria, rubrics, grading standards, and decision thresholds, and coordinate with applied scientists and product stakeholders to create representative eval sets. Scale research collaboration and tools that improve reliability, validity, and impact of human and agentic evaluation.

Responsibilities

• Develop and scale human-centered model evaluation approaches combining quantitative and qualitative methods.
• Translate user goals, behaviors, and expectations into measurable constructs, eval criteria, rubrics, grading standards, and decision thresholds.
• Design and conduct human grading, calibration, and “grade the grader” research to improve reliability and validity of evaluation systems.
• Partner with applied scientists and product teams to build representative eval sets, diagnose eval misalignment, and turn findings into actionable inputs for model and eval development.
• Establish an Office-wide framework for research partnerships with applied science across shared and app-specific teams (engagement models, coverage priorities, reusable methods, quality standards).
• Align and advance research-led and research-influenced evaluation practices across partner teams.
• Develop reusable tools, processes, templates, guidance, and “rhythms” for scalable research contribution to model evaluation.
• Amplify the role and impact of user research through storytelling, education, consultation, and visible examples of research influence.

Requirements

• Doctorate in a relevant field AND 3+ years UX research experience
  OR Master’s degree in a relevant field AND 4+ years UX research experience
  OR Bachelor’s degree in a relevant field AND 6+ years UX research experience
  OR equivalent experience.
• Ability to meet security screening requirements, including passing the Microsoft Cloud background check upon hire/transfer and every two years thereafter.

Benefits

Certain roles may be eligible for benefits and other compensation.