Senior Data Scientist - M365 Copilot Agents & Platform

Quick Facts

• Role: Senior Data Scientist (M365 Copilot Agents & Platform)

• Focus: Data analysis, evaluation dataset development, experimentation, and evidence-based product recommendations for enterprise AI features

Description

Work with a multidisciplinary team to advance AI-powered productivity experiences in Microsoft 365 Copilot by building measurement and evaluation frameworks across the AI development lifecycle. You will prepare and analyze large-scale datasets, design studies and rubrics for text/voice and multi-turn scenarios, run A/B tests, and translate results into decision-ready product recommendations.

Responsibilities

• Write efficient code (Python/SQL/KQL or similar) to prepare and analyze large-scale datasets (including customer feedback, text, and consented multimodal logs)

• Build representative evaluation datasets across enterprise and consumer scenarios (e.g., information retrieval, text creation/editing, image/file workflows, education, coding, technical support, health, sports, news, finance)

• Design evaluation coverage for text and voice inputs, single- and multi-turn conversations, and location-/geography-sensitive prompts

• Partner cross-functionally to define measurement strategies and inform product decisions

• Own experiment design and impact measurement for optimization hypotheses, including metrics, guardrails, and interpretation

• Conduct response-quality side-by-side evaluations and assess statistical and practical significance

• Compare LLM-based and human evaluations; investigate false positives and bias; refine studies

• Develop decision-relevant metrics, dashboards, and narratives to support product recommendations

• Present findings and analytical limitations to technical and executive audiences

• Stay current with AI and competitive developments; identify feature gaps using data and evaluation results

• Provide mentorship via analytical code reviews and study-design guidance

Requirements

• Doctorate + 1+ years data-science experience OR Master’s + 3+ years OR Bachelor’s + 5+ years OR equivalent experience in data science

• Ability to meet security screening requirements, including passing the Microsoft Cloud background check upon hire/transfer and every two years thereafter

• 4+ years using Python, SQL, KQL, R (or similar) for analyzing large-scale structured and unstructured datasets

• Strong cross-organizational communication and collaboration in ambiguous environments

• Ability to work end to end in complex analytical problem domains, from problem definition and study design through execution, interpretation, and communication

Benefits

• Base pay range (U.S.): USD $119,800–$234,700 per year; SF Bay Area and NYC metro: USD $160,200–$261,000 per year

• Eligible roles may receive benefits and other compensation; additional information available via the company pay link