Principal Data Scientist

Quick Facts

• Role: Principal Data Scientist (senior individual contributor, hands-on)

Description

Own and standardize experimentation at scale, shaping how controlled tests drive product decisions across marketplace and digital products. Lead end-to-end experimentation requirements, design modern methods for different traffic regimes, and operationalize them into production systems. Serve as a senior technical mentor who evangelizes data science best practices and improves how results are interpreted and communicated.

Responsibilities

• Consolidate and own experimentation design standards across the organization: randomization, metric definition, guardrails, sample size and duration, and how results are read and acted on.

• Partner with engineering and product to adopt the standard via instrumentation, assignment, and platform behavior.

• Define requirements for holistic experimentation end to end, including platform requirements, telemetry, metric layers, analysis tooling, and review process.

• Design methods that fit real traffic and iteration constraints; build an experimentation portfolio across different volume regimes.

• Evaluate and operationalize modern experimentation methods (e.g., Bayesian decision frameworks, always-valid inference and sequential testing, multi-armed bandits, variance reduction, interference-aware designs).

• Teach experimentation practice across product, engineering, and business stakeholders; improve how experiment results are interpreted and communicated.

Requirements

• 8+ years relevant experience in data science or applied statistics with substantial experimentation focus; or a PhD in a quantitative discipline plus 6+ years

• Deep expertise in experimental design and causal inference, including failure modes of online controlled experiments (power/sensitivity, variance reduction, multiple testing correction, heterogeneous treatment effects)

• Working command of both frequentist and Bayesian approaches with judgment on when to use each

• Experience designing experimentation methodology for a real product, including how experiments should be run

• Track record setting technical standards that other teams adopted and influencing engineering partners without authority

• Strong applied experience with time series and web/behavioral data at scale

• Fluency in Python and SQL; comfort with modern data ecosystems such as Snowflake, Spark, Airflow, and dbt

• Exceptional written and verbal communication, including the ability to explain and defend methodological trade-offs

Benefits

• Inclusive, diverse environment committed to supporting people of many backgrounds.

• AI is used to enhance candidate evaluation workflows, while final hiring decisions remain under human oversight.