Machine Learning Engineer, AI/ML Model Quality

Quick Facts

• Role: Machine Learning Engineer (AI/ML Model Quality)

• Focus: Data quality, ML/GenAI validation, and model observability/monitoring for production models

Description

Own the health of the data ecosystem behind ML and Generative AI features across Wallet, Payments, and Commerce. Build validation frameworks, observability metrics, telemetry analysis, and synthetic data quality mechanisms to ensure models are trained, evaluated, and monitored on trusted data. Translate quality signals into actionable insights for engineering and product teams.

Responsibilities

• Curate, analyze, and maintain gold-standard ground-truth datasets for continuous ML and GenAI validation.

• Audit training data for systemic bias and fairness gaps before deployment; implement checks for bias introduced by data drift.

• Define and analyze telemetry across GenAI workflows (output coherence, latency, task completion rates, regression patterns).

• Design privacy-preserving mechanisms to detect production model degradation.

• Define and own ML observability metrics (model performance, output distributions, training-serving skew, silent degradation, feature drift).

• Build observability dashboards and reporting workflows for consistent, real-time model health visibility.

• Identify degradation patterns and GenAI domain-specific failure modes and recommend actions for model/data teams.

• Define, track, and report key data quality metrics (completeness, accuracy, timeliness, validity).

Requirements

• Bachelor’s degree with exceptional hands-on ML/AI model quality experience (or M.S./Ph.D. preferred) in ML/Computer Science/Data Science/Statistics/Math/Engineering or related.

• 3+ years defining and tracking ML model health metrics in production (monitoring, feature drift detection, observability instrumentation).

• 3+ years in ML/data science/analytics with focus on data quality, model evaluation, or ML observability in production.

• Proficiency in Python (Pandas, NumPy, Scikit-learn) and experience with PyTorch and SQL.

• Experience querying and analyzing large-scale datasets using distributed frameworks (PySpark/Spark/distributed SQL).

• Solid understanding of statistical methods (hypothesis testing, distribution analysis, data drift detection, statistical process control).

• Experience with GenAI/LLM systems, including quality failure modes, output evaluation approaches, and telemetry instrumentation.

• Strong communication skills for translating findings into actionable insights for technical and non-technical stakeholders.

Benefits

• Work at the foundation of ML features used by hundreds of millions of users, ensuring high-quality data for both conventional ML and the latest generative AI systems.

• Collaborate with Data Engineering, Privacy, and Legal teams to deliver privacy-preserving model quality and monitoring.