ML Platform Engineer (Evaluation Systems)

Bask Health

15m ago • Remote

Senior

Remote

  1. Select offer
  2. Add resume
  3. Analyze
  4. Improve
  5. Apply
  6. Auto-send · soon

Match your resume with similar open roles

Upload your resume to automatically match with active openings similar to this role.

PDF up to 10 MB

Your resume is private & secure

Resume Checker & Builder preview

Quick Facts

  • Role: ML Platform Engineer (Evaluation Systems)

  • Focus: Infrastructure for reliable machine learning evaluation, benchmarking, and experiment repeatability at scale

Description

Build and maintain ML evaluation infrastructure that enables consistent and scalable assessment of machine learning and AI applications. You will design platform services, data pipelines, execution environments, orchestration, and monitoring to support reproducible experiments and accurate model comparisons before and after deployment. The role also requires secure, privacy-aware handling of sensitive healthcare-related evaluation data.

Responsibilities

  • Design, build, and maintain machine learning evaluation infrastructure

  • Develop platform services to execute and manage automated model evaluation workloads

  • Implement job orchestration, scheduling, and execution management for evaluation pipelines

  • Store, version, and manage evaluation datasets and evaluation artifacts/results

  • Build experiment tracking for model configurations (prompts/parameters) and evaluation outcomes

  • Create reproducible workflows to support consistent model comparisons

  • Implement automated benchmarking pipelines

  • Provide APIs/services for teams to submit, run, and monitor evaluation jobs

  • Compare evaluation results across model versions and experiments

  • Support evaluation workflows for large language models, predictive models, and retrieval systems

  • Build asynchronous processing for computationally intensive evaluations

  • Add monitoring, logging, and tracing for performance, failures, latency, and resource usage

  • Integrate evaluation infrastructure with model registries and ML development workflows

  • Implement automated quality checks for controlled releases

  • Add reliability mechanisms such as retries, timeouts, and failure recovery

  • Optimize execution performance, utilization, and operational costs; support distributed execution on cloud compute

  • Ensure secure access controls and tenant boundary isolation for datasets and results

  • Investigate incidents, improve reliability, and document operational standards

  • Own ML evaluation platform projects from design through deployment and continuous improvement

Requirements

  • 3+ years of professional experience in software engineering, machine learning engineering, platform engineering, or a closely related field

  • Production experience building and maintaining backend services or infrastructure

  • Strong programming skills in Python, TypeScript, or comparable languages

  • Familiarity with machine learning development and evaluation workflows

  • Experience developing APIs and backend service integrations

  • Understanding of distributed systems and asynchronous processing

  • Experience working with cloud-based infrastructure and production deployments

  • Familiarity with automated testing and continuous integration practices

  • Understanding of machine learning model versioning and experiment reproducibility

  • Experience working with relational databases and structured datasets

  • Familiarity with data pipelines and batch processing workflows

  • Experience designing reliable background job systems

  • Understanding of infrastructure scalability, performance, and fault tolerance

  • Familiarity with logging, monitoring, and production observability

  • Experience implementing retries, timeouts, and error recovery

  • Understanding of authentication, authorization, and secure service communication

  • Familiarity with configuration management and deployment automation

  • Ability to write maintainable, well-tested production code

  • Ability to troubleshoot complex backend/infrastructure issues

  • Understanding of software release processes and operational reliability

  • Experience using Git and collaborative engineering workflows

  • Ability to translate ML requirements into scalable infrastructure

  • Strong analytical thinking and problem-solving skills

  • Ability to collaborate with ML engineers and applied scientists

  • Ability to independently own projects from design through deployment

See similar jobs

Explore similar jobs and use your resume score to choose where to apply next

See more

Unlock Full Resume Report