Senior Machine Learning Engineer, Conversational AI (Agent & Inference Services)

Apple

Cupertino, CA • 11m ago • On-site

Senior

On-site

184,700 - 324,800 USD/yr

  1. Select offer
  2. Add resume
  3. Analyze
  4. Improve
  5. Apply
  6. Auto-send · soon

Match your resume with similar open roles

Upload your resume to automatically match with active openings similar to this role.

PDF up to 10 MB

Your resume is private & secure

Potential salary for Senior Machine Learning Engineer, Conversational AI (Agent & Inference Services) role

Improve your resume now

$184,700 – $324,800/year

Quick Facts

  • Role: Senior Machine Learning Engineer for Conversational AI services and agentic reasoning

Description

Hands-on role to own the conversational intelligence layer for a real-time, large-scale assistant. You will design multi-turn conversational experiences, build agent logic for planning and tool-calling, and develop query understanding models that convert natural language into structured intent. You will also ground answers using retrieval and knowledge graph services connected to real-time data, while implementing automated evaluation (including LLM-as-judge) integrated into the development workflow.

Responsibilities

  • Own the agent planning and tool-calling layer, including tool schemas, dynamic tool selection, and structured output generation; ground responses in retrieval/knowledge graph/real-time data.

  • Design and ship conversational understanding components: intent understanding, context tracking, and disambiguation using LLM- and small-model query parsers.

  • Optimize LLM inference latency and cost using distillation, quantization, compact output formats, dynamic batching, KV-cache tuning, semantic caching, and appropriate inference runtimes.

  • Build automated evaluation for multi-turn conversational and generative quality using LLM-as-judge pipelines, judge calibration against human ratings, and error-analysis tooling; integrate evaluation into CI.

  • Define offline metrics that predict user outcomes and collaborate on A/B tests to validate them.

  • Take models and services from experiment to the live request path with versioning, monitoring, and rollback; own quality, latency, and reliability post-launch.

  • Collaborate with research scientists, engineers, and product teams to productionize conversational AI, retrieval, and inference research.

Requirements

  • Master’s degree in Computer Science, Machine Learning, NLP, or related field

  • 7+ years of experience as a software or machine learning engineer with a significant focus on conversational AI, search, or LLM-powered systems

  • Strong programming skills in Go and Python; plus at least one of Java or Scala

  • Hands-on experience building and shipping conversational AI/dialogue systems (intent/slot understanding, multi-turn context, clarification/disambiguation)

  • Production experience with LLM-powered applications: prompt design, tool/function calling, structured output generation, and agentic orchestration

  • Demonstrated LLM inference optimization: distillation, quantization, batching, KV-cache tuning, caching, and profiling

  • Experience deploying/serving LLMs or ML models in the production request path using vLLM, SGLang, Triton, or TorchServe

  • Knowledge graphs plus low-latency lookup/retrieval services experience

  • Fluency with PyTorch and the Hugging Face ecosystem (Transformers, PEFT)

  • Familiarity with AWS and containerization (Docker, Kubernetes)

  • Excellent communication and collaboration

Benefits

  • Work on a large-scale, real-time conversational AI system with stringent production latency and cost requirements

  • Build grounded, accurate responses using retrieval, knowledge graphs, and real-time data

  • Own the full lifecycle from experimentation to CI-integrated automated quality evaluation

See similar jobs

Explore similar jobs and use your resume score to choose where to apply next

See more

Unlock Full Resume Report