Senior Machine Learning Engineer, Conversational AI (Agent & Inference Services)
Apple
Cupertino, CA • 11m ago • On-site
Senior
On-site
184,700 - 324,800 USD/yr
Match your resume with similar open roles
Upload your resume to automatically match with active openings similar to this role.
Potential salary for Senior Machine Learning Engineer, Conversational AI (Agent & Inference Services) role
Improve your resume now
$184,700 – $324,800/year
Quick Facts
Role: Senior Machine Learning Engineer for Conversational AI services and agentic reasoning
Description
Hands-on role to own the conversational intelligence layer for a real-time, large-scale assistant. You will design multi-turn conversational experiences, build agent logic for planning and tool-calling, and develop query understanding models that convert natural language into structured intent. You will also ground answers using retrieval and knowledge graph services connected to real-time data, while implementing automated evaluation (including LLM-as-judge) integrated into the development workflow.
Responsibilities
Own the agent planning and tool-calling layer, including tool schemas, dynamic tool selection, and structured output generation; ground responses in retrieval/knowledge graph/real-time data.
Design and ship conversational understanding components: intent understanding, context tracking, and disambiguation using LLM- and small-model query parsers.
Optimize LLM inference latency and cost using distillation, quantization, compact output formats, dynamic batching, KV-cache tuning, semantic caching, and appropriate inference runtimes.
Build automated evaluation for multi-turn conversational and generative quality using LLM-as-judge pipelines, judge calibration against human ratings, and error-analysis tooling; integrate evaluation into CI.
Define offline metrics that predict user outcomes and collaborate on A/B tests to validate them.
Take models and services from experiment to the live request path with versioning, monitoring, and rollback; own quality, latency, and reliability post-launch.
Collaborate with research scientists, engineers, and product teams to productionize conversational AI, retrieval, and inference research.
Requirements
Master’s degree in Computer Science, Machine Learning, NLP, or related field
7+ years of experience as a software or machine learning engineer with a significant focus on conversational AI, search, or LLM-powered systems
Strong programming skills in Go and Python; plus at least one of Java or Scala
Hands-on experience building and shipping conversational AI/dialogue systems (intent/slot understanding, multi-turn context, clarification/disambiguation)
Production experience with LLM-powered applications: prompt design, tool/function calling, structured output generation, and agentic orchestration
Demonstrated LLM inference optimization: distillation, quantization, batching, KV-cache tuning, caching, and profiling
Experience deploying/serving LLMs or ML models in the production request path using vLLM, SGLang, Triton, or TorchServe
Knowledge graphs plus low-latency lookup/retrieval services experience
Fluency with PyTorch and the Hugging Face ecosystem (Transformers, PEFT)
Familiarity with AWS and containerization (Docker, Kubernetes)
Excellent communication and collaboration
Benefits
Work on a large-scale, real-time conversational AI system with stringent production latency and cost requirements
Build grounded, accurate responses using retrieval, knowledge graphs, and real-time data
Own the full lifecycle from experimentation to CI-integrated automated quality evaluation
See similar jobs
Explore similar jobs and use your resume score to choose where to apply next
See more
Senior Agentic AI Engineer NDA
GT
Senior
Technology
Warsaw, Poland · Remote
N/A
7 days ago

Senior AI Engineer
Photon
Senior
Technology
Dallas, TX · On-site
$100k - $120k/yr
1 days ago
Senior Applied AI Engineer
Flyps
Senior
Technology
Poznań, Pl-30, Poland · Remote
312k zł - 396k zł/yr
13 days ago
Senior AI Engineer
Craftware
Senior
Technology
Warsaw, Maz, Poland · Remote
N/A
11 days ago

Senior AI Engineer
AgileEngine
Senior
Technology
Kraków, Poland · Remote
$6k - $8k/yr
14 days ago
Tailor Resume to fit the Job Offer
Use AI to find out how well the skills on your resume fit this job description.

Senior/Staff Software Engineer (AI Agents)
Virco Talent
Senior
Technology
Remote
$210k - $230k/yr
8 days ago

Senior Machine Learning Engineer (Agentic Systems)
Adobe Systems
Senior
Technology
San Jose, CA · On-site
$173k - $307k/yr
4 days ago

Senior AI Solutions Engineer with Python
EPAM Systems
Senior
Technology
Poznań, Poland · Remote
N/A
13 days ago

Senior Software Engineer, AI
Thomson Reuters U.S. Inc.
Senior
Technology
Eagan, MN · On-site
$118k - $220k/yr
2 days ago
Unlock Full Resume Report
ATS Pass
Missing keywords
Tailored AI suggestions
Job match analysis
Interview-focused insights