LLM Inference Quality Engineer

Apple

Seattle, WA • 20m ago • On-site

Senior

On-site

142,300 - 214,300 USD/yr

  1. Select offer
  2. Add resume
  3. Analyze
  4. Improve
  5. Apply
  6. Auto-send · soon

Match your resume with similar open roles

Upload your resume to automatically match with active openings similar to this role.

PDF up to 10 MB

Your resume is private & secure

Potential salary for LLM Inference Quality Engineer role

Improve your resume now

$142,300 – $214,300/year

Quick Facts

  • Senior AI systems engineering role focused on LLM inference quality in production at data center scale.

Description

Own the correctness, consistency, and stability of LLM inference for every model, prompt, serving stack, and infrastructure change before it reaches users. Lead qualification systems and automation frameworks for output correctness, including evaluation for agentic workflows and tool-using models. Build AI-assisted QA tooling and define quality bars for “correct” and “stable” inference despite generative nondeterminism.

Responsibilities

  • Design and develop qualification frameworks validating output correctness, consistency, and stability across model, weight, prompt, runtime, and infrastructure changes.
  • Build the LLM inference output-correctness automation framework for large-scale deployment and platform validation.
  • Architect end-to-end testing and qualification for distributed ML serving (unit, integration, regression, end-to-end).
  • Create validation harnesses using mocking, trace replay, and synthetic traffic generation.
  • Gate releases in CI with before/after evals for changes to models, weights, prompts, tool definitions, or runtimes.
  • Create prompt-level and scenario-based regression suites and track quality across versions over time.
  • Validate tool-use, function-calling, and multi-turn agentic workflows for correct invocation, argument formation, task completion, and coherence; detect wrong/empty/silently degraded outputs.
  • Develop safety, refusal, and policy-compliance coverage to ensure guardrails and alignment behavior holds across releases.
  • Probe AI risks including prompt injection, data leakage across users/permissions, hallucinations/ungrounded output, and cost/latency regressions.
  • Build adversarial and robustness tests for long-context, ambiguous, and underrepresented inputs.
  • Qualify inference optimizations (quantization, KV-cache, speculative decoding, batching) via output parity against full-precision references.
  • Validate output equivalence across serving backends and accelerators.
  • Run functional, performance, fault-injection, and reliability testing; benchmark quality vs latency/throughput trade-offs.
  • Develop AI/agentic QA tooling to improve test-case generation, defect detection, flaky-test analysis, and triage automation.
  • Define metrics, logging pipelines, distributed tracing, and observability tooling for correctness, quality, drift, latency, and system health.
  • Use Splunk and internal debugging tools to root-cause failures in LLM serving systems.
  • Perform regression/performance/integration testing across platforms, including iOS and macOS, for on-device and server-backed AI features.
  • Develop production AI infrastructure software in Python, C++, and/or Swift with modularity, scalability, testability, and performance principles.

Requirements

  • 1+ year experience with hardware and/or software development lifecycle processes.
  • 1+ year experience in one or more compiled languages (C, C++, Objective-C/Swift).
  • Proficiency in one or more scripting languages (Python, Go, or JavaScript).
  • Experience using AI/LLMs to improve testing (test-case generation, defect detection, or reducing manual QA effort).
  • Familiarity with LLM serving and inference optimization (batching, KV-cache, quantization, speculative decoding, distributed/multi-accelerator serving) and GPU-accelerated software.
  • Experience with CI/CD, Kubernetes, containers, and cloud-native systems; integrate automated evals into release pipelines.
  • Experience with ML/LLMs and common frameworks (PyTorch, Hugging Face Transformers, or comparable stacks).
  • Experience with observability and distributed tracing for ML systems (Splunk, Prometheus, Grafana, LangSmith, or similar) and monitoring for model drift.
  • B.S., M.S., or Ph.D. in Computer Engineering, Electrical Engineering, Computer Science, or equivalent experience.

Benefits

  • Ship frontier AI quickly while maintaining rigorous correctness, safety, and user trust through automated qualification.
  • Hands-on work across ML systems engineering, distributed inference, and quality engineering at data center scale.

See similar jobs

Explore similar jobs and use your resume score to choose where to apply next

See more

Unlock Full Resume Report