Senior Machine Learning Engineer

Quick Facts

• Senior/Staff-level role for distributed systems and AI execution across training and inference
• Focus on scalable model training, post-training, fine-tuning, reinforcement learning, and high-performance LLM serving

Description

Build core capabilities for large-scale model training, post-training, fine-tuning, reinforcement learning, and optimized high-performance inference. Develop scalable AI execution frameworks and developer-facing APIs/SDKs that improve experimentation speed and help teams efficiently train, adapt, and serve advanced models. Drive LLM inference runtime optimizations to improve throughput, latency, cost, and reliability across heterogeneous hardware and model architectures.

Responsibilities

• Design and build scalable distributed AI execution frameworks for training, post-training, reinforcement learning, and inference using Ray and related technologies
• Develop high-level APIs/SDKs and reusable workflows for fine-tuning, preference optimization, and reinforcement learning workflows
• Enable end-to-end post-training workflows from dataset preparation and experiment orchestration through distributed execution, evaluation, and iterative model improvement
• Optimize LLM inference runtimes for cost, latency, throughput, and resource efficiency
• Implement deep runtime optimizations across GPU execution, memory management, kernel utilization, communication, batching, KV-cache management, and distributed inference
• Evaluate and integrate modern inference/training engines (e.g., vLLM, SGLang)
• Design distributed execution strategies for large models, including parallelism, workload placement, resource allocation, and communication efficiency
• Improve training and inference reliability via fault tolerance, checkpointing, workload recovery, observability, and performance diagnostics
• Partner with AI researchers, model developers, and product teams to translate model and workload requirements into reusable capabilities
• Establish benchmarks to quantify improvements in training efficiency, post-training iteration speed, inference latency/throughput, and cost per token
• Contribute architectural improvements to Ray and related open-source AI systems when appropriate

Requirements

• 6+ years of experience building distributed systems, AI platforms, model execution frameworks, or high-performance software systems
• Strong hands-on experience in at least one: large-scale AI training, LLM post-training and reinforcement learning, or LLM inference and serving optimization
• Proficiency in Python and experience with a systems-level language such as C++, Rust, Go, or Java
• Experience with PyTorch and modern foundation model architectures
• Understanding of distributed execution and parallelism strategies for large models (data, tensor, pipeline, or expert parallelism)
• Experience profiling and optimizing AI workloads across GPUs/accelerators, identifying bottlenecks in compute, memory, and communication
• Strong understanding of at least one: distributed training/post-training workflows, reinforcement learning systems, LLM inference runtimes, or high-performance model serving
• Experience building APIs, SDKs, or reusable software components for AI model development, training, adaptation, evaluation, or deployment
• Experience with Ray (or comparable distributed execution frameworks) for production-scale AI workloads
• Strong software design, debugging, and performance analysis skills for reliable, scalable, production-quality systems
• B.S., M.S., or Ph.D. in Computer Science/Artificial Intelligence (or related) or comparable experience

Benefits

• Build end-to-end training-to-inference execution capabilities for large-scale foundation models
• Work on runtime and serving optimizations that reduce latency and cost while increasing throughput
• Contribute to open-source distributed AI execution frameworks through architectural improvements