Senior Software Engineer (LLM Evaluation & Test Infrastructure)
Apple
Cupertino, CA • 15m ago • On-site
Senior
On-site
216,200 - 324,800 USD/yr
Match your resume with similar open roles
Upload your resume to automatically match with active openings similar to this role.
Potential salary for Senior Software Engineer (LLM Evaluation & Test Infrastructure) role
Improve your resume now
$216,200 – $324,800/year
Quick Facts
- Senior role focused on LLM evaluation and test/validation infrastructure for generative AI models
Description
Define and develop generative AI validation infrastructure in collaboration with cross-functional partners across ML, data science, software engineering, and product management. Build or adopt client and server CI pipelines and scalable automation tools and frameworks to enable rigorous testing for accuracy, performance, reliability, and stability across products and use cases. Work with product and program teams to align validation infrastructure with the roadmap and drive adoption among product and ML engineers.
Responsibilities
- Design and develop LLM online and offline validation infrastructure, tools, frameworks, and methodologies
- Implement or enhance client/server CI pipelines for generative AI validation
- Create scalable automation for evaluation across all supported platforms
- Enable cross-functional teams to run rigorous model testing (accuracy, performance, reliability, stability)
- Coordinate validation infrastructure development in lockstep with product and feature roadmaps
Benefits
- High visibility, impact, and influence building advanced generative AI systems
- Fast-paced environment with meaningful cross-team collaboration
Requirements
- BS/MS in Computer Science, Machine Learning, or related field
- 6+ years of proven software development expertise in Java, Python, Swift, or Objective-C
- Strong ability to build scalable and robust infrastructure for training/inference pipelines using on-device and server-side ML models
- Fundamental knowledge of ML concepts and of ML-based product lifecycle and operations
- Familiarity with ML toolkits such as PyTorch, TensorFlow, and scikit-learn
- Hands-on experience building automation infrastructure, tools, and frameworks for large-scale AI/ML or cloud-based applications
- Demonstrated experience evaluating and testing machine learning models, including performance metrics, validation techniques, and statistical analysis
See similar jobs
Explore similar jobs and use your resume score to choose where to apply next
See moreLLM Inference Quality Engineer
Apple
Senior
Technology
Seattle, WA · On-site
$142k - $214k/yr
2 days ago
Senior Machine Learning Platform Engineer (Responsible AI & Safety)
Apple
Senior
Technology
Cupertino, CA · On-site
$185k - $278k/yr
4 days ago

Senior AI/ML Engineer (Generative AI)
Raas Infotek LLC
Senior
Technology
Texas City, TX · On-site
N/A
3 days ago

Senior Generative AI Engineer
Raas Infotek LLC
Senior
Technology
Remote
N/A
4 days ago

Senior Machine Learning Engineer, Evaluation & Reliability
Bask Health
Senior
Technology
Remote
N/A
4 days ago
Tailor Resume to fit the Job Offer
Use AI to find out how well the skills on your resume fit this job description.
Senior LLM Inference Quality Engineer
Apple
Senior
Technology
Seattle, WA · On-site
$175k - $309k/yr
7 days ago

Senior Generative AI Architect
Johnson Service Group, Inc.
Senior
Technology
Irvine, CA · Hybrid
N/A
3 days ago
Senior Software Engineer (AI/ML)
Apple
Senior
Technology
Cupertino, CA · On-site
$150k - $278k/yr
2 days ago
Senior Machine Learning Platform Engineer (Evaluation)
Apple
Senior
Technology
Seattle, WA · On-site
$175k - $309k/yr
14 days ago
Unlock Full Resume Report
ATS Pass
Missing keywords
Tailored AI suggestions
Job match analysis
Interview-focused insights