Principal Software Engineer, AI Frameworks
Microsoft
13m ago • Remote
Senior
Remote
142,800 - 274,800 USD/yr
Match your resume with similar open roles
Upload your resume to automatically match with active openings similar to this role.
Potential salary for Principal Software Engineer, AI Frameworks role
Improve your resume now
$142,800 – $274,800/year
Quick Facts
Role: Principal Software Engineer (AI Frameworks)
Focus: AI frameworks, performance engineering, benchmarking, and developer tooling
Description
You will lead hands-on engineering initiatives that define technical vision and architecture for critical AI framework capabilities at cloud scale. The role emphasizes cross-stack investigations, disciplined measurement/observability, and turning one-off findings into durable platform improvements that increase model onboarding velocity, runtime performance, reliability, and hardware utilization.
Responsibilities
Define technical vision, architecture, and multi-release strategy for AI frameworks, performance, benchmarking, or developer productivity capabilities
Lead ambiguous, cross-stack investigations across models, frameworks, compilers, runtimes, systems, services, and silicon
Establish common measurement, automation, observability, and engineering mechanisms to scale analyses into platform capabilities
Drive measurable improvements in model onboarding velocity, runtime performance, reliability, hardware utilization, and Azure capacity efficiency
Influence architecture and priorities across teams; align researchers, product groups, infrastructure owners, and hardware partners
Provide hands-on leadership via prototypes, critical-path implementation, design/code reviews, complex debugging, and operational readiness
Raise the engineering bar through mentoring, technical leadership, and standards for quality and maintainability
Requirements
Bachelor’s degree in Computer Science or a related technical field
6+ years of technical engineering experience coding in C++ and/or Python (or equivalent)
Ability to meet required security screening (Microsoft Cloud Background Check at hire/transfer and every two years)
Deep expertise in GPU (or equivalent accelerator) programming, compilation, and low-level execution
Expertise in LLM parallelism strategies (tensor, pipeline, data, expert parallelism)
Expertise in distributed inference acceleration (prefill/decode disaggregation, KV-cache transfer, collective communication, compute/communication overlap)
Strong understanding of LLM serving architectures such as vLLM, SGLang, or equivalent systems
Demonstrated leadership of cross-team technical initiatives from strategy/design through implementation, deployment, and sustained production impact
Track record of creating reusable platforms, influencing stakeholders, and mentoring engineers
Ability to use AI-assisted development tools effectively while maintaining correctness, performance, and maintainability
Benefits
Competitive base pay range for the U.S.: USD $142,800–$274,800 per year (with different ranges for specific locations)
Eligible roles may receive additional benefits and compensation
See similar jobs
Explore similar jobs and use your resume score to choose where to apply next
See morePrincipal Software Engineer, AI Frameworks
Microsoft
Senior
Technology
Remote
$166k - $296k/yr
10 days ago
Senior/Principal Software Engineer, AI Frameworks
Microsoft
Senior
Technology
Redmond, WA · Hybrid
$143k - $275k/yr
3 days ago

Principal Software Engineer
Vertafore
Senior
Technology
Pulaski, TN · On-site
$115k - $150k/yr
5 days ago
Principal Software Engineer
Microsoft
Senior
Technology
Redmond, WA · Hybrid
$143k - $275k/yr
10 days ago
Principal Software Engineer
Microsoft
Senior
Technology
Redmond, WA · Hybrid
$166k - $296k/yr
10 days ago
Tailor Resume to fit the Job Offer
Use AI to find out how well the skills on your resume fit this job description.

Principal Software Engineer (AI)
ServiceNow
Senior
Technology
Santa Clara, CA · On-site
$221k - $387k/yr
6 days ago

Principal Software Engineer
Microsoft Corporation
Senior
Technology
Redmond, WA, United States of America · On-site
$143k - $275k/yr
6 days ago
Principal AI Software Engineer
Microsoft
Senior
Technology
Redmond, WA · Hybrid
$143k - $275k/yr
5 days ago
Senior Software Engineer, AI Frameworks
Microsoft
Senior
Technology
Remote
$120k - $235k/yr
6 days ago
Unlock Full Resume Report
ATS Pass
Missing keywords
Tailored AI suggestions
Job match analysis
Interview-focused insights