Principal Software Engineer, AI Frameworks

Quick Facts

Role: Principal Software Engineer – AI Frameworks (technical leadership / hands-on engineering)

Focus: AI serving stack and performance systems enabling frontier model execution at cloud scale

Key platforms: NVIDIA GPUs, AMD GPUs, and Microsoft silicon

Description

Own the technical vision, architecture, and delivery strategy for critical AI framework and serving-stack capabilities. Lead ambiguous, cross-stack investigations and make architectural decisions to improve model onboarding velocity, runtime performance, production reliability, and hardware utilization across accelerators. Stay hands-on in critical implementation areas and turn measurement into durable platform improvements.

Responsibilities

• Own the technical vision, architecture, roadmap, and multi-release execution strategy for AI framework, performance, benchmarking, or developer-productivity capabilities
• Lead cross-team technical alignment by shaping architecture/priorities, resolving competing requirements, and driving measurable outcomes
• Build technical leadership capacity through mentoring, defining quality/maintainability standards, and fostering an inclusive engineering culture
• Steer ambiguous, cross-stack investigations across models, frameworks, compilers, runtimes, systems, services, and silicon; set priorities and resolve trade-offs
• Provide hands-on technical leadership through prototypes, critical-path implementation, design/code reviews, complex debugging, and operational readiness
• Spearhead common measurement, automation, observability, and engineering mechanisms that scale one-off analyses into platform capabilities
• Drive measurable improvements in model onboarding velocity, runtime performance, reliability, hardware utilization, and Azure capacity efficiency
• Embody Microsoft’s culture and values

Requirements

• Bachelor’s degree in Computer Science (or related technical field) and 8+ years of technical engineering experience coding in C++ and/or Python, or equivalent experience
• Ability to meet security screening requirements, including the Microsoft Cloud Background Check at hire/transfer and every two years thereafter
• Demonstrated experience serving as a technical lead for complex cross-team initiatives (strategy, architectural decisions, stakeholder alignment, concept-to-production execution)
• Experience designing and shipping complex high-performance or distributed software systems
• Deep foundation in software architecture, computer architecture, and accelerator-aware optimization
• Advanced experience with LLM inference frameworks or AI accelerator kernel optimizations
• Track record of creating reusable platforms and scaling engineering practices across teams

Benefits

• Base pay range (U.S.): USD $165,600 – $296,400 per year (different location-specific ranges may apply)
• May be eligible for benefits and other compensation