Principal Software Engineer, AI Frameworks

Quick Facts

• Role: Principal Software Engineer, AI Frameworks

Description

Lead the technical vision and architecture for AI frameworks and the serving stack for frontier OpenAI models, including performance, benchmarking, and developer-productivity capabilities at cloud scale. Drive cross-team execution and hands-on implementation of optimizations for NVIDIA and AMD GPUs and Microsoft silicon to improve model velocity, reliability, and hardware utilization. Build measurement, automation, observability, and reusable platform mechanisms that turn one-off analyses into durable capabilities.

Responsibilities

• Own the technical vision, architecture, roadmap, and multi-release execution strategy for critical AI framework and performance/benchmarking or developer-productivity capabilities.

• Lead technical alignment across teams by shaping architecture and priorities, resolving competing requirements, and ensuring clear decisions and measurable outcomes.

• Mentor engineers and build technical leadership capacity through high standards for quality and maintainability and an inclusive engineering culture.

• Steer ambiguous, cross-stack investigations spanning models, frameworks, compilers, runtimes, systems, services, and silicon; set priorities and resolve trade-offs.

• Provide hands-on technical leadership via prototypes, critical-path implementation, design/code reviews, complex debugging, and operational readiness.

• Establish common measurement, automation, observability, and engineering mechanisms to scale platform capabilities.

• Drive measurable improvements in model onboarding velocity, runtime performance, reliability, hardware utilization, and Azure capacity efficiency.

Requirements

• Bachelor’s Degree in Computer Science or a related technical field and 8+ years of technical engineering experience coding in languages such as C++ or Python (or equivalent experience).

• Ability to meet Microsoft security screening requirements, including the Microsoft Cloud Background Check, and pass it upon hire/transfer and every two years thereafter.

• Demonstrated experience serving as a technical lead for complex cross-team initiatives (strategy, architectural decisions, stakeholder alignment, end-to-end execution to production).

• Robust/extensive experience designing and shipping complex, high-performance or distributed software systems.

• Deep foundation in software architecture, computer architecture, and accelerator-aware optimization.

• Advanced experience with LLM inference frameworks or AI accelerator kernel optimizations.

• Proven track record of creating reusable platforms and scaling engineering practices across teams.

Benefits

• Competitive base pay range for the U.S.: USD $165,600–$296,400 per year (with different ranges for specific locations such as San Francisco Bay area and New York City metro).

• Eligibility for benefits and other compensation.

• Minimum posting duration of 5 days; applications accepted on an ongoing basis until the position is filled.