Principal Software Engineer, AI Frameworks

Microsoft

13m ago • Remote

Senior

Remote

142,800 - 274,800 USD/yr

  1. Select offer
  2. Add resume
  3. Analyze
  4. Improve
  5. Apply
  6. Auto-send · soon

Match your resume with similar open roles

Upload your resume to automatically match with active openings similar to this role.

PDF up to 10 MB

Your resume is private & secure

Potential salary for Principal Software Engineer, AI Frameworks role

Improve your resume now

$142,800 – $274,800/year

Quick Facts

  • Role: Principal Software Engineer (AI Frameworks)

  • Focus: AI frameworks, performance engineering, benchmarking, and developer tooling

Description

You will lead hands-on engineering initiatives that define technical vision and architecture for critical AI framework capabilities at cloud scale. The role emphasizes cross-stack investigations, disciplined measurement/observability, and turning one-off findings into durable platform improvements that increase model onboarding velocity, runtime performance, reliability, and hardware utilization.

Responsibilities

  • Define technical vision, architecture, and multi-release strategy for AI frameworks, performance, benchmarking, or developer productivity capabilities

  • Lead ambiguous, cross-stack investigations across models, frameworks, compilers, runtimes, systems, services, and silicon

  • Establish common measurement, automation, observability, and engineering mechanisms to scale analyses into platform capabilities

  • Drive measurable improvements in model onboarding velocity, runtime performance, reliability, hardware utilization, and Azure capacity efficiency

  • Influence architecture and priorities across teams; align researchers, product groups, infrastructure owners, and hardware partners

  • Provide hands-on leadership via prototypes, critical-path implementation, design/code reviews, complex debugging, and operational readiness

  • Raise the engineering bar through mentoring, technical leadership, and standards for quality and maintainability

Requirements

  • Bachelor’s degree in Computer Science or a related technical field

  • 6+ years of technical engineering experience coding in C++ and/or Python (or equivalent)

  • Ability to meet required security screening (Microsoft Cloud Background Check at hire/transfer and every two years)

  • Deep expertise in GPU (or equivalent accelerator) programming, compilation, and low-level execution

  • Expertise in LLM parallelism strategies (tensor, pipeline, data, expert parallelism)

  • Expertise in distributed inference acceleration (prefill/decode disaggregation, KV-cache transfer, collective communication, compute/communication overlap)

  • Strong understanding of LLM serving architectures such as vLLM, SGLang, or equivalent systems

  • Demonstrated leadership of cross-team technical initiatives from strategy/design through implementation, deployment, and sustained production impact

  • Track record of creating reusable platforms, influencing stakeholders, and mentoring engineers

  • Ability to use AI-assisted development tools effectively while maintaining correctness, performance, and maintainability

Benefits

  • Competitive base pay range for the U.S.: USD $142,800–$274,800 per year (with different ranges for specific locations)

  • Eligible roles may receive additional benefits and compensation

See similar jobs

Explore similar jobs and use your resume score to choose where to apply next

See more

Unlock Full Resume Report