AI Inference Infrastructure Software Engineer (PhD Graduate)

Quick Facts

• Role: AI Inference Infrastructure Software Engineer (PhD Graduate)

• Focus: LLM inference infrastructure and GPU-optimized orchestration

• Start: 2027

Description

You will join the Inference Infrastructure team building AIBrix, a Kubernetes-native control plane for large-scale LLM inference. The role focuses on designing and operating cloud-native, GPU-optimized systems that take AI workloads from research to production at scale. You’ll collaborate on production inference using vLLM, SGLang, TensorRT-LLM, and other LLM engines.

Responsibilities

• Design and build large-scale, container-based cluster management and orchestration systems with extreme performance, scalability, and resilience.

• Architect next-generation cloud-native GPU and AI accelerator infrastructure for cost-efficient and secure ML platforms.

• Collaborate across teams to deliver inference solutions using vLLM, SGLang, TensorRT-LLM, and other LLM engines.

• Keep up with open-source and systems research (e.g., Kubernetes, Ray) and integrate best practices into production.

• Write high-quality, production-ready code that is maintainable, testable, and scalable.

Requirements

• Completing or recently completed a PhD in Software Development, Computer Science, Computer Engineering, or a related field.

• Strong understanding of large model inference and distributed/parallel systems and/or high-performance networking.

• Hands-on experience building cloud/ML infrastructure (resource management, scheduling, request routing, monitoring, or orchestration).

• Knowledge of containers and orchestration (Docker, Kubernetes).

• Proficiency in Go, Rust, Python, or C++.

• Must commit to an onboarding date by the end of the year; state availability and graduation date in your resume.

Benefits

• Day one access to medical, dental, and vision insurance.

• 401(k) savings plan with company match.

• Paid parental leave.

• Short-term and long-term disability coverage.

• Life insurance and wellbeing benefits.

• 10 paid holidays per year, 10 paid sick days per year, and 17 days of Paid Personal Time (prorated upon hire with increasing accruals by tenure).

Compensation (Annually)

Base salary range (selected city): $153,900 – $300,960. Total package may include additional discretionary bonuses/incentives and restricted stock units.