Principal AI Infrastructure Software Engineer

NVIDIA has been transforming accelerated computing with innovation that’s fueled by great technology—and amazing people. As part of Nvidia's applied AI team for chip design, you will have the opportunity to tap into the unlimited potential of AI and change the landscape of the chip industry. Our team operates at the intersection of research, engineering, and product development, transforming innovative ideas and research breakthroughs into real-world solutions.  

You will collaborate closely with researchers to design and scale agents - enabling them to reason, plan, call tools and code just like human engineers. You will work on building and maintaining the core infrastructure for deploying and running these agents in production, powering all our agentic tools and applications and ensuring their seamless and efficient performance. If you're passionate about the latest research and cutting-edge technologies shaping generative AI, this role and team offer an exciting opportunity to be at the forefront of innovation. 

What you'll be doing:  

• Design, develop, and maintain large-scale enterprise AI Infrastructure that brings to bear LLMs for building AI applications to improve efficiency for NVIDIA software and hardware engineers.  

• Work with HW chip designers and LLM research teams to grasp GPU design needs and align LLM infrastructure accordingly.  

• Optimize the infrastructure for performance, scalability, and reliability, ensuring secure and efficient management of data.  

• Stay ahead by engaging with the latest industry advancements in AI, continuously looking for opportunities to apply these advancements to improve LLM infrastructure.  

• Lead with purpose and maintain high-quality engineering practices that inspire others to achieve excellence.  

What we need to see:  

• Master or PhD degree in Computer Science, Electrical Engineering, or a relevant subject area (or equivalent experience).

• 15+ years of experience managing large-scale distributed systems or enterprise AI infrastructure.

• Expert-level proficiency in Python (required), advanced experience in JavaScript and deep proficiency in software engineering principles, high-performance coding, and system optimization.  

• Extensive track record in architecting, scaling, and governing robust enterprise infrastructure—including CI/CD, Docker, Kubernetes, messaging systems (Kafka), data pipelines, and both SQL/NoSQL (esp. MongoDB/Redis)—for secure, reliable production deployments.  

• Industry-leading expertise in AI/LLM infrastructure and agentic systems, including end-to-end design and integration of LLM/agent frameworks (LangChain, LangGraph, CrewAI, AutoGen), RAG, vector databases, and secure, compliant production deployments.  

• Demonstrated leadership by defining technical direction, shaping system design, and launching crucial platforms from idea to operation; adept at guiding, persuading, and forming successful international teams.  

• Excellent communication, collaboration, and problem-solving skills. Proven experience collaborating with and building teams for large-scale, user-facing GenAI/LLM applications across organizational boundaries. 

NVIDIA is widely considered to be one of the technology world's most desirable employers. We have some of the most forward-thinking and hardworking people in the world working for us. Are you a creative and autonomous engineer who loves challenges? Come join our AI Infrastructure team and help us build the future of chip design. 

#LI-Hybrid