Performance & Debug Software Engineer (AI Inference)

Quick Facts

• Role: Performance & Debug Software Engineer (AI Inference)

Description

Improve end-to-end large language model (LLM) inference performance on custom AI accelerators, including FPGA-based Atlas systems and systems built on an ASIC. You will profile and optimize latency, throughput, and utilization across hardware-accelerated attention, device runtime, host software, and on-device execution.

Responsibilities

• Profile and optimize inference performance across the full stack using performance counters, traces, telemetry, and simulation
• Develop and optimize distributed inference, including efficient KV cache management and data movement across heterogeneous hardware
• Build representative benchmarks and workload traces to expose bottlenecks in compute, memory, chip-to-chip interconnect, networking, scheduling, and model loading
• Partner with test engineers to turn performance findings into regression tests and release gates
• Influence architecture and interfaces where profiling shows structural limitations

Requirements

• Strong C++ and Python skills; hands-on profiling and optimization experience for complex systems using Linux VFIO interfaces
• Deep understanding of memory hierarchy, concurrency, data movement, batching, scheduling, and performance measurement
• Experience using profilers, performance counters, traces, and statistical methods to distinguish signal from noise
• Demonstrated ability to debug across application, runtime, driver, firmware, and hardware boundaries
• Experience bringing up software on new silicon and analyzing performance optimizations in production deployments
• Enthusiasm for AI-accelerated software development, including using coding agents and coordinating subagents

Benefits

• Fully company-paid medical, dental, and vision insurance for you and dependents
• Company-paid life and disability coverage, plus optional supplemental hospital, critical illness, and accident coverage
• Unlimited paid time off and 13 paid company holidays
• Remote-first culture with company-provided computer and home office setup
• Competitive salary and equity; 401(k) with company matching (eligible from day one)
• Visa support via H-1B transfer facilitation for eligible candidates (no new visa sponsorship)