Principal Engineer, On-Device Inference & GPU Kernels
Opera Software
Warsaw, Poland • 39m ago • On-site
Senior
On-site
Match your resume with similar open roles
Upload your resume to automatically match with active openings similar to this role.
Quick Facts
Role: Principal Engineer (On-Device Inference & GPU Kernels)
Focus: High-performance on-device AI inference and GPU optimization
Description
We are looking for a hands-on Principal Engineer to drive performance-critical on-device AI inference, spanning speculative decoding, GPU kernels, quantization, and memory management. You’ll build and tune kernels and runtime components to make language models faster and more efficient on consumer hardware across multiple backends. Work includes instrumentation and regression gates to catch performance, numerical, and UI-impact issues early.
Role & Responsibilities
Speculative decoding: implement and improve draft-modeling approaches to raise acceptance length while reducing drafting cost
Kernels: write, fuse, and tune kernels for attention, quantized matmul, MoE routing, sampling, and draft verification across Metal/MLX, CUDA, Vulkan, and OpenVINO
Quantization: design mixed-precision schemes for weights, activations, and KV cache; evaluate quality-cost tradeoffs per hardware format
Prefill and decode: improve time-to-first-token and tokens per second using flash/paged attention, sliding-window/hybrid attention, chunked prefill, and kernel-level scheduling
Tiered memory: build hierarchical KV/context caching across GPU memory, system RAM, and SSD with prefix reuse, eviction, and prefetching
Testing & performance: create a test matrix, performance telemetry, and regression gates for slow drivers, wrong numerics, and UI stalls
Instrumentation: expose runtime hooks (e.g., activations, attention) for research and security without sacrificing fused-kernel speed
Requirements
Deep, hands-on GPU programming experience in at least one of CUDA, Metal/MLX, or Vulkan compute, plus evidence of delivering work to a second backend
Hands-on speculative decoding experience (implemented or trained drafters) with understanding of acceptance rate drivers and end-to-end speed-up
Proven ability to make real inference workloads measurably faster (tokens/sec, latency, or memory) with explainable metrics
Strong command of quantization and how it affects model quality, including measurement methods
Correctness discipline: validate numerics against a reference and measure quality, not just speed
Benefits
Advanced computing resources (Apple Silicon, NVIDIA including Blackwell-class, AMD, Intel; cloud GPUs for training)
Technical ownership driving performance-critical on-device AI, working directly with AI leadership
Global impact: AI solutions used by hundreds of millions of browser users across diverse hardware
Research opportunities: contribute to open source and publish technical findings where appropriate
See similar jobs
Explore similar jobs and use your resume score to choose where to apply next
See morePrincipal Software Engineer, AI Frameworks
Microsoft
Senior
Technology
Remote
$143k - $275k/yr
1 days ago

Principal Hardware Engineer
Microsoft Corporation
Senior
Technology
Redmond, WA · On-site
$143k - $275k/yr
2 days ago

Advanced AI Workloads Performance and Scalability Engineer
AMD (Advanced Micro Devices)
Senior
Technology
San Jose, CA · On-site
$210k - $210k/yr
10 days ago

Principal Machine Learning Engineer (AI)
Next Step Systems
Senior
Technology
Remote
$170k - $200k/yr
10h ago
Principal Hardware Engineer
Microsoft
Senior
Technology
Redmond, WA · Hybrid
$143k - $275k/yr
5 days ago
Tailor Resume to fit the Job Offer
Use AI to find out how well the skills on your resume fit this job description.
Principal AI Software Engineer
Microsoft
Senior
Technology
Redmond, WA · Hybrid
$143k - $275k/yr
6 days ago

Performance & Debug Software Engineer (AI Inference)
Positron.ai
Senior
Technology
Remote
$200k - $300k/yr
10 days ago

Principal Software Engineer
Robert Half
Senior
Technology
Knoxville, TN · Remote
N/A
5 days ago
Principal Software Engineer, Core AI
Microsoft
Senior
Technology
Redmond, WA · Hybrid
$143k - $275k/yr
7 days ago
Unlock Full Resume Report
ATS Pass
Missing keywords
Tailored AI suggestions
Job match analysis
Interview-focused insights