Principal Engineer, On-Device Inference & GPU Kernels

Opera Software

Warsaw, Poland • 39m ago • On-site

Senior

On-site

  1. Select offer
  2. Add resume
  3. Analyze
  4. Improve
  5. Apply
  6. Auto-send · soon

Match your resume with similar open roles

Upload your resume to automatically match with active openings similar to this role.

PDF up to 10 MB

Your resume is private & secure

Quick Facts

  • Role: Principal Engineer (On-Device Inference & GPU Kernels)

  • Focus: High-performance on-device AI inference and GPU optimization

Description

We are looking for a hands-on Principal Engineer to drive performance-critical on-device AI inference, spanning speculative decoding, GPU kernels, quantization, and memory management. You’ll build and tune kernels and runtime components to make language models faster and more efficient on consumer hardware across multiple backends. Work includes instrumentation and regression gates to catch performance, numerical, and UI-impact issues early.

Role & Responsibilities

  • Speculative decoding: implement and improve draft-modeling approaches to raise acceptance length while reducing drafting cost

  • Kernels: write, fuse, and tune kernels for attention, quantized matmul, MoE routing, sampling, and draft verification across Metal/MLX, CUDA, Vulkan, and OpenVINO

  • Quantization: design mixed-precision schemes for weights, activations, and KV cache; evaluate quality-cost tradeoffs per hardware format

  • Prefill and decode: improve time-to-first-token and tokens per second using flash/paged attention, sliding-window/hybrid attention, chunked prefill, and kernel-level scheduling

  • Tiered memory: build hierarchical KV/context caching across GPU memory, system RAM, and SSD with prefix reuse, eviction, and prefetching

  • Testing & performance: create a test matrix, performance telemetry, and regression gates for slow drivers, wrong numerics, and UI stalls

  • Instrumentation: expose runtime hooks (e.g., activations, attention) for research and security without sacrificing fused-kernel speed

Requirements

  • Deep, hands-on GPU programming experience in at least one of CUDA, Metal/MLX, or Vulkan compute, plus evidence of delivering work to a second backend

  • Hands-on speculative decoding experience (implemented or trained drafters) with understanding of acceptance rate drivers and end-to-end speed-up

  • Proven ability to make real inference workloads measurably faster (tokens/sec, latency, or memory) with explainable metrics

  • Strong command of quantization and how it affects model quality, including measurement methods

  • Correctness discipline: validate numerics against a reference and measure quality, not just speed

Benefits

  • Advanced computing resources (Apple Silicon, NVIDIA including Blackwell-class, AMD, Intel; cloud GPUs for training)

  • Technical ownership driving performance-critical on-device AI, working directly with AI leadership

  • Global impact: AI solutions used by hundreds of millions of browser users across diverse hardware

  • Research opportunities: contribute to open source and publish technical findings where appropriate

See similar jobs

Explore similar jobs and use your resume score to choose where to apply next

See more

Unlock Full Resume Report