Software Engineer, Pretraining Data

Quick Facts

• Role: Software Engineer, Pretraining Data

Description

Build data systems that convert raw web, code, multimodal, and acquired data into training-ready datasets for frontier pretraining runs. Own pretraining data pipelines, orchestration, and tooling so data iteration is fast, reliable, observable, and reproducible at scale, with clear signals for data quality and lineage. Partner across crawling, data quality, acquisition, and initial training teams to measure improvements in loss, evals, and model capability.

Responsibilities

• Data Platform

  • Build the platform that turns raw data into training-ready datasets for pretraining
  • Own pipelines, orchestration, and tooling for fast and reliable data iteration at scale
  • Provide signals for data quality, lineage, freshness, and pipeline health
  • Partner with teams to turn new data ideas into measurable training improvements

• Data Quality

  • Build and own high-throughput, fully telemetered pipelines with end-to-end traceability
  • Detect breaks or drift before training runs
  • Train and ship data processing models (classify, rank, filter, clean, identify) at extreme throughput
  • Run scaling-ladder experiments to turn dataset intuition into hard evidence
  • Partner to find missing/low-quality sources and close the loop on loss and downstream eval gains
  • Write performance-critical code and design careful experiments

• Crawling

  • Build and scale web crawling systems to discover, schedule, fetch, and parse high-quality documents
  • Improve URL seeding, scoring, and fair host scheduling
  • Increase crawl success and parsing quality; reduce antibot failures; improve extractors
  • Harden crawl infrastructure for availability, recovery, and ingestion lag; automate pipeline delivery
  • Work with AI agents to ensure new coverage becomes better tokens for training

Requirements

• Minimum 3 years of software engineering experience

• Experience building distributed systems and/or large-scale data infrastructure

• Ability to work in person onsite at the Palo Alto office

Benefits

• Annual salary range: $160,000 - $250,000 USD plus equity

• Comprehensive medical, vision, and dental coverage

• 401(k) retirement plan

• Short- and long-term disability insurance

• Life insurance and additional discounts/perks