Senior Principal Machine Learning Engineer (Security/Advanced Threat Protection)

PaloAlto Networks

Santa Clara, CA • 1h ago • On-site

Senior

On-site

190,400 - 308,000 USD/yr

  1. Select offer
  2. Add resume
  3. Analyze
  4. Improve
  5. Apply
  6. Auto-send · soon

Match your resume with similar open roles

Upload your resume to automatically match with active openings similar to this role.

PDF up to 10 MB

Your resume is private & secure

Potential salary for Senior Principal Machine Learning Engineer (Security/Advanced Threat Protection) role

Improve your resume now

$190,400 – $308,000/year

Quick Facts

  • Role: Senior Principal Machine Learning Engineer (Advanced Threat Protection)
  • Focus: Chief technical authority for AI/ML innovation and architectural strategy in cloud security (SECaaS)

Description

Lead architectural strategy and design excellence for security cloud systems, defining a multi-year engineering roadmap and guiding foundation model adoption. Architect large-scale telemetry ingestion/feature pipelines and production-grade agentic ecosystems, and take hands-on ownership of frontier model deployment, diagnosis, and remediation in real environments. Partner across product, security research, data science, and infrastructure to translate threat landscapes into executable designs.

Responsibilities

  • Set technical direction by evaluating AI research and steering investments toward production-viable architecture
  • Drive engineering culture via RFC standards, architectural review boards, and mentoring Staff/Principal engineers
  • Lead end-to-end system design for distributed, fault-tolerant cloud systems processing real-time security data streams
  • Architect agentic frameworks with dynamic planning, stateful memory persistence, secure tool execution, and deterministic guardrails
  • Build data/telemetry pipelines for ingestion, preprocessing, and feature engineering using Apache Spark, SQL, and BigQuery on Google Cloud
  • Guide model architecture choices across LLMs, SLMs, and multimodal systems to optimize cost, latency, privacy, and efficacy
  • Diagnose production model failures (drift, context issues, alignment collapse, hallucinations) and remediate via fine-tuning and parameter-efficient methods
  • Debug distributed GPU training/inference issues including CUDA OOM, NCCL overhead, and kernel performance stalls

Requirements

  • 10+ years progressive software engineering and systems design with technical leadership in high-availability enterprise-scale platforms
  • Ability to author complex system designs from scratch and manage architectural trade-offs, including legacy-to-modern migrations
  • Proven cross-functional alignment across product, security research, data science, SecOps, and infrastructure
  • Deep hands-on knowledge of modern AI architectures (Transformers, attention, tokenization) and production experience deploying/serving LLMs and SLMs
  • Expert in model triage and remediation pipelines (data curation, synthetic data generation, SFT, DPO, RL, LoRA/PEFT)
  • Practical debugging and profiling of distributed GPU workloads (memory leaks, concurrency issues, cluster communication bottlenecks)
  • Advanced Python for AI/ML pipelines and scientific computing
  • Production-level fluency in Go or Java for low-latency concurrent microservices
  • Strong data engineering skills: Apache Spark, distributed processing, SQL, BigQuery, and Google Cloud orchestration (Dataproc, GKE)
  • BS/MS/PhD in a relevant technical discipline

Benefits

  • Starting base salary range: $190,400.00 - $308,000.00 per year (plus potential restricted stock units and bonus)
  • Immigration sponsorship eligibility: Yes

See similar jobs

Explore similar jobs and use your resume score to choose where to apply next

See more

Unlock Full Resume Report