Model Policy Manager, Emerging AI Misuse

OpenAI

San Francisco, CA • 1h ago • Hybrid

Mid

Hybrid

207,000 - 335,000 USD/yr

  1. Select offer
  2. Add resume
  3. Analyze
  4. Improve
  5. Apply
  6. Auto-send · soon

Match your resume with similar open roles

Upload your resume to automatically match with active openings similar to this role.

PDF up to 10 MB

Your resume is private & secure

Or go to the original job offer

Potential salary for Model Policy Manager, Emerging AI Misuse role

Improve your resume now

$207,000 – $335,000/year

Quick Facts

  • Role: Model Policy Manager, Emerging AI Misuse
  • Focus: Threat modeling, model behavior policies, and evaluations for emerging AI misuse risks
  • Collaboration: Research, engineering, security, and product
  • Work pattern: Office three days per week; optional work from home Thursdays and Fridays (relocation support available)

Description

As AI capabilities increase, this role builds threat models for how people could misuse AI in digital and physical settings and defines when models should help, limit assistance, or refuse. You will translate safety expertise into model instructions, training examples, and evaluation criteria, then analyze model outputs, red-team findings, and deployment data to identify where misuse is enabled or legitimate work is blocked. You’ll partner across teams to apply findings to training, deployment decisions, and monitoring.

Responsibilities

  • Build and update threat models as capabilities evolve
  • Assess plausible misuse actions, required expertise/resources, and how AI changes barriers
  • Define model responses for help, limitation, or refusal in harmful vs legitimate contexts
  • Turn expert safety judgment into instructions, training examples, and evaluations
  • Convert threat models into evaluation scenarios and grading guidance
  • Analyze outputs, red-team findings, and deployment data to find misuse and gaps
  • Collaborate with research, engineering, security, and product to apply safeguards to training, deployment, and monitoring

Requirements

  • Hands-on experience building, testing, or assessing dual-use technologies or threat modeling grounded in real-world systems
  • Deep practical experience in a relevant domain and judgment to bring in experts from other domains
  • Ability to evaluate whether test-demonstrated capability changes real-world misuse potential
  • Ability to translate technical evidence and uncertainty into clear policy boundaries and measurable evaluation criteria
  • Experience analyzing gold sets, evaluation results, and training data, including label inconsistencies and annotator disagreement effects
  • Comfort working hands-on with model data and evaluation results, including inspecting examples and analyzing failure patterns and data quality
  • Ability to use empirical evidence to develop and refine safety policies and safeguards
  • Ability to work effectively across disciplines and update judgment as capabilities and evidence change
  • Comfort working in a fast-paced, collaborative research environment with shifting priorities

Benefits

  • Relocation support
  • Work from the office three days a week with optional work from home Thursdays and Fridays

See similar jobs

Explore similar jobs and use your resume score to choose where to apply next

See more

Unlock Full Resume Report