Certification Assessment Specialist

OpenAI

San Francisco, CA • 32m ago • Hybrid

Mid

Hybrid

207,000 - 230,000 USD/yr

  1. Select offer
  2. Add resume
  3. Analyze
  4. Improve
  5. Apply
  6. Auto-send · soon

Match your resume with similar open roles

Upload your resume to automatically match with active openings similar to this role.

PDF up to 10 MB

Your resume is private & secure

Or go to the original job offer

Potential salary for Certification Assessment Specialist role

Improve your resume now

$207,000 – $230,000/year

Quick Facts

  • Role: Certification Assessment Specialist (hands-on individual contributor)
  • Focus: Designing and maintaining technical, role-based certification assessments

Description

As a Certification Assessment Specialist, you will design, build, test, launch, and maintain the hands-on assessments behind an OpenAI technical role-based certification. You will translate competency models, assessment blueprints, and scoring rubrics into realistic scenarios, assessment environments, and workflows that produce reviewer-verifiable evidence and traceable scores. You will also build and validate model-assisted scoring with qualified human reviewers and ensure end-to-end reliability across live and virtual delivery.

Responsibilities

  • Translate assessment blueprints into realistic scenarios, candidate instructions, and hands-on tasks aligned to rubrics
  • Define assessment environment requirements and build/configure repeatable provisioning and reset workflows (accounts, tools, data, permissions)
  • Build and maintain pipelines to capture submissions and scoring evidence, apply scoring rules, and route results for review with sensitive-data protections
  • Build and test model-assisted scoring with human review; compare model scores to calibrated expert judgments and manage routing rules
  • Test assessments end to end (environment setup, evidence capture, scoring), using dry runs/pilots and failure recovery
  • Develop delivery integration requirements; partner to test candidate access, submission, and results handoffs
  • Prepare assessments for launch with platform and operations partners; document dependencies and troubleshooting
  • Maintain versioned tasks, environments, and scoring configurations; use feedback and data to update or retire tasks
  • Investigate technical issues in delivery or candidate review; reproduce, document, and verify fixes with partners

Requirements

  • Experience building performance-based, hands-on assessments/labs/simulations for technical roles through the full lifecycle (testing, launch, maintenance)
  • Scripting and automation experience to build and troubleshoot assessment environments
  • Ability to use blueprints and rubrics to define realistic tasks and distinguish competent performance from task completion
  • Experience capturing submissions/evidence and producing traceable, reviewer-scoring outputs
  • Experience implementing or validating model-assisted scoring/evaluation against expert human judgments and knowing when to route to human review
  • Experience translating requirements into end-to-end testable integrations with external delivery providers
  • Experience using pilot results, candidate feedback, and scoring/delivery data to improve assessment quality
  • Knowledge of accessibility and protecting candidate evidence and confidential assessment content

Benefits

  • Opportunity to shape how technical skills are demonstrated through trusted, role-based certifications
  • Work across certification, measurement, engineering, operations, and technical subject matter experts to support durable AI skills adoption
  • Own assessments from prototype through ongoing maintenance across live and virtual delivery

See similar jobs

Explore similar jobs and use your resume score to choose where to apply next

See more

Unlock Full Resume Report