
Certification Assessment Specialist
OpenAI
San Francisco, CA • 32m ago • Hybrid
Mid
Hybrid
207,000 - 230,000 USD/yr
Match your resume with similar open roles
Upload your resume to automatically match with active openings similar to this role.
Potential salary for Certification Assessment Specialist role
Improve your resume now
$207,000 – $230,000/year
Quick Facts
- Role: Certification Assessment Specialist (hands-on individual contributor)
- Focus: Designing and maintaining technical, role-based certification assessments
Description
As a Certification Assessment Specialist, you will design, build, test, launch, and maintain the hands-on assessments behind an OpenAI technical role-based certification. You will translate competency models, assessment blueprints, and scoring rubrics into realistic scenarios, assessment environments, and workflows that produce reviewer-verifiable evidence and traceable scores. You will also build and validate model-assisted scoring with qualified human reviewers and ensure end-to-end reliability across live and virtual delivery.
Responsibilities
- Translate assessment blueprints into realistic scenarios, candidate instructions, and hands-on tasks aligned to rubrics
- Define assessment environment requirements and build/configure repeatable provisioning and reset workflows (accounts, tools, data, permissions)
- Build and maintain pipelines to capture submissions and scoring evidence, apply scoring rules, and route results for review with sensitive-data protections
- Build and test model-assisted scoring with human review; compare model scores to calibrated expert judgments and manage routing rules
- Test assessments end to end (environment setup, evidence capture, scoring), using dry runs/pilots and failure recovery
- Develop delivery integration requirements; partner to test candidate access, submission, and results handoffs
- Prepare assessments for launch with platform and operations partners; document dependencies and troubleshooting
- Maintain versioned tasks, environments, and scoring configurations; use feedback and data to update or retire tasks
- Investigate technical issues in delivery or candidate review; reproduce, document, and verify fixes with partners
Requirements
- Experience building performance-based, hands-on assessments/labs/simulations for technical roles through the full lifecycle (testing, launch, maintenance)
- Scripting and automation experience to build and troubleshoot assessment environments
- Ability to use blueprints and rubrics to define realistic tasks and distinguish competent performance from task completion
- Experience capturing submissions/evidence and producing traceable, reviewer-scoring outputs
- Experience implementing or validating model-assisted scoring/evaluation against expert human judgments and knowing when to route to human review
- Experience translating requirements into end-to-end testable integrations with external delivery providers
- Experience using pilot results, candidate feedback, and scoring/delivery data to improve assessment quality
- Knowledge of accessibility and protecting candidate evidence and confidential assessment content
Benefits
- Opportunity to shape how technical skills are demonstrated through trusted, role-based certifications
- Work across certification, measurement, engineering, operations, and technical subject matter experts to support durable AI skills adoption
- Own assessments from prototype through ongoing maintenance across live and virtual delivery
See similar jobs
Explore similar jobs and use your resume score to choose where to apply next
See more
Technical Certification Lead
OpenAI
Senior
Technology
San Francisco, CA · Hybrid
$266k - $295k/yr
32m ago

Certification Architect
OpenAI
Senior
Technology
San Francisco, CA · Hybrid
$266k - $295k/yr
6 days ago

Certification Operations and Policy Program Manager
OpenAI
Mid
Technology
San Francisco, CA · Hybrid
$207k - $230k/yr
32m ago

Software Engineer, AI Quality & Evaluation
Bask Health
Mid
Technology
Remote
N/A
3 days ago

AI Evaluation Researcher
Figma
Senior
Technology
San Francisco, CA · On-site
$169k - $250k/yr
10 days ago
Tailor Resume to fit the Job Offer
Use AI to find out how well the skills on your resume fit this job description.

AI Developer Education Specialist
OpenAI
Mid
Technology
San Francisco, CA · Hybrid
$207k - $230k/yr
3 days ago
KI-Spezialist (m/w/d) Prozesse, Sicherheit & Innovation
syskomp gehmeyr GmbH
Mid
Technology
Amberg, Germany · On-site
€30k - €60k/yr
7 days ago

Applied AI Engineer (Enterprise)
OpenAI
Mid
Technology
On-site
$251k - $278k/yr
13 days ago
Senior AI & Integration Engineer
Elliott Davis
Senior
Technology
Remote
N/A
13 days ago
Unlock Full Resume Report
ATS Pass
Missing keywords
Tailored AI suggestions
Job match analysis
Interview-focused insights