Software Engineer II (SSO Intelligence)

Quick Facts

• Software Engineer II focused on SSO Intelligence
• Own production evaluation and release systems

Description

You will own the evaluation and release systems that determine what ships and when. Build production evaluation pipelines to assess changes to models, prompts, tools, and agent architecture, including systems designed for asynchronous, nondeterministic agent behavior. You will also design and operate online A/B testing and controlled experiments to support safe release decisions, including shadow evaluation, canary releases, dashboards, and rollback.

Responsibilities

• Build automated production evaluation pipelines for model/prompt/tool/agent architecture changes (realistic environments, deployed agent runs, grading, baseline comparisons)
• Design reusable evaluation workflows that support asynchronous, nondeterministic agent behavior and standardize the evaluation/release path
• Design online evaluation and A/B testing systems (shadow evaluation, canary releases, controlled experiments)
• Implement treatment assignment, exposure logging, guardrail metrics, stopping criteria, rollback, and define quality/safety/latency/cost/reliability metrics
• Create dashboards to surface regressions and provide readouts explaining what changed and what to monitor after launch
• Turn agent prototypes into production capabilities by owning end-to-end integration (APIs, data contracts, versioning, telemetry, release tests, and production requirements for tenancy, durability, scaling, and failure recovery)

Requirements

• 3+ years of non-internship professional software development experience
• 2+ years of non-internship experience designing or architecting new and existing systems (design patterns, reliability, scaling)
• 1+ years of software development engineer or related occupational experience
• 1+ years designing and developing large-scale, multi-tiered, multi-threaded, embedded or distributed software applications/tools/systems/services using C#, C++, Java, or Perl
• 1+ years of Object Oriented Design experience
• Bachelor’s degree or foreign equivalent in Computer Science, Engineering, Mathematics, or related field
• Experience programming with at least one software programming language

Benefits

• Health insurance (medical, dental, vision, prescription)
• Basic Life & AD&D insurance and option for Supplemental life plans
• EAP and Mental Health Support; Medical Advice Line
• Flexible Spending Accounts
• Adoption and Surrogacy Reimbursement coverage
• 401(k) matching
• Paid time off
• Parental leave