Senior Software Engineer, Agentic Platform

Quick Facts

• Title: Senior Software Engineer, Agentic Platform

Description

You will build and own the runtime infrastructure that powers AI agents, combining LLM/ML expertise with distributed systems to orchestrate, execute, and deliver reliable, low-latency agent responses at enterprise scale. The role covers an agent orchestration engine, model routing/adaptation, distributed session management, event-driven pipelines, structured concurrency, observability, caching/state layers, and tool connectivity (including MCP and agent-to-agent). You’ll ensure correctness, observability, and resilience while agents run multi-step workflows and can pause for human input and resume.

Role Description & Responsibilities

• Build an agent orchestration engine (state machine) for long-running agent sessions across multiple LLM calls and tool invocations
• Implement model routing and adaptation, including selecting and fine-tuning/distilling LLMs for specific agent tasks with enterprise constraints (permissions, freshness, explainability)
• Develop distributed session management using lease-based ownership, conditional writes, heartbeat protocols, and checkpoint-based crash recovery
• Create an event-driven message pipeline with ordered delivery (FIFO), consumers, and real-time streaming via gRPC and Socket.IO
• Apply structured concurrency with fail-fast semantics and graceful cancellation for concurrent tasks per session
• Deliver observability infrastructure using OpenTelemetry, distributed trace context propagation, and custom span lifecycle management for long sessions
• Build caching and state layers using Redis and DynamoDB KV stores with per-org/per-bot scoping, batch read optimization, and hot-reload configuration
• Build MCP and agent-to-agent (A2A) connectivity: tool discovery, namespacing, and tool selection across large tool catalogs
• Improve token efficiency and context engineering to reduce cost/latency (prompt discipline, caching, speculative/parallel scheduling, multi-step reasoning token reduction)
• Support adoption and evaluation of open-weights LLMs across quantization levels by benchmarking accuracy/latency/cost for production readiness

Requirements

• 5+ years building production backend/infrastructure systems
• Strong in Python or Go (ideally both)
• Experience designing and operating systems that handle real traffic at scale
• Hands-on experience building production systems that use LLMs or other ML models
• Deep experience in at least 3 of: distributed systems, concurrent/async programming, event-driven architectures, database systems for infrastructure, observability, gRPC/protobuf
• Comfort with ambiguity and solving novel problems without textbook solutions

Benefits

• Base pay range: $161,300 - $274,200, plus equity (when applicable), variable/incentive compensation, and benefits
• Health plans, flexible spending accounts, 401(k) with company match, ESPP, matching donations, flexible time away, and family leave programs