Senior Software Engineer, Agentic Platform

Quick Facts

• Role: Senior Software Engineer (Agentic Platform)

• Focus: Building agent runtime infrastructure for enterprise LLM-based agents at scale

Description

You’ll build and own the runtime systems that orchestrate, execute, and deliver responses from AI agents to millions of enterprise users in real time. The platform supports multi-step workflows, tool calling, waiting for human input, and resuming—while maintaining correctness, observability, and low latency. You’ll combine LLM/ML and distributed-systems expertise to make agents reliable in production.

Responsibilities

• Develop an agent orchestration engine using a state machine for long-running agent sessions

• Implement model routing and adaptation, including selecting and fine-tuning/distilling LLMs for tasks with enterprise constraints

• Build distributed session management with lease-based ownership, heartbeat protocols, and crash recovery via checkpointing

• Create an event-driven message pipeline for ordered delivery and real-time streaming

• Use structured concurrency for concurrent tasks per session with fail-fast semantics and graceful cancellation

• Deliver observability infrastructure with OpenTelemetry instrumentation and distributed tracing

• Implement caching and state layers using Redis and DynamoDB KV stores with scoped data and hot-reload configuration

• Enable tool and agent connectivity for LLMs, including MCP and agent-to-agent (A2A) with discovery and tool selection

• Optimize token efficiency and context engineering to reduce cost and latency

• Stand up and evaluate open-weights models (including quantization levels) and benchmark accuracy, latency, and cost

Requirements

• 5+ years building production backend/infrastructure systems

• Strong in Python or Go (ideally both)

• Experience designing and operating systems handling real traffic at scale

• Hands-on experience building production systems using LLMs or other ML models

• Deep experience in at least 3 of: distributed systems, concurrent/async programming, event-driven architectures, database systems for infrastructure, observability, gRPC/protobuf, machine learning

• Comfort with ambiguity

Benefits

• Opportunity to build production agent runtime infrastructure that powers agentic AI for enterprise employees at scale