AIOps Engineer (AI Infrastructure & Orchestration)

Quick Facts

• Focus: AI Infrastructure & Orchestration for LLM inference (vLLM) on Kubernetes/OpenShift with bare-metal GPUs
• Core topics: GPU resource management, model lifecycle automation, autoscaling, observability, secure API routing, usage tracking

Description

You will design, deploy, and maintain vLLM inference services on OpenShift/Kubernetes running on bare-metal GPU infrastructure. The role centers on optimizing NVIDIA GPU utilization, automating the model lifecycle (onboarding, versioning, deployment, hot-swapping, rollback), and operating reliable, secure AI inference at scale. You will also implement monitoring/alerting, token consumption tracking for quota/chargeback needs, and production-ready API gateway controls.

Responsibilities

• Deploy and maintain vLLM inference services on OpenShift/Kubernetes
• Manage NVIDIA GPU resources, including partitioning and allocation for multiple models and tenants
• Automate model lifecycle management from private registries (e.g., Hugging Face Enterprise, S3)
• Implement HPA tuned to demand and GPU utilization; optimize vLLM serving parameters for performance, throughput, and efficiency
• Build observability (metrics, logging, tracing) and instrument vLLM endpoints (latency, throughput, token consumption, error rates)
• Create usage tracking by API key/user/team/department to support quotas and chargeback/showback
• Build Grafana dashboards for infrastructure health, GPU utilization, inference performance, availability, and consumption metrics
• Configure Prometheus and Alertmanager proactive monitoring and alerting
• Implement API gateway capabilities: authentication, authorization, rate limiting, intelligent routing
• Ensure secure operation via network segmentation and ingress/egress controls; maintain audit logging
• Collaborate with AI Engineering, Platform Engineering, Security, and Infrastructure teams; troubleshoot incidents and conduct root-cause analysis

Requirements

• 5+ years of experience in DevOps, SRE, Platform Engineering, or Infrastructure Operations
• At least 2 years of hands-on experience supporting MLOps, AI Infrastructure, or LLM platforms
• Production experience administering Kubernetes and OpenShift
• Proven experience deploying and operating vLLM-based inference platforms in production
• Strong understanding of LLM serving concepts (paged attention, continuous batching, inference optimization)
• Deep knowledge of NVIDIA GPU technologies and troubleshooting (CUDA drivers, NVIDIA Container Toolkit)
• Hands-on experience with Prometheus, Grafana, OpenTelemetry, and ELK Stack
• Strong Python skills for automation and operational tooling
• Experience with Bash and Linux systems administration
• Familiarity with GitLab CI, Jenkins, ArgoCD, and Infrastructure-as-Code practices
• Analytical/problem-solving skills in distributed environments and strong communication/collaboration skills

Benefits

• Medical, sport and life insurance packages at preferential terms
• Access to Percipio and Coursera learning platforms
• Preferential access to products and services
• Employment contract-based cooperation
• Training or financial bonus for recommending new employees