Senior LLMOps / AI Platform Engineer

About the role

We are looking for a Senior LLMOps / AI Platform Engineer to build, deploy, optimize, and operate production‑grade large language model (LLM) and generative AI infrastructure. The role blends LLM inference, GPU optimization, Kubernetes, cloud services, observability, retrieval‑augmented generation (RAG) and overall AI platform engineering.

Key responsibilities

• Deploy and operate self‑hosted LLMs using vLLM, SGLang and Ollama.
• Optimize LLM inference for latency, throughput, concurrency, GPU memory usage and cost.
• Manage GPU workloads across multiple NVIDIA GPUs.
• Deploy and maintain AI services on Kubernetes / AWS EKS using Docker and Helm.
• Implement reliability mechanisms such as health checks, monitoring, automated recovery and model refresh strategies.
• Set up observability with Langfuse/LangSmith, OpenTelemetry, Prometheus and Grafana.
• Deploy and tune RAG pipelines, embedding models and vector databases like Qdrant or Milvus.
• Support AI agents and workflows built with LangChain and LangGraph.
• Build and maintain CI/CD pipelines for AI services and infrastructure.
• Troubleshoot production issues across LLMs, GPUs, Kubernetes, networking and AI applications.

Required skills

• Strong Python and FastAPI development.
• Experience with vLLM, Hugging Face and self‑hosted LLM deployment.
• Proficiency in Kubernetes, Docker, Helm and AWS services.
• Deep knowledge of NVIDIA GPU inference and performance optimization.
• Familiarity with LangChain, LangGraph, LangSmith and RAG concepts.
• Hands‑on with vector databases (e.g., Qdrant) and embedding models.
• Skills in LLM observability, monitoring and logging tools.
• Experience with PostgreSQL, Redis, GitHub Actions and CI/CD pipelines.
• Strong production troubleshooting abilities.