Senior LLMOps / AI Platform Engineer

About the role

The Senior LLMOps / AI Platform Engineer will design, build, and operate production‑grade large language model (LLM) and generative AI infrastructure for a financial services firm.

Key responsibilities

• Deploy and operate self‑hosted LLMs using vLLM, SGLang and Ollama.
• Optimize LLM inference for latency, throughput, concurrency, GPU memory, KV cache and cost.
• Manage GPU workloads across multiple NVIDIA GPUs.
• Deploy AI services on Kubernetes / AWS EKS with Docker and Helm.
• Implement reliability mechanisms, health checks, monitoring, automated recovery and model refresh strategies.
• Set up observability using Langfuse/LangSmith, OpenTelemetry, Prometheus and Grafana.
• Deploy and optimise Retrieval‑Augmented Generation (RAG) systems, embedding models and vector databases such as Qdrant and Milvus.
• Support AI agents and workflows built with LangChain and LangGraph.
• Build and maintain CI/CD pipelines for AI services and infrastructure.
• Troubleshoot production issues across LLMs, GPUs, Kubernetes, networking and AI applications.

Required profile

• Strong experience with Python and FastAPI.
• Hands‑on expertise in deploying self‑hosted LLMs and using Hugging Face models.
• Deep knowledge of Kubernetes, Docker, Helm and AWS cloud services.
• Proven ability to optimise NVIDIA GPU inference performance.
• Experience with LangChain, LangGraph and LangSmith tooling.

Required skills

• Python, FastAPI
• vLLM, Hugging Face, self‑hosted LLM deployment
• Kubernetes, Docker, Helm, AWS
• NVIDIA GPU inference and performance tuning
• LangChain, LangGraph, LangSmith, Langfuse
• RAG, embeddings, vector databases (Qdrant, Milvus)
• Observability: OpenTelemetry, Prometheus, Grafana
• PostgreSQL, Redis
• GitHub Actions, CI/CD pipelines