Senior LLMOps / AI Platform Engineer

About the role

IC Markets Global is seeking a Senior LLMOps / AI Platform Engineer to design, deploy, and operate production‑grade large language model (LLM) and generative AI infrastructure. The role blends LLM inference, GPU optimization, Kubernetes, cloud services, observability, and Retrieval‑Augmented Generation (RAG) engineering.

Key responsibilities

• Deploy and operate self‑hosted LLMs using vLLM, SGLang, and Ollama.
• Optimize LLM inference for latency, throughput, concurrency, GPU memory, KV cache, and cost.
• Manage GPU workloads across multiple NVIDIA GPUs.
• Deploy and maintain AI services on Kubernetes / AWS EKS with Docker and Helm.
• Implement reliability mechanisms: health checks, monitoring, automated recovery, and model refresh strategies.
• Set up observability using Langfuse/LangSmith, OpenTelemetry, Prometheus, and Grafana.
• Deploy and tune RAG systems, embedding models, and vector databases such as Qdrant and Milvus.
• Support AI agents and workflows built with LangChain and LangGraph.
• Build and maintain CI/CD pipelines for AI services and infrastructure.
• Troubleshoot production issues across LLMs, GPUs, Kubernetes, networking, and AI applications.

Required skills

• Strong Python and FastAPI experience.
• vLLM, Hugging Face, and self‑hosted LLM deployment.
• Kubernetes, Docker, Helm, and AWS.
• NVIDIA GPU inference and performance optimization.
• LangChain / LangGraph / LangSmith.
• RAG, embeddings, and vector databases (Qdrant, Milvus).
• LLM observability and monitoring (Langfuse, OpenTelemetry, Prometheus, Grafana).
• PostgreSQL and Redis.
• GitHub Actions and CI/CD pipelines.
• Strong production troubleshooting skills.