Senior MLOps Engineer

About the role

The Institute of Foundation Models (IFM) is a research lab focused on building, understanding, deploying and risk-managing large-scale AI systems. As a Senior MLOps Engineer you will design, build and maintain machine learning infrastructure across training, inference and deployment pipelines, taking ownership of the model lifecycle from data ingestion to real-time serving. You will work with researchers and engineers to productionize LLMs, voice models and multimodal systems in Kubernetes-based environments.

Key responsibilities

• Design and manage scalable ML infrastructure on AWS using EKS, EC2, RDS, S3 and IAM-based access control.
• Build and maintain Kubernetes deployments for LLM and TTS inference using Helm, ArgoCD and Prometheus/Grafana monitoring.
• Implement and optimize model serving pipelines with vLLM, SGLang, TensorRT or similar frameworks.
• Develop CI/CD and MLOps automation for data versioning, model validation and deployment (GitHub Actions, Jenkins or AWS CodePipeline).
• Integrate OpenWebUI, Gradio or similar interfaces for model demos and evaluation tools.
• Collaborate with researchers to productize TTS, ASR and LLM-based chat systems.
• Ensure observability, cost optimization and reliability across environments, and contribute to tools for dataset curation, monitoring and retraining.
• Maintain infrastructure-as-code with Terraform and Helm charts, and support real-time multimodal workloads.

Required profile

• 4+ years of experience in MLOps, DevOps or cloud infrastructure engineering for ML systems.
• Strong proficiency in Kubernetes, Helm and container orchestration.
• Experience deploying ML models with vLLM, SGLang, TensorRT or Ray Serve.
• Solid experience with Python, Docker, Git and CI/CD pipelines.
• Understanding of model lifecycle management, data pipelines and observability tools (Grafana, Prometheus, Loki).
• Preferred: prior work in LLM deployment, fine-tuning pipelines or foundation model research, and familiarity with multi-GPU scheduling, NCCL optimization and HPC clusters.

Required skills

• Kubernetes, Helm, ArgoCD, Prometheus, Grafana, Loki
• AWS (EKS, EC2, S3, RDS, CloudWatch, IAM)
• vLLM, SGLang, TensorRT, Ray Serve, GitHub Actions, Jenkins, AWS CodePipeline
• Terraform, Docker, Git, Python
• Value added: ElevenLabs, Whisper, RVC, OpenWebUI, Gradio