Senior HPC Engineer – Large-Scale GPU Infrastructure

About the role

MBZUAI’s Institute of Foundation Models is looking for a Senior HPC Engineer to lead the design, operation, and evolution of its large‑scale GPU infrastructure that powers frontier AI research. The role combines technical leadership with hands‑on engineering to keep one of the world’s biggest AI‑focused supercomputing environments running at peak performance.

Key responsibilities

• Lead operation and optimisation of large‑scale GPU clusters.
• Drive reliability, scalability and performance improvements.
• Lead troubleshooting and root‑cause analysis of complex issues.
• Design and validate new cluster deployments and upgrades.
• Collaborate with researchers to optimise distributed AI training.
• Lead vendor engagement and technical reviews.
• Mentor junior engineers.
• Define monitoring, operational standards and capacity‑planning processes.
• Participate in major incident management and escalations.

Required profile

• Bachelor’s degree in Computer Science, Engineering, Applied Mathematics, Physics or related field (Master’s preferred).
• 5+ years of experience in HPC, Linux or cloud infrastructure, distributed systems or large‑scale production environments.
• Proven experience with Slurm and Linux administration.
• Strong background in troubleshooting compute, storage and networking systems.

Required skills

• Slurm workload manager
• Linux system administration
• GPU cluster operations
• NVIDIA CUDA, NCCL, NVLink, GPUDirect
• InfiniBand networking
• Weka, Lustre, BeeGFS storage platforms
• Azure, AWS or GCP cloud services
• Terraform, Ansible (Infrastructure‑as‑Code)
• PyTorch Distributed, Megatron‑LM, DeepSpeed, FSDP