Research Scientist – Training Infrastructure Engineer

About the role

You will build distributed systems that train Luma's large‑scale multimodal models across thousands of GPUs, enabling researchers to focus on innovation on top of reliable, efficient, and scalable infrastructure.

Key responsibilities

• Design, implement, and optimise efficient distributed training systems for models across thousands of GPUs.
• Research and implement advanced parallelisation techniques such as FSDP, Tensor Parallel, Pipeline Parallel, and Expert Parallel.
• Build monitoring, visualisation, and debugging tools for large‑scale training runs.
• Optimise training stability, convergence, and resource utilisation across massive clusters.

Required profile

• Extensive experience with distributed PyTorch training and foundation‑model parallelism.
• Deep understanding of GPU clusters, networking, and storage systems.
• Familiarity with communication libraries like NCCL and MPI.
• Strong Linux systems administration and scripting skills.

Required skills

• PyTorch
• CUDA
• FSDP, Tensor Parallel, Pipeline Parallel, Expert Parallel
• NCCL, MPI
• Linux administration & scripting
• Containerisation and orchestration (e.g., Docker, Kubernetes)
• Cloud infrastructure