VLM Engineer

About the role

The Technology Innovation Institute’s AI Research Centre is seeking a VLM Engineer to advance its vision-language model research. You will work with a world-class team that created the Falcon models, focusing on integrating large language models with visual understanding.

Key responsibilities

• Conduct comprehensive ablation studies on vision models to evaluate component impact.
• Perform data ablation research to identify optimal data types and structures for vision-language training.
• Develop and apply robust evaluation protocols across benchmarks and real-world scenarios.
• Train and optimise vision-language models, integrating LLMs with architectures such as CLIP.

Required profile

• PhD in deep learning or a closely related field.
• Proven research and development experience in vision-language models.
• Strong publication record in top-tier conferences (highly desirable).

Required skills

• Machine learning expertise, especially vision-language models and LLMs.
• Deep knowledge of CLIP and similar architectures.
• Proficiency with distributed training and multi-GPU optimisation.
• Hands-on experience with PyTorch.
• Analytical ability to design and interpret ablation studies.
• Dataset curation and processing for vision and language tasks.