Data Engineer - AI & Cloud Pipelines

About the role

Inception42, a G42 company, is looking for a Data Engineer to build scalable data pipelines and develop intelligent AI solutions using Azure Agent Foundry, OpenAI services, LangChain, Databricks and modern cloud architectures. The role suits someone with strong data engineering foundations and hands-on experience designing and deploying enterprise-grade AI systems.

Key responsibilities

• Develop and deploy AI agents using Azure Agent Foundry, OpenAI, LangChain and n8n.
• Build RAG pipelines, embeddings, vector database integrations and knowledge graph solutions.
• Design and maintain scalable data pipelines, ETL/ELT workflows and data architectures.
• Build and maintain APIs and data services for AI and analytics workloads.
• Optimise ML workflows on Azure Databricks and integrate with Azure Fabric.
• Ensure data quality, reliability, governance and compliance across all systems.
• Collaborate with data scientists, ML engineers and product teams.
• Support MLOps and DevOps, including CI/CD, model lifecycle management and containerised deployments.

Required profile

• Bachelor's degree in Computer Science, Engineering, Data Science or a related field.
• 3+ years of experience in Data Engineering, AI Engineering or ML Engineering.
• Experience delivering scalable AI/data solutions in cloud environments (Azure preferred).
• Knowledge of big data tools (Spark, Hadoop, Flink) and modern data storage (SQL, NoSQL).

Required skills

• Python, SQL, PySpark and ML/AI frameworks.
• Azure Agent Foundry, OpenAI APIs, LLMOps, RAG systems and vector databases.
• Azure Databricks, REST/GraphQL APIs, Kafka/RabbitMQ, Docker/Kubernetes.
• Data modelling, ETL and data quality best practices.

What we offer

• An open, diverse and inclusive culture with a global vision.
• Learning and development through structured training programmes and high-tech projects.
• A competitive remuneration package with healthcare, education support and leave benefits.