Data Engineer – AI Solutions

About the role

Inception42 is seeking a Data Engineer to design and operate scalable data pipelines that power AI‑driven products across the G42 ecosystem. The role combines core data‑engineering expertise with hands‑on development of intelligent agents using Azure Agent Foundry, OpenAI services and modern cloud technologies.

Key responsibilities

• Develop and deploy AI agents leveraging Azure Agent Foundry, OpenAI, LangChain and N8N.
• Build Retrieval‑Augmented Generation (RAG) pipelines, embeddings, vector‑database integrations and knowledge‑graph solutions.
• Design, implement and maintain scalable ETL/ELT workflows and data architectures on Azure.
• Create and manage APIs and data services that support AI and analytics workloads.
• Optimize machine‑learning workflows on Azure Databricks and integrate them with Azure Fabric.
• Ensure data quality, reliability, governance and compliance across all platforms.
• Collaborate with data scientists, ML engineers and product teams to deliver intelligent features.
• Support MLOps and DevOps processes, including CI/CD, model lifecycle management and containerised deployments.

Required profile

• Bachelor’s degree in Computer Science, Engineering, Data Science or a related field.
• Minimum 3 years of experience in data engineering, AI engineering or ML engineering.
• Proven track record delivering scalable AI or data solutions in cloud environments, preferably Azure.
• Strong problem‑solving abilities and effective communication skills.

Required skills

• Python, SQL, PySpark
• Azure Databricks, Azure Agent Foundry, Azure Fabric, Azure cloud services
• OpenAI APIs, LangChain, LLMOps, RAG systems, vector databases
• Big‑data tools: Spark, Hadoop, Flink
• Data storage: SQL, NoSQL
• API technologies: REST, GraphQL
• Messaging: Kafka, RabbitMQ
• Containerisation and orchestration: Docker, Kubernetes