Senior Data Engineer

About the role

We are looking for a Senior Data Engineer to design and implement defense‑grade Data Lakehouse architectures that serve as the single source of truth for AI‑driven supply‑chain intelligence. The role is based in Abu Dhabi and focuses on building secure, high‑performance pipelines that ingest both structured ERP data and unstructured technical documents.

Key responsibilities

• Architect and deploy Data Lakehouse environments (Databricks/Delta Lake) to support real‑time analytics and generative AI tools.
• Lead Workstream 1: Data & Platform Foundations, integrating SAP S/4HANA, Ariba and PLM systems.
• Build ingestion pipelines for high‑volume transactional data and external market feeds (e.g., S&P Global, Orbis).
• Design and manage vector databases (Weaviate, Milvus) for Retrieval‑Augmented Generation applications.
• Implement RBAC, audit logging and data redaction to meet export‑control and on‑premise security standards.

Required profile

• 5+ years of data engineering experience, with at least 2 years building pipelines for machine learning or generative AI.
• Proven experience extracting data from complex ERP environments, especially SAP S/4HANA and SAP Ariba.
• Strong background in supply‑chain, manufacturing or defense domains and familiarity with Bill of Materials structures.
• Ability to work within agile “Sprint Zero” frameworks while adhering to strict governance and stage‑gate reviews.

Required skills

• Python
• SQL
• Apache Spark
• Kafka
• Airflow
• Databricks / Delta Lake
• PostgreSQL
• Weaviate or Milvus
• dbt, Dagster or Prefect
• Docker & Kubernetes
• CI/CD pipelines