Data Engineer

About the role

Timely, reliable data pipelines turn platform designs into business value. The Data Engineer will build and operate Azure‑based pipelines that move data from source systems into a lakehouse and deliver it to analytics and AI consumers. The role works closely with data architects, quality engineers, business analysts and AI engineers.

Key responsibilities

• Design and develop batch, incremental, CDC and streaming ingestion pipelines into the Bronze layer using Azure Data Factory and Microsoft Fabric.
• Create Silver‑ and Gold‑layer transformations with SQL, PySpark and Spark notebooks, applying cleansing, enrichment and business‑rule logic.
• Implement dimensional models, SCDs, surrogate keys and data marts according to architectural designs.
• Integrate data‑quality checks, validation, quarantine and reconciliation in partnership with senior data‑quality engineers.
• Publish metadata and lineage to Microsoft Purview and ensure pipelines are scheduled, parameterised and monitored.
• Optimise performance and cost by tuning partitioning, file formats (Delta/Parquet) and compute usage.
• Apply security controls, masking, encryption and credential management to meet UAE PDPL and related regulations.
• Maintain code in GitHub, deploy via CI/CD pipelines using Azure DevOps and support production operations and knowledge transfer.

Required profile

• 4+ years of data‑engineering experience, preferably in UAE government or regulated sectors.
• Advanced proficiency in SQL, Python and PySpark/Spark.
• Hands‑on experience with Azure Data Factory, Microsoft Fabric, Databricks or Synapse.
• Strong understanding of lakehouse/medallion architecture, Delta/Parquet formats and dimensional modelling.
• Experience with CI/CD pipelines, source control and DataOps practices.
• Familiarity with Microsoft Purview metadata and lineage, and data‑quality tooling.
• Excellent English communication; Arabic is a plus.

Required skills

• Azure Data Factory
• Microsoft Fabric
• Databricks
• Synapse
• Python
• SQL
• PySpark / Spark
• Delta Lake
• Parquet
• Microsoft Purview
• GitHub
• Azure DevOps