Data Engineer – Data Quality (Onsite, Abu Dhabi)

About the role

We are seeking an experienced Data Engineer to join our Data & AI team in Abu Dhabi. The role focuses on building robust, Python‑based data pipelines and ensuring high data quality across our modern data platform.

Key responsibilities

• Design, develop, and maintain end‑to‑end data pipelines using Python (pandas, PySpark, etc.).
• Implement ETL/ELT processes following the Medallion architecture (bronze, silver, gold).
• Create and optimise SQL transformations, stored procedures, and views for validation and reconciliation.
• Develop data ingestion frameworks for flat files, APIs, databases, and streaming sources.
• Apply data‑quality checks covering completeness, validity, consistency, uniqueness, accuracy, and timeliness.
• Monitor pipeline health, set up alerts, and troubleshoot production issues.
• Contribute to CI/CD pipelines, automated testing, and version control for data workloads.
• Document pipelines, data models, and operational runbooks clearly.

Required profile

• Strong Python development experience with production data pipelines.
• Solid SQL skills for complex queries and performance tuning.
• Hands‑on experience designing scalable ETL/ELT processes.
• Good understanding of Medallion architecture and data lake/lakehouse patterns.
• Familiarity with cloud data platforms (Azure, AWS, or GCP) and both relational and non‑relational databases.

Required skills

• Python (pandas, PySpark)
• SQL
• ETL/ELT design
• Medallion architecture
• Data orchestration tools
• Azure, AWS, GCP
• Informatica IDQ, Microsoft Purview, Great Expectations
• Informatica EDC, Informatica Axon