Senior Data Engineer

About the role

The Abu Dhabi Department of Economic Development is seeking a Senior Data Engineer to design, build, and maintain scalable data pipelines that support analytics, BI, and AI/ML initiatives. You will work closely with data architects, product owners, and business teams to deliver reliable, high‑performance data solutions.

Key responsibilities

• Design, develop, and operate robust ETL/ELT pipelines.
• Optimize data ingestion, transformation, and processing workflows for performance and scalability.
• Implement data quality checks, validation, and monitoring within pipelines.
• Collaborate with data architects to implement lakehouse, data lake, and data warehouse architectures.
• Integrate data from APIs, databases, files, and SaaS platforms.
• Support real‑time streaming pipelines using Kafka and event‑driven designs.
• Apply CI/CD, Git, and DevOps practices to ensure reliable deployments.
• Mentor junior engineers and conduct code reviews.

Required profile

• 5+ years of experience in data engineering, data platforms, or analytics engineering.
• Proven ability to build and operate production‑grade data pipelines in complex, multi‑source environments.
• Strong understanding of data modeling, analytics‑ready schemas, and data governance principles.
• Bachelor's degree in Computer Science, Engineering, or a related field; certifications are a plus.

Required skills

• SQL (performance tuning)
• Python
• Scala
• Apache Spark
• Airflow
• Dagster
• Azure Data Factory
• Kafka
• Data Lakes, Data Warehouses, Lakehouse architectures
• Cloud platforms: AWS, Azure, GCP
• Git and DevOps practices