Data Engineer – Cloud, AI & Lakehouse

About the role

We are looking for an experienced Data Engineer to join a high‑performing data and analytics team that drives enterprise‑wide transformation initiatives. You will work with modern cloud‑native architectures, large‑scale datasets, AI use cases, and advanced data technologies that power analytics, business intelligence, and machine learning solutions.

Key responsibilities

• Design, develop, and optimise scalable batch and real‑time data pipelines.
• Build and maintain enterprise data lakes, lakehouse architectures, and curated analytical datasets.
• Write high‑performance SQL and Python code for large‑scale data processing workloads.
• Implement robust data‑quality, validation, monitoring, and observability frameworks.
• Develop cloud‑native data solutions and infrastructure‑as‑code deployments.
• Collaborate closely with Data Scientists, BI teams, and business stakeholders to deliver trusted, governed data products.
• Support AI and machine‑learning initiatives by preparing high‑quality training datasets and enabling MLOps workflows.

Required profile

• 5+ years of professional experience in data engineering.
• Strong background designing and building ETL/ELT pipelines at scale.
• Advanced expertise in SQL and Python.
• Hands‑on experience with cloud data platforms, preferably Azure.
• Experience with modern data‑warehousing and lakehouse architectures.
• Familiarity with CI/CD and infrastructure automation practices.

Required skills

• Microsoft Fabric
• Azure Data Factory (ADF)
• Databricks
• PySpark
• Python
• Advanced SQL
• Lakehouse Architecture
• Delta Lake
• Azure Synapse Analytics
• Terraform
• Infrastructure as Code (IaC)
• GitHub Actions
• Azure DevOps
• Data Governance
• Data Quality Frameworks
• Change Data Capture (CDC)
• API Integrations
• Streaming Data Pipelines
• MLOps Enablement
• Metadata Management
• Data Lineage
• Scala (nice‑to‑have)
• Hadoop ecosystem (nice‑to‑have)
• Machine‑Learning pipeline experience (nice‑to‑have)