Data Engineer

About the role

The Data Engineer will design, build and maintain data pipelines for the Events & Exhibitions ecosystem, turning scattered vendor data into a unified, AI‑ready dataset on Azure.

Key responsibilities

• Develop and operate ETL/ELT pipelines with Azure Data Factory to ingest data from REST APIs, webhooks and SFTP into Azure Data Lake Gen2 (Bronze layer).
• Transform, clean and deduplicate data using PySpark on Azure Databricks/Synapse and SQL to create Silver and Gold layers, including identity resolution for a “Golden Record”.
• Optimize SQL queries and Spark jobs to reduce compute costs, maintain data dictionaries, and enforce GDPR‑compliant masking and security.
• Collaborate with the Senior Data Manager and Events Tech team to build the semantic layer for Power BI and resolve data discrepancies.

Required profile

• 3–5 years of data engineering experience, preferably in cloud environments.
• Proven expertise with the core Azure stack: Data Factory, Synapse Analytics and Data Lake Gen2.
• Strong background in building Medallion Architecture pipelines (Bronze/Silver/Gold).

Required skills

• SQL (complex joins and performance tuning)
• Python / PySpark
• Azure Data Factory
• Azure Synapse Analytics
• Azure Data Lake Gen2
• REST API integration, JSON/XML handling