Databricks Specialist - Finance Automation

About the role

Pinnacle DXB, a Dubai-based IT solutions company, is hiring a full-time Databricks Specialist for its Finance Automation Program. The specialist brings finance data from the ERP and other source systems into Databricks, cleans and transforms it, and prepares reliable datasets for reporting, Power BI, AI agents and the program's web applications. The client wants to close the role within 2-3 weeks.

Key responsibilities

• Design, build and maintain batch and incremental pipelines on Databricks
• Ingest data from the ERP, databases, files, APIs and other finance systems
• Organise data in bronze / silver / gold layers using Delta Lake
• Write efficient PySpark and SQL transformations
• Add data quality checks, monitoring, alerts and error handling to every pipeline
• Set up access control and governance with Unity Catalog
• Orchestrate with Databricks Workflows or Azure Data Factory, tuning performance and compute cost
• Prepare curated datasets for reporting, Power BI, AI agents and the web application, using Git and CI/CD

Required profile

• 4-7 years in data engineering, including at least 2 years hands-on with Databricks
• Finance-domain data experience strongly preferred
• Full-time, dedicated to the Finance Automation Program (one person per role)
• Location: Dubai (onsite) or remote
• Professional English required

Required skills

• PySpark and SQL
• Delta Lake, Unity Catalog and Databricks Workflows in production
• At least one cloud platform (Azure preferred)
• Data modelling and ETL/ELT design
• Git and basic CI/CD
• Experience integrating data from ERP or other enterprise systems; AP, AR, GL data preferred

Nice to have: Delta Live Tables, Auto Loader, structured streaming, Databricks Asset Bundles, Terraform, SAP data extraction, Power BI, Power BI ML, and a Databricks Certified Data Engineer credential. Compensation is discussed with shortlisted candidates.