Senior Databricks Data Platform Engineer

About the role

DXC Luxoft is looking for a Senior Databricks Data Platform Engineer to join its core Lakehouse Platform team in Abu Dhabi. The role focuses on engineering and evolving the Databricks-based Lakehouse platform itself, building reusable services, automation frameworks and enterprise integrations that support ADIA's Data & AI ecosystem. It is not a traditional data engineering role focused on ETL jobs or datasets.

Key responsibilities

• Collaborate with stakeholders on requirements and sprint planning, then design and implement technical solutions on the Lakehouse platform, prototyping and productising new capabilities.
• Develop reusable Python libraries, SDKs and internal frameworks, and expose platform capabilities to support the Data Products strategy.
• Build automation for provisioning, governance controls, configuration management and operational standards.
• Integrate the platform with incident and monitoring systems (e.g. ServiceNow), identity and access management solutions, and observability platforms (e.g. Dynatrace, Prometheus, Grafana), using secure API-based integrations.
• Maintain unit and integration tests, support QA acceptance testing, and act as a third-line engineer for complex production incidents.
• Work with cloud and infrastructure teams to deliver a secure, scalable and reliable platform architecture.

Required profile

• Minimum 10 years of experience in IT/data.
• Minimum 3 years of Databricks experience from a platform enablement perspective.
• 3+ years designing, writing and maintaining APIs for platform automation, serverless workflows, deployment pipelines, release and repository management.
• Advanced Python skills for production-grade library and service development.
• Experience designing and integrating REST APIs, and advanced SQL for troubleshooting and performance tuning (Oracle, MS SQL).
• Clear, precise and articulate communication.

Required skills

• Git and CI/CD workflows.
• Monitoring and observability tools such as ServiceNow, Prometheus and Grafana.
• Scheduling and orchestration tools such as Stonebranch, Control-M and Airflow.
• Data quality and governance frameworks such as Great Expectations.
• Cloud fundamentals: DNS, certificates, identity and access management, networking, load balancing.
• Nice to have: Docker, Kubernetes, Terraform, investment data domain knowledge, English C1.