Senior Data Engineer - Trading & Brokerage
FinTop Consulting · Doubaï
Job description
About the role
Fintop Consulting has partnered with a pioneer in the financial industry and is looking for a hands-on Senior Data Engineer with strong expertise in Azure Databricks, PySpark, SQL and Kafka-based streaming technologies. The role covers building and managing scalable data pipelines, handling complex data integrations and ensuring data quality and reliability in a high-volume financial data environment. It suits someone with strong experience in trading or brokerage data, production support, performance optimization and end-to-end pipeline ownership.
Key responsibilities
- Design, develop and maintain data ingestion and transformation pipelines using Azure Databricks, PySpark, SQL and Delta Live Tables (DLT).
- Build and manage Kafka-based streaming integrations using Confluent Cloud and its connectors.
- Develop and enhance API-based ingestion frameworks and third-party data integrations.
- Own data pipelines from source ingestion through Bronze and Silver layers, ensuring data quality, reliability and SLA adherence.
- Troubleshoot production issues, perform root cause analysis and implement permanent fixes.
- Optimize Databricks workloads, Spark performance and cloud infrastructure costs.
- Implement data governance and deployment practices using Unity Catalog, Databricks Workflows and CI/CD.
- Collaborate with business and technical stakeholders on data requirements and integration standards.
- Mentor junior engineers and contribute to code reviews, technical design and engineering best practices.
Required profile
- 10+ years of hands-on experience in data engineering.
- Strong production experience with Azure Databricks, Apache Spark, PySpark, SQL and Delta Lake.
- Practical experience with Delta Live Tables, Unity Catalog and Databricks Workflows.
- Hands-on experience with Apache Kafka, preferably Confluent Cloud and Kafka Connect.
- Proven experience optimizing Databricks performance and operational costs.
- Experience supporting production data platforms and resolving critical data issues.
- Good to have: MetaTrader (MT4/MT5) back-office data, trading bridges and liquidity platforms, enterprise CRM systems, and a background in brokerage, trading or regulated financial services.
Required skills
- Azure Databricks, Apache Spark, PySpark, SQL, Delta Lake and Delta Live Tables
- Apache Kafka, Confluent Cloud, Kafka Connect
- Data modelling, incremental loads, CDC, partitioning and performance tuning
- API-based data ingestion frameworks
- Unity Catalog, Databricks Workflows, CI/CD
Questions fréquentes
Why are you reporting this job?
Explore further
Salaries, guides and searches in the United Arab Emirates.
Salaries by job title
Apply in 30 seconds
Enter your email to apply. An account will be created automatically.
By continuing, you accept our terms of use.
Already have an account? Login
A question about this job?
Ask it here: you will get the full job summary by e-mail, right away.
Published 11 hours ago
Expires 1 month from now
7 views · 0 interested
Boost your chances
Upload your CV — we will match you with relevant openings.
Analyzing your CV...
FinTop Consulting
Doubaï