Data Engineer - Azure Databricks

About the role

techcarrot FZ LLC is hiring a Data Engineer in Dubai. The role builds robust, reusable and scalable data pipelines with the Microsoft and Databricks stack, works closely with squads and other engineering teams, and supports the integration of prototypes into core intelligence products.

Key responsibilities

• Collect progress updates from the Squad regularly and turn them into weekly project status reports shared with Squad Leads.
• Take part in early state design and feature definition activities.
• Implement and maintain robust, reusable and scalable data pipelines.
• Collaborate with engineers across several teams to integrate proven prototypes into core products.
• Communicate complex data insights clearly to non-technical stakeholders.
• Develop and manage L3-level project plans with tasks, milestones and dependencies.

Required profile

• Minimum 5-7 years of practical experience.
• Bachelor's degree in computer science, software engineering, information technology or a related field.
• In-production experience with the Azure cloud stack.
• Team player with strong communication skills; project management experience is an advantage.

Required skills

• Azure Databricks with Spark for ETL pipelines.
• Relational and non-relational databases.
• Data analysis, manipulation and statistical modelling with Spark, Python, Scala, SQL or similar.
• JIRA dashboards based on tasks.
• Azure DevOps for CI/CD deployment of pipelines (advantage).