AI Architect - Data Engineering

About the role

Cognizant is hiring an AI Architect in Dubai, focused on building and maintaining data pipelines and data products. The engineer takes ownership from design through production support, writes clean, well-tested code and collaborates with teammates and stakeholders.

Key responsibilities

• Design, build and maintain batch and real-time data pipelines.
• Develop data products using Databricks Lakeflow Declarative Pipelines, PySpark and SQL.
• Build streaming solutions with Spark Structured Streaming and Lakeflow declarative pipelines.
• Optimise pipelines for scalability, reliability and performance.
• Develop automated tests and take part in code reviews.
• Contribute to CI/CD and engineering best practices.
• Monitor, troubleshoot and support production workloads.
• Use AI-assisted development tools to improve productivity while keeping high engineering standards.

Required profile

• Owns work from design through production support.
• Writes clean, maintainable, well-tested code and builds reusable, production-ready solutions.
• Strong analytical and problem-solving skills.
• Nice to have: experience in Agile or SAFe environments.

Required skills

• Databricks Lakeflow Declarative Pipelines and DXQ, DB Genie, MCP Server
• Apache Spark (PySpark), Spark Structured Streaming, Auto Loader, Unity Catalog, Delta Lake
• Python, SQL
• Azure Event Hubs or equivalent event streaming platforms
• Git, unit testing, CI/CD
• Nice to have: performance tuning and Spark optimisation, data modelling, Azure DevOps, AI coding assistants such as GitHub Copilot or Cursor