Databricks Developer – Data Engineering & AI

About the role

Tata Consultancy Services (TCS) is seeking a Databricks Developer to design and implement scalable data solutions on the Databricks platform. You will work on enterprise‑level analytics, reporting, and AI/ML initiatives, ensuring high performance and cost efficiency.

Key responsibilities

• Design, develop, and optimise data engineering solutions on Databricks.
• Build high‑performance Apache Spark jobs using PySpark and Spark SQL.
• Develop and manage Databricks notebooks, jobs, and workflows.
• Construct and maintain robust ETL/ELT pipelines with Python and Spark.
• Ensure data quality, reliability and scalability of pipelines.
• Create analytics‑ready datasets and support BI/reporting platforms.
• Prepare data for AI/ML workloads, supporting feature engineering and model scoring.
• Collaborate with Data Science, Analytics and Platform teams while adhering to governance and security standards.

Required profile

• Strong expertise in SQL, including complex queries and performance tuning.
• Proficiency in Python for data engineering and analytics.
• Hands‑on experience with Apache Spark (PySpark / Spark SQL).
• Proven experience as a Databricks Developer building data pipelines.
• Experience supporting analytics, reporting and AI/ML data workloads.

Required skills

• SQL
• Python
• Apache Spark
• PySpark
• Spark SQL
• Databricks
• ETL/ELT pipeline development
• Data engineering
• AI/ML data preparation