Data Engineer - Web Scraping

About the role

We are looking for a Data Engineer specialized in web scraping to join our team in the United Arab Emirates. You will build scalable data collection solutions and end‑to‑end pipelines that feed business‑critical insights, working closely with analysts and cross‑functional engineers.

Key responsibilities

• Design, develop and maintain automated web scrapers for structured and unstructured sources.
• Clean, transform, validate and manipulate large datasets using Python and Pandas.
• Build and operate data ingestion pipelines into databases or data warehouses.
• Schedule, monitor and optimise scraping workflows with Apache Airflow.
• Implement quality‑control checks to ensure data integrity and availability.
• Troubleshoot pipeline issues and resolve time‑sensitive production incidents.
• Enhance internal tools and automation frameworks to improve operational efficiency.
• Collaborate with cross‑functional teams to deliver reliable data solutions.

Required profile

• Bachelor’s or Master’s degree in Computer Science or a related technical field.
• 2‑4 years of professional software development experience.
• Strong problem‑solving mindset and proactive attitude.
• Experience working with web technologies such as HTML, JavaScript and APIs.

Required skills

• Python programming
• SQL and relational databases
• Pandas library
• Web‑scraping tools: Selenium, Scrapy, XPath, Fiddler, Postman
• Workflow orchestration: Apache Airflow
• Understanding of HTML, JavaScript and API protocols