Senior Data Infrastructure Engineer (Scraping & Scale)

About the role

We are looking for a Senior Data Infrastructure Engineer to design and scale high‑performance web‑scraping and data ingestion systems. The role is fully remote and focuses on building core infrastructure that collects millions of social‑media profiles, posts and video records daily.

Key responsibilities

• Lead development and maintenance of scalable data collection pipelines for platforms such as Instagram, TikTok, YouTube and X.
• Design and manage web‑scraping infrastructure using residential proxies, TLS fingerprinting, browser automation and session rotation.
• Build fault‑tolerant, high‑throughput pipelines and orchestrate distributed queues with Temporal, Ray, Apache Kafka or Celery.
• Architect storage solutions for structured and unstructured data using Parquet, Apache Iceberg, Snowflake and Amazon S3.
• Implement monitoring, alerts and recovery mechanisms to handle platform changes, blocking patterns and proxy failures.

Required profile

• At least 4 years of experience in high‑volume web scraping, data engineering or reverse engineering.
• Strong expertise in overcoming advanced anti‑bot systems (Cloudflare, Akamai, PerimeterX) using TLS impersonation, browser automation and proxy management.
• Proven ability to design scalable, fault‑tolerant systems and troubleshoot complex ingestion challenges.

Required skills

• Python or Go programming.
• Browser‑automation frameworks: Playwright, Puppeteer, Scrapy, Selenium.
• Distributed systems and task queues: Temporal, Ray, Apache Kafka, Celery, Redis.
• SQL and modern analytical data lakes/warehouses (Parquet, Apache Iceberg, Snowflake, Amazon S3).

What we offer

• Fully remote work arrangement.
• Full‑time employment.