Data Engineer – Build Scalable Data Pipelines

About the role

Deriv is hiring a data engineer to improve the data platform that supports trading, product analysis, compliance, business intelligence, and AI/ML across its global fintech network. The role focuses on creating stable, reliable pipelines that deliver fresh data for critical business processes.

Key responsibilities

• Build ETL/ELT pipelines in batch mode and real‑time, leveraging AI‑based development.
• Automate data quality validation, freshness checks, schema‑drift detection, lineage tracking, and alerting.
• Develop and manage data contracts for SLAs, SLOs, schema agreements, and producer‑consumer alignment.
• Optimize warehouse performance and cost through query tuning, partitioning, clustering, and orchestration.
• Create scalable data models, semantic layers, and abstractions to support analytics, compliance, AI/ML, and BI requirements.

Required profile

• 2–8 years of experience in data engineering, designing and managing production data pipelines.
• Hands‑on experience with GCP and BigQuery, including cloud data processing and warehouse optimization.
• Proficiency in Python and SQL, with experience in batch and stream processing.
• Experience using Airflow and familiarity with dbt or Dataform.
• Knowledge of data‑modeling concepts such as Kimball star schema, Data Vault, or Medallion.

Required skills

• GCP, BigQuery
• Python, SQL
• Airflow, dbt, Dataform
• Kafka or Pub/Sub for event‑based processing
• Data observability, schema management, lineage, CI/CD pipelines