Skip to content
BP
← All work

AI & Data

Retail Data Migration Platform

Incremental migration of retail transactions from legacy SQL Server into a layered Snowflake warehouse, orchestrated with Airflow and modelled with dbt.

Client
Engineering case study
Stack
Python, Airflow, dbt, Snowflake

The problem

Legacy on-prem SQL Server data was locked away from modern analytics, and full reloads were slow and error-prone.

The approach

  1. 01Modular Python ETL with watermark-based incremental loading and ROW_NUMBER deduplication.
  2. 02Snowflake RAW → STAGING → MARTS architecture with dbt transformations.
  3. 03Airflow DAG orchestration, automated data-quality tests, audit logging and failure alerting.

The outcome

A repeatable, observable pipeline pattern that moves legacy data to the cloud safely, and the data foundation AI work depends on.

Stack

  • Python
  • Airflow
  • dbt
  • Snowflake
  • SQL Server
  • Docker
  • PostgreSQL

More work

Free 30-minute discovery call

Have a problem AI could solve?

Tell me what's slowing your business down. You'll leave the call with ideas you can use — whether or not we work together.