Skip to content
Muhammad Safyan

Data engineering

Data engineering: one source of truth for your numbers

I build data warehouses and ETL pipelines that pull your ERP, e-commerce and operations systems into one model — so finance, operations and leadership read the same numbers, refreshed automatically.

Is this you?

This is a good fit if…

  • Your reports are assembled by hand in spreadsheets every week.
  • Shopify, NetSuite and your other systems disagree about the same sale.
  • Dashboards are slow or nobody trusts them.
  • Your cloud data bill is higher than it should be.

What you get

What data engineering covers

Data warehouses

Dimensional models (facts, dimensions, snapshots) on Azure SQL or Snowflake, designed around the questions the business asks.

ETL and ELT pipelines

Azure Data Factory and Snowpark pipelines with incremental loads, MERGE logic, schema-drift handling and alerting.

Integrations

Shopify (REST and GraphQL), NetSuite, UKG, Apparel Magic, Keepa and other APIs reconciled into one model.

Dashboards and reporting

Power BI dashboards for revenue, backlog, inventory and utilisation with automated weekly snapshots.

Tools I use for this

  • Azure Data Factory
  • Azure SQL
  • T-SQL
  • Snowflake
  • Snowpark
  • Python
  • Power BI
  • Databricks

Proof

Data engineering case studies

75%+

catalogue coverage, up from 15%

E-commerce intelligence platform · Data engineering

From 15% to 75% catalogue coverage on Snowflake

faster database loads
10–20xfaster database loads
products per run
100k+products per run
Read the case study

FAQ

Common questions

Which data platforms do you use?

Azure Data Factory with Azure SQL for most warehouse builds, Snowflake with Snowpark (Python) for large catalogue and enrichment pipelines, and Power BI for reporting.

Can you fix numbers that do not match between systems?

Yes. On one retail project I traced a 1–3% monthly sales gap to gift-card refunds that Shopify's REST and GraphQL APIs classify differently, and rebuilt the pipeline to reconcile them.

How long does a data warehouse take?

A first version covering one or two source systems and the key reports is usually a few weeks. Each further source is added as its own increment.

Will it be expensive to run?

Cost is part of the design. On one Azure project I cut monitoring costs from about $1,000 a month to $0.10 by removing redundant alert rules.

Need help with data engineering?

Tell me what you are building and where it hurts, and I will come back with questions or a plan.