Transformation as Code with dbt
Shipping application code changes behavior; shipping transformation code changes history too. What that asks of models, tests, documentation, and deployment.
-
-
Pipeline Orchestration and Scheduling
Running at 07:00 is not the same as running when the data is there. How orchestrators decide what runs, in what order, and what happens when a step fails.
-
Pipeline Correctness: Duplicates and Loss
Every job succeeded and the total is still wrong. Here is where duplicates and missing rows come from, what “exactly once” really promises, and how to verify.
-
Incremental Loading and CDC
A nightly job filtering on updated_at looks right until rows quietly go missing. Here is how each change-capture method works, and what none of them can see.
-
ETL vs ELT, Batch vs Streaming
“We need a pipeline” hides several decisions. See when to copy data or call its owner, where to transform, how fresh to be, and who pays and owns each part.