Python ETL Engineering for Production
Run IDs, batch IDs, ledger commits, structured events, failure classes, retries with jitter, and signal handling, demonstrated with code that ran.
-
-
Designing Maintainable Data Workflows
How to structure a data job so its inputs are visible, its effects are local, and changes can be checked: dependency injection, contract tests, configuration precedence and validation, and effective-configuration records.
-
Researching Errors and Organizing Analytical Work
A ride from 23:00 to 06:00 reports minus seventeen hours. Follow the evidence from a failing example to a tested rule, and record where that rule applies.
-
Software Engineering Habits for Data Work
Make data changes easier to verify and recover with tests, focused reviews, incremental refactoring, and useful operational documents.
-
Relational Database Internals
Explore how indexes, statistics, WAL, snapshots, and locks shape query behavior through SQLite examples and PostgreSQL concurrency mechanisms.
-
CTEs and Temporary Tables
A WITH clause is not a temporary table. Compare scope, lifetime, visibility, and staleness with DuckDB examples.
-
SQL for Data Engineers
Understand SQL through examples for filtering, cleaning, joins, aggregation, windows, and transactions in DuckDB.
-
Pandas Foundations
A spreadsheet of support tickets, five questions, one table. Filter it, group it, join it to a lookup table, and see where pandas quietly changes the answer.
-
Python for Data Engineers
Build installable Python data jobs with explicit configuration, input validation, tests, and a deliberate policy for rejected records and reruns.
-
Python Functions, Modules, and Classes
Copy-pasted cleaning logic breaks when the rule changes. Wrap it in a function, document it, import shared code, and let a class carry data with its checks.
-
Python Collections and Comprehensions
One name per value stops working at scale. Pick the container that matches the question you ask of the data, then build new collections with comprehensions.
-
Python Strings, Conditions, and Loops
Text rows rarely arrive tidy. Slice and strip them, decide with conditions, repeat the work with loops, and format the result for a reader.
-
Python Variables, Types, and Notebooks
A first Python session in a notebook: what a name refers to, why a value’s type decides what + does, and how the order of cell runs changes results.