Python and NumPy Basics
Every code block in this series is NumPy or PyTorch. This is the array vocabulary they both assume.
-
-
Python ETL Engineering for Production
Run IDs, batch IDs, ledger commits, structured events, failure classes, retries with jitter, and signal handling, demonstrated with code that ran.
-
Designing Maintainable Data Workflows
How to structure a data job so its inputs are visible, its effects are local, and changes can be checked: dependency injection, contract tests, configuration precedence and validation, and effective-configuration records.
-
Structured Logging
Emit stable event fields for data jobs, separate candidate counts from committed results, and keep JSON formatting distinct from redaction and delivery.
-
Dependency Injection
Pass a workflow its reader or publisher from outside, test the dependency contract, and keep fake-adapter evidence separate from real integration behavior.
-
Configuration Validation
Check configuration keys, types, ranges, and relationships before work begins, and distinguish a valid setting from a safe destination or correct rule.
-
Minimal Reproducible Example
Build a small runnable example that preserves a failure, state expected and actual results, and replace real data without removing the bug.
-
Error Message and Traceback
Learn how to read an error message and traceback, trace a failing value through calls, and keep useful details when searching for an explanation.
-
Researching Errors and Organizing Analytical Work
A ride from 23:00 to 06:00 reports minus seventeen hours. Follow the evidence from a failing example to a tested rule, and record where that rule applies.
-
pandas Series
Learn what a pandas Series is, how one column differs from a narrow table, why its index controls alignment, and how it skips missing values.
-
pandas DataFrame
Learn what a pandas DataFrame holds, why each column has its own type, what the index is for, and where a table in memory stops being the right tool.
-
Pandas Foundations
A spreadsheet of support tickets, five questions, one table. Filter it, group it, join it to a lookup table, and see where pandas quietly changes the answer.
-
Boolean Masking
Learn how a boolean mask filters rows in pandas, why conditions need parentheses and element-wise operators, and how missing values drop out silently.
-
Unit and Integration Tests
Test small behavior with explicit expectations, then verify real component boundaries; distinguish a passing example from a general correctness claim.
-
Context Manager
Read with as a scoped setup-and-cleanup contract, check what a library actually closes or commits, and distinguish cleanup from data correctness.
-
Generator and Iterator
Understand one-pass iteration, deferred execution, and why a generator saves memory only when its consumer and retained state stay bounded.
-
Type Hints and Runtime Validation
Use type hints to describe calls and runtime validation to check incoming values; neither replaces the other or defines business rules automatically.
-
Python Package and Wheel
Distinguish import packages, installable distributions, and wheels, and check the built artifact outside the source tree.
-
Dependency Lock File
Separate dependency declarations from resolved locks, understand platform selections, and distinguish checking a lock from updating it.
-
Virtual Environment
Understand what a Python virtual environment isolates, how to select its interpreter, and why an environment folder is not a portable deployment.
-
Python for Data Engineers
Build installable Python data jobs with explicit configuration, input validation, tests, and a deliberate policy for rejected records and reruns.
-
Refactoring
Learn what refactoring changes and what it must not, how duplication becomes one function, and why a way to verify behavior comes first.
-
Class and Instance
Learn how a class defines a type, what init and self do, how an attribute differs from a method, and when a function is the better choice.
-
Docstring
Learn how a docstring documents a function’s purpose, inputs, and result, how it differs from a comment, and where help and tooling read it from.
-
Variable Scope
Learn which names a function can see, why assigning inside a body makes a name local, what UnboundLocalError means, and when global is the wrong fix.
-
Parameter and Argument
Learn how parameters differ from arguments, how positional and keyword arguments match, and why a mutable default value persists between calls.
-
Python Function
Learn how def defines a function, why return differs from print, what a call actually runs, and when a function returns None.
-
Python Functions, Modules, and Classes
Copy-pasted cleaning logic breaks when the rule changes. Wrap it in a function, document it, import shared code, and let a class carry data with its checks.
-
Mutable and Immutable Objects
Learn which Python objects can change in place, why two names can share one list, how copies differ, and where mutable defaults cause bugs.
-
List Comprehension
Learn the comprehension syntax for lists, dicts, and sets, how filtering works, when a plain loop reads better, and what the speed claim rests on.
-
Python Set
Learn how a Python set keeps unique values, why it has no positions, which operators compare two sets, and when membership beats a list.
-
Python Dictionary
Learn how a dictionary maps keys to values, which keys are allowed, when to use get instead of indexing, and how insertion order behaves.
-
Python Tuple
Learn why a tuple is immutable, how unpacking assigns its fields, why the comma makes the tuple, and when a tuple beats a list.
-
Python List
Learn how a Python list stores ordered values, which methods add and remove them, and how remove and pop fail differently.
-
Python Collections and Comprehensions
One name per value stops working at scale. Pick the container that matches the question you ask of the data, then build new collections with comprehensions.
-
String Formatting
Learn how f-strings and format() insert values, how width, alignment, and precision work, and why formatting changes display but not the value.
-
Loop and Iteration
Learn when a for loop or a while loop fits, how range builds sequences, what break and continue do, and why a loop can never end.
-
Conditional Statement
Learn how if, elif, and else choose one branch, why test order and indentation matter, and what happens when no branch runs.
-
Boolean Logic and Comparison
Learn what comparisons return in Python, how strings order by code point, what counts as true, and why and/or return operands, not just booleans.
-
String Indexing and Slicing
Learn zero-based and negative positions, half-open slices, why an index can fail while a slice quietly clamps, and how to locate substrings.
-
Python String
Learn how Python strings store text, why they cannot be changed in place, how escape sequences work, and which operators apply to text.
-
Python Strings, Conditions, and Loops
Text rows rarely arrive tidy. Slice and strip them, decide with conditions, repeat the work with loops, and format the result for a reader.
-
Data Type
Learn how Python data types decide which operations work, when numbers convert automatically, and why explicit conversion of text can fail.
-
Python Variable
Learn what assignment binds in Python, why a name has no fixed type, how shared lists behave, and which variable names are valid or risky.
-
Jupyter Notebook
Learn how a Jupyter notebook combines code, output, and notes, why execution order changes results, and how to rerun it from a fresh kernel.
-
Python Variables, Types, and Notebooks
A first Python session in a notebook: what a name refers to, why a value’s type decides what + does, and how the order of cell runs changes results.