Split-Apply-Combine
Split-apply-combine separates rows into groups, applies an operation, then combines results. An aggregation produces one result per group; a transformation such as transform("mean") can instead return a value for each original row. Here we focus on summaries: select a value column and aggregate it with mean, count, or another suitable function. SQL GROUP BY is a useful comparison for that aggregation case.
The grouping object on its own is not a result; it is a plan waiting for a function. Select the column you want summarized before aggregating, because asking for the mean of an entire table raises when a column holds text that has no mean. Older pandas versions dropped such columns silently, which is why an old notebook can produce a narrower table than its code suggests, and why numeric_only=True appears in code written across that change.
Named aggregation answers several questions in one pass: each output column gets a name you choose, along with the source column and the function it comes from. Any function that takes a Series and returns a single value can join the built-in names, which is how a percentile or a business-specific rule becomes one more column. Group by several keys when the question has two dimensions, and the labels become the combinations that actually occur.
Two habits keep grouped numbers honest. Report the group size beside every average, because a mean over one row is that row and not a finding. And know that rows whose key is missing are dropped by default, so a total from the summary can disagree with a total from the source; dropna=False keeps them as their own group. Note also that size() counts rows while count() on a column counts only its present values.
Further reading
See the worked examples in Pandas Foundations: Tables, Filtering, Grouping, and Joins.
Discover more from Insightful Data Lab
Subscribe to get the latest posts sent to your email.
