Copy-on-Write and Chained Assignment
Chained assignment is the pattern of selecting part of a DataFrame and then assigning into the result: filter the rows, then set a column on what came back. It reads like an edit to the table. When rows are filtered first, and under pandas 3 copy-on-write, it is not one: the selection produces a new object, the assignment lands on that object, and the object is discarded, so the original table is unchanged and no line of the code looks wrong. Older pandas behaved less predictably, as described below.
Under pandas 3 copy-on-write, a distinct Series or DataFrame derived from another behaves independently for pandas assignments. Physical buffers may be shared until a write requires copying. Chained assignment cannot update the source and can emit a ChainedAssignmentError warning. This does not isolate two names pointing to the same object: alias = df still shares df. Nor does it recursively copy mutable Python objects stored inside object cells.
In older pandas with copy-on-write disabled, chained assignments could behave differently depending on the selection and might emit SettingWithCopyWarning. A warning was not a reliable test of whether the source changed. Use a single .loc[rows, column] = value assignment when the source is the intended target.
Two rules replace the guesswork. To modify the original, address it in a single step with loc, naming rows and column together, which tells pandas exactly which cells to write. To work on a subset without touching the source, take an explicit copy() and modify that. The same distinction applies when a function receives a table: state whether it returns a new object or modifies the one it was given, because under copy-on-write an in-place edit of a derived object no longer reaches the caller’s data.
Further reading
See the worked examples in Pandas Foundations: Tables, Filtering, Grouping, and Joins.
Discover more from Insightful Data Lab
Subscribe to get the latest posts sent to your email.
