Copy-on-Write and Chained Assignment
Chained assignment is the pattern of selecting part of a DataFrame and then assigning into the result: filter the rows, then set a column on what came back. It reads like an edit to the table and is not one. The selection produces a new object, the assignment lands on that object, and the object is discarded, so the original table is unchanged and no line of the code looks wrong.
Under pandas 3 copy-on-write, a distinct Series or DataFrame derived from another behaves independently for pandas assignments. Physical buffers may be shared until a write requires copying. Chained assignment cannot update the source and can emit a ChainedAssignmentError warning. This does not isolate two names pointing to the same object: alias = df still shares df. Nor does it recursively copy mutable Python objects stored inside object cells.
In older pandas with copy-on-write disabled, chained assignments could behave differently depending on the selection and might emit SettingWithCopyWarning. A warning was not a reliable test of whether the source changed. Use a single .loc[rows, column] = value assignment when the source is the intended target.
Two rules replace the guesswork. To modify the original, address it in a single step with loc, naming rows and column together, which tells pandas exactly which cells to write. To work on a subset without touching the source, take an explicit copy() and modify that. The same distinction applies when a function receives a table: state whether it returns a new object or modifies the one it was given, because under copy-on-write an in-place edit of a derived object no longer reaches the caller’s data.
References: pandas copy-on-write, pandas view versus copy. See it in use in Pandas Foundations: Tables, Filtering, Grouping, and Joins.
Discover more from Insightful Data Lab
Subscribe to get the latest posts sent to your email.
