Copy-on-Write and Chained Assignment

Chained assignment is the pattern of selecting part of a DataFrame and then assigning into the result: filter the rows, then set a column on what came back. It reads like an edit to the table and is not one. The selection produces a new object, the assignment lands on that object, and the object is discarded, so the original table is unchanged and no line of the code looks wrong.

Under pandas 3 copy-on-write, a distinct Series or DataFrame derived from another behaves independently for pandas assignments. Physical buffers may be shared until a write requires copying. Chained assignment cannot update the source and can emit a ChainedAssignmentError warning. This does not isolate two names pointing to the same object: alias = df still shares df. Nor does it recursively copy mutable Python objects stored inside object cells.

In older pandas with copy-on-write disabled, chained assignments could behave differently depending on the selection and might emit SettingWithCopyWarning. A warning was not a reliable test of whether the source changed. Use a single .loc[rows, column] = value assignment when the source is the intended target.

Two rules replace the guesswork. To modify the original, address it in a single step with loc, naming rows and column together, which tells pandas exactly which cells to write. To work on a subset without touching the source, take an explicit copy() and modify that. The same distinction applies when a function receives a table: state whether it returns a new object or modifies the one it was given, because under copy-on-write an in-place edit of a derived object no longer reaches the caller’s data.

References: pandas copy-on-write, pandas view versus copy. See it in use in Pandas Foundations: Tables, Filtering, Grouping, and Joins.


Discover more from Insightful Data Lab

Subscribe to get the latest posts sent to your email.

Similar Posts

Questions, corrections, or additional insights?

This site uses Akismet to reduce spam. Learn how your comment data is processed.