Partition Overwrite
Partition overwrite replaces all existing rows in a declared partition with its complete desired contents. The partition might be an order date or a region. It is not an instruction to update only the changed rows: every row that should remain in the replacement scope must be supplied.
If yesterday has 100 orders and the extract contains two changed orders, deleting yesterday and inserting those two loses 98. Conversely, if the source has deleted every order for that date, the correct replacement is empty. Pass the dates separately from the rows so an empty result still removes the old partition. Validate source completeness before treating empty as authoritative.
Commit deletion and insertion together using a supported transaction or table-version protocol. Coordinate overlapping writers; atomic writes can still publish an older snapshot after a newer one. When an order moves dates, both the old and new partitions need attention. Repeating the same complete snapshot can be idempotent while repeating an incomplete snapshot remains consistently wrong.
Reference: SQLite atomic commit.
Discover more from Insightful Data Lab
Subscribe to get the latest posts sent to your email.
