Chunked Checksum
A chunked checksum compares summaries for bounded key ranges rather than transferring every row for comparison. A summary can pair a row count with a hash of canonical row content. Unequal summaries identify ranges to investigate; matching hashes are evidence, not mathematical proof of equality.
Compare the same columns at an aligned snapshot or applied source position. Define column order, row ordering, NULLs, time zones and numeric representation. Include both source and destination key domains: if the source’s highest key was deleted, source-only bounds can hide the leftover destination row.
Computing a checksum still reads the data unless suitable summaries already exist. The saving may be transfer or targeted repair, not source I/O. Before replacing a differing range, verify the reference boundary and coordinate newer writes so repair does not restore stale data.
Reference: Python SHA-256 API.
Discover more from Insightful Data Lab
Subscribe to get the latest posts sent to your email.
