Object Version and ETag

An object key names a storage location; a version ID identifies a particular stored version when versioning supports it. An ETag is a validator associated with an object representation, not a universal content checksum. In Amazon S3 its meaning depends on upload and encryption details, so do not assume every ETag is the file’s MD5.

A notification about version A may arrive after version B has replaced the key. Fetching only the key can therefore read B while recording A as processed. Fetch the named version if retained and permitted, and verify its checksum under the source contract. A version ID is opaque; lexical order does not establish which version is newer.

S3 event sequencers can order supported events for the same key, not different keys. Compare unequal-length hexadecimal strings after padding or numeric conversion. A latest-state consumer may reject older notices after successful publication; an archive may need every version. Deduplication and progress must be durable and coordinated with output when retries or concurrent workers are possible.

Further reading

See the worked examples in Worked API and file ingestion lab.


Discover more from Insightful Data Lab

Subscribe to get the latest posts sent to your email.

Similar Posts

Questions, corrections, or additional insights?

This site uses Akismet to reduce spam. Learn how your comment data is processed.