Separating Storage, Compute, and Metadata
Splitting a warehouse replaces one product with five choices. See what object storage cannot do, how a table format fills the gap, and what copies cost.
-
-
File Compaction
Combine small files while preserving records, publish replacements safely, and evaluate request savings alongside rewrite and retention costs.
-
Dataset Publication Manifest
Publish an exact immutable file set through a manifest, reject duplicate or foreign references, and retain files needed by pinned readers.
-
Conditional Object Write
Use create-only and compare-and-swap preconditions for object writes, handle conflicts, and keep publication order separate from source freshness.
-
Designing an Object-Storage Data Lake
Build a data lake with discoverable datasets, immutable runs, conditional publication, safe cleanup, and layouts matched to queries.
-
Ingesting Data from APIs, Files, and Storage
Collect partner data with explicit identity, revision, retry, and completeness rules.