ORC

ORC is a typed columnar file format for analytical scans. It groups rows into stripes, with column streams and index information inside each stripe. File metadata lets readers locate those structures.

Projection selects needed columns. Statistics and supported indexes can eliminate ranges that cannot match a predicate. Optional bloom filters can prove absence for a value; a possible match still needs verification because false positives are possible.

A stripe is a grouping unit, not a business partition. Stripe sizes, codecs, and reader support affect memory and read behavior. Compare formats with representative queries and explicit settings rather than assuming one extension is always smaller or faster.

An ORC file alone supplies no multi-file transaction protocol. A table system must coordinate publication, updates, and reader snapshots. Reading selected columns successfully does not demonstrate index pruning.

See Data Formats: Parquet, ORC, Avro, and Arrow for worked examples.

Further reading


Discover more from Insightful Data Lab

Subscribe to get the latest posts sent to your email.

Similar Posts

Questions, corrections, or additional insights?

This site uses Akismet to reduce spam. Learn how your comment data is processed.