Parquet

Parquet is a typed columnar file format used for analytical data. Values from one column are stored together within a row group, allowing a reader to select columns without reconstructing every complete record.

A file footer locates column chunks and describes the schema. Chunks contain pages; optional statistics help readers exclude ranges. A row group whose maximum date is before the requested interval cannot contain a match, while overlapping ranges still need row filtering.

Encoding and compression have different jobs. Dictionary encoding replaces repeated values with references; a codec then compresses encoded bytes. Results depend on data and writer settings. Missing statistics or unsupported features can reduce skipping.

A Parquet file does not define an atomic snapshot across many files. Record the schema, units, supported readers, and publication rules separately. Check values as well as sizes when changing layout.

See Data Formats: Parquet, ORC, Avro, and Arrow for worked examples.

Further reading

• Official format documentation