Column Pruning

Column pruning removes columns that a query does not need from parts of its plan. With a suitable columnar reader, reducing the required schema can avoid reading unused column data.

The final SELECT list is not the whole requirement. A query that returns amount but filters country still needs country to evaluate the predicate unless a supported source mechanism fully resolves that condition. Join keys, grouping keys, and expressions also contribute required inputs.

For a Parquet query that groups Korean orders by country and sums amount, an unused note can disappear from ReadSchema while country and amount remain. Check the scan’s required schema and any residual filter together; deleting the filter column blindly can change or break the computation.

A smaller projection does not always mean proportionally fewer bytes read. Row-oriented formats, metadata reads, compression units, reader support, and caching affect actual work. Use the plan to identify the intended reduction and storage metrics to assess realized savings.

See Spark DataFrames and Spark SQL for worked examples.

Reference: Apache Spark documentation.