Polars

Polars is a DataFrame library with an expression engine for filtering, grouping, joining, and transforming tabular data. A DataFrame represents materialized data; a LazyFrame represents a query plan to execute later.

Use scan_parquet for a lazy file query. Calling lazy() after an eager file read does not undo the read. Planning can move suitable filters and column selection toward the source, but an explain plan alone does not measure bytes read.

collect(engine=”streaming”) requests streaming execution. Supported work can run in batches, while unsupported operations may use in-memory execution. The collected result must still fit available resources; streaming is not a promise of constant memory for every query.

Check the intended meaning of null keys, duplicate joins, types, and ordering. Interchanging Arrow data can preserve values without proving that no buffers were allocated or copied. Test semantics and resource behavior separately.

See Single-Node Analytics: DuckDB, Polars, and Arrow for worked examples.

Reference: Official documentation.