Lazy Evaluation
Each step returns instantly and nothing has run. Deferring work is what lets an engine see the whole query — and why the last line gets all the blame.
-
-
Apache Spark Architecture and Execution
Understand Spark drivers, executors, partitions, memory limits, caching, and the conditions for recomputation and recovery.