How Spark Decides What to Run: Plans, Stages, and the Cost of Moving Rows
Nothing runs until an action asks for a result. Then a plan becomes stages, stages become tasks, and every shuffle boundary shows up in the runtime.
Nothing runs until an action asks for a result. Then a plan becomes stages, stages become tasks, and every shuffle boundary shows up in the runtime.