Broadcast Join

A broadcast join distributes a build relation so tasks can match it against their streamed input without a key shuffle of that input required by the join. The broadcast itself still transfers data.

Source skew, large input partitions, expensive matching, or a later aggregation can remain imbalanced. Duplicate build keys can multiply the output. Switching join algorithms does not change the required match multiplicity.

Check driver-side construction, executor memory, the built representation, timeout, and the supported join type. Compressed file size and estimated table size are not the memory footprint of a built hash relation. A hint is not a guarantee that the strategy is applicable.

Inspect BroadcastExchange and the chosen join operator, then compare full outputs against a reference plan. A local timing ratio does not establish a production speedup or isolate the cost of a single hot task.

See Spark Partitioning, Shuffles, and Data Skew for worked examples.

Further reading

• Apache Spark documentation