Spark SQL broadcast for multiple join

Question

I know how to bradcast for 2 tables join, like:

SELECT  /*+ MAPJOIN(Table1) */ COLUMN 
FROM Table1 JOIN Table2 
ON Table1.key = Table2.key

But is there a way to use broadcast for 3 tables join?

SELECT  /* ？ */ COLUMN 
FROM Table1 JOIN Table2 ON ...
            JOIN Table3 ON ...

score -1 · Accepted Answer · answered Mar 18 '19 at 09:29

In general case, small tables will automatically be broadcasted based on the configuration spark.sql.autoBroadcastJoinThreshold.

And broadcast join algorithm will be chosen.

spark.sql.autoBroadcastJoinThreshold defaults to 10 MB (i.e. 10L * 1024 * 1024) and Spark will check what join to use (see JoinSelection execution planning strategy).

Spark SQL broadcast for multiple join

1 Answers1