When using native UDFs does performance vary based on language used (scala vs python vs sql)
I have a normal sql statement with 2 CST running on spark through databricks. Then I did that same process in same order using pyspark in databricks on same data. No custom UDF involved.
I found that pyspark took 6 seconds and sql took 5 seconds.
