In spark can we pass one dataframe df1 to udf of df2?
I am trying following but in udf i am getting null pointer exception while accessing df1 in udf.
example -
df2.select(df2.columns.map(c => testUDF(df1, c)(col(c)).alias(c + "_test")): _*)
def testUDF(df1: DataFrame, cn: String) =
udf((c:String) => {
val t = df1.where(col("fn") === cn)
//somecode
})