0

In spark can we pass one dataframe df1 to udf of df2?

I am trying following but in udf i am getting null pointer exception while accessing df1 in udf.

example -

df2.select(df2.columns.map(c => testUDF(df1, c)(col(c)).alias(c + "_test")): _*)



def testUDF(df1: DataFrame, cn: String) =
   udf((c:String) => {
    val t = df1.where(col("fn") === cn)
    //somecode
})
user1122
  • 297
  • 2
  • 9
  • 15

0 Answers0