2

I want to pass Array[List] called alarm_stat as the argument of UDF function. This is the content of alarm_stat:

alarm_stat_ds: [
                Row(
                    f_id=2, 
                    type='activity', 
                    qty=1, 
                    number_hosts=1
                   )
               ]

My code:

schema = StructType([
    StructField('prob', StringType()), StructField('level', StringType())
])

get_alert_udf = func.udf(ah.get_alert, schema)

df = df \
    .withColumn("val", func.when(func.col("is_inside") == 1,
                                 get_alert_udf(
                                               func.col("id"),
                                               alarm_stat
                                              ))
                            .otherwise(func.struct(func.lit("0"),func.lit("0")).cast(schema))

However I get the following error message:

py4j.protocol.Py4JError: An error occurred while calling z:org.apache.spark.sql.functions.col. Trace:
py4j.Py4JException: Method col([class java.util.ArrayList]) does not exist
Markus
  • 3,562
  • 12
  • 48
  • 85

0 Answers0