I want to pass Array[List]
called alarm_stat
as the argument of UDF function. This is the content of alarm_stat
:
alarm_stat_ds: [
Row(
f_id=2,
type='activity',
qty=1,
number_hosts=1
)
]
My code:
schema = StructType([
StructField('prob', StringType()), StructField('level', StringType())
])
get_alert_udf = func.udf(ah.get_alert, schema)
df = df \
.withColumn("val", func.when(func.col("is_inside") == 1,
get_alert_udf(
func.col("id"),
alarm_stat
))
.otherwise(func.struct(func.lit("0"),func.lit("0")).cast(schema))
However I get the following error message:
py4j.protocol.Py4JError: An error occurred while calling z:org.apache.spark.sql.functions.col. Trace:
py4j.Py4JException: Method col([class java.util.ArrayList]) does not exist