Using median instead of mean as aggregation function in Spark

Question

Say I have a dataframe that contains cars, their brand and their price. I would like to replace the avg below by median (or another percentile):

df.groupby('carBrand').agg(F.avg('carPrice').alias('avgPrice'))

However, it seems that there is no aggregation function that allows to compute this in Spark.

Answer 1

You can try the approxQuantile function (see http://spark.apache.org/docs/latest/api/python/pyspark.sql.html#module-pyspark.sql.functions)