How to convert rows into string values in Apache Spark

Question

I have a spark dataframe like this:

fruit | name
--------------
fruit | apple
fruit | orange
fruit | mango

I want to convert it into this:

fruit | string
----------------------------
fruit | apple, orange, mango

How can I achieve this in Apache Spark?

Possible duplicate of [SPARK SQL replacement for mysql GROUP\_CONCAT aggregate function](http://stackoverflow.com/questions/31640729/spark-sql-replacement-for-mysql-group-concat-aggregate-function) — mtoto, Nov 30 '16 at 12:30

score 2 · Accepted Answer · edited May 23 '17 at 12:24

2

Look at collect_list

df.groupBy("fruit").agg(collect_list("name"))

it will group values and create array of them as a new column.

If you want to have string, please see this question (thanks @mtoto)

edited May 23 '17 at 12:24

Community

answered Nov 30 '16 at 10:13

T. Gawęda

1 Answers1