-1

How does the code below concatenate multiple values in the list?

concat(myList.map(fld => col(fld)): _*)

Punter Vicky
  • 15,954
  • 56
  • 188
  • 315
  • 1
    What certain problem? ":_*" - converting array to varargs arguments: https://stackoverflow.com/questions/6051302/what-does-colon-underscore-star-do-in-scala ? – User9123 Jun 17 '21 at 22:26

1 Answers1

1

According to Spark documentation the signature of the concat function is concat(col1, col2, ..., colN). Given your list contains the column names, i.e: c1, c2 ... cN, map will convert each one of these into Column class objects. The conversion is done using the col function. Finally, the _* will unpack the (converted to Column) list items, similarly to how python's * operator works, assigning concat arguments.

abiratsis
  • 7,051
  • 3
  • 28
  • 46
  • Thank you , this is scala and not python. Does _* work similarly in scala? – Punter Vicky Jun 18 '21 at 01:08
  • yes exactly! And by the way [here](https://github.com/apache/spark/blob/ea907469bbf15b4b43540f6c33184cc790fadcb7/sql/core/src/main/scala/org/apache/spark/sql/functions.scala#L3736) you can find the code for concat – abiratsis Jun 18 '21 at 08:09