scala-the operation about how to split one column

Question

for example:there are a dataframe like:

val df2 = sc.parallelize(List((1, "A|B"), (1, "C|D"), (2, "E|A"), (2, "S|D")))
.toDF("key1","key2")

Now I want to split the column "key2" to two column. The each row of the result will be showed like that:

1 "A" "B"

What should I code ? Thx!

Answer 1

You can split key2 column by | and then map each element to a new column. You can try following code,

df2.withColumn("key",split($"key2","\\|")).select(
      $"key1",
      $"key".getItem(0).as("key2"),
      $"key".getItem(1).as("key3")
    ).show()

// output
// +----+----+----+
// |key1|key2|key3|
// +----+----+----+
// |   1|   A|   B|
// |   1|   C|   D|
// |   2|   E|   A|
// |   2|   S|   D|
// +----+----+----+