Operate RDD in SPARK, merge every 3 lines into one line

Question

I have a copy of the data now, the data for each line is as follows。

A
B
C
QW
OO
P
...

Now I want to merge every three lines of behavior, as follows:

ABC
QWOOP
...

What should I code this function?

eg. val data = sc.textFile("path")

Thanks!

Answer 1

val lineRdd = sc.textFile("path")

val yourRequiredRdd = lineRdd
  .zipWithIndex
  .map({ case (line, index) => (index / 3, (index, line)))
  .aggregateByKey(List.empty[(Long, String)])(
    { case (aggrList, (index, line)) => (index, line) :: aggrList },
    { case (aggrList1, aggrList2) => aggrList1 ++ aggrList2 }
  )
  .map({ case (key, aggrList) =>
    aggrList
      .sortBy({ case (index, line) => index })
      .map({ case (index, line) => line })
      .mkString("")
  })