-1

My Data looks like this

H1234|1234|1999-12-03.3.22.34.132456
G1345|2345|1998-11-03-12.22.45.23456

I stored this data on a List[String], while converting it to a data frame by doing:

val dataframe = list.map(r => r.split("\\|")).map(r => (r(0),r(1),r(2)).toDF("ID","Number","Timestamp ")

but when I am using dataframe.show, the Timestamp Column, I am getting it as below:

(FYI every value is a String)

Timestamp
1999-12-03.3.22....
1998-11-03-12.22...

Could you please tell me how to solve this.

Tomer Shetah
  • 8,413
  • 7
  • 27
  • 35
Buzz97
  • 33
  • 5
  • I guess this is what you are looking for: https://stackoverflow.com/questions/41867147/convert-list-into-dataframe-spark-scala – Michael Heil Dec 16 '20 at 16:04

1 Answers1

1

Your data is actually intact, but show is truncating the output. Do dataframe.show(false) to avoid truncation of output.

mck
  • 40,932
  • 13
  • 35
  • 50
  • @mike I saw your link, but given that the OP got a timestamp column with two rows, I suspect the OP managed to build a proper dataframe... anyway let's wait for the OP's response – mck Dec 16 '20 at 16:06
  • I just wrote a little bit of code where i am applying transformation.... If you want I can write the whole code. I just wanted to give you guys a basic idea of what my data looks like and what problem I faced @mck – Buzz97 Dec 16 '20 at 16:48