0

I'm new in spark Java API. I want to transform double with scientific format example: 1.7E7---->17000000,00.

MyDataSet is:

+---------+------------+
|  account|    amount  |
+---------+------------+
| c1      |      1.7E7 |
| c2      |      1.5E8 |
| c3      |      142.0 |

I want to transform my dataset to something like this.

    +---------+----------------------+
    |  account|    amount            |
    +---------+----------------------+
    | c1      |      17000000,00     |
    | c2      |      1500000000,00   |
    | c3      |      142,00          |

Can someOne guide me with an expression in spark Java to resolve this. Thanks in advance.

Oviic
  • 45
  • 1
  • 5
  • 1
    Possible duplicate of [How to turn off scientific notation in pyspark?](https://stackoverflow.com/questions/40206592/how-to-turn-off-scientific-notation-in-pyspark) – Rcordoval May 29 '18 at 08:16

1 Answers1

3

I think you can do it like this. Don't forget import spark.sql.functions

Dataset<Row> myDataset = ....
myDataset = myDataset.withColumn("newAmount", col("amount").cast(DataTypes.DoubleType))
    .drop(col("amount"))
    .withColumnRenamed("newAmount","amount")
    .toDF();
Danila Zharenkov
  • 1,720
  • 1
  • 15
  • 27
  • I resolve my probleme using this: myDataset = myDataset.withColumn("newAmount", col("amount").cast(DataTypes.createDecimalType(20, 2))) .drop(col("amount")) .withColumnRenamed("newAmount","amount") .toDF(); – Oviic May 29 '18 at 11:14