Spark: DataFrame renaming columns and updating row values

Question

I want to rename a column using spark data frame and add a condition.

For example: I have a column "Code" which I want to rename to "Source Code" and I want to add a condition. When the values in the column equal "A" it should be renamed to "Agent" and when the value is "O" it should be renamed to "Other".

Check this conversation [http://stackoverflow.com/questions/35592917/renaming-column-names-of-a-data-frame-in-spark-scala](http://stackoverflow.com/questions/35592917/renaming-column-names-of-a-data-frame-in-spark-scala) — Noman Khan, Jan 05 '17 at 06:24

score 2 · Answer 1 · answered Jan 04 '17 at 22:04

2

renaming column

df.withColumnRenamed("oldName", "newName")

updating value

val newColumnValue = df("c")... //some calculation
df.withColumn("updatedValue", newColumn)

answered Jan 04 '17 at 22:04

Natalia

4,362
24
25

Spark: DataFrame renaming columns and updating row values

1 Answers1