Spark SQL - regexp_replace not updating the column value

Asked Mar 31 '18 at 11:32

Active Mar 31 '18 at 12:03

Viewed 427 times

I ran the following query in Hive and it successfully updated the column value in the table: select id, regexp_replace(full_name,'A','C') from table

But when I ran the same query from Spark SQL, it did not update the actual records

hiveContext.sql("select id, regexp_replace(full_name,'A','C') from table")

but when I do a hiveContext.sql("select id, regexp_replace(full_name,'A','C') from table").show() -- it displays A replaced with C successfully ... only in the display and not in the actual table

I tried to assign the result to another variable

val vFullName = hiveContext.sql("select id, regexp_replace(full_name,'A','C') from table")

and then

vFullName.show() -- it displays the original values without replacement

How do I get the value replaced in the table from SparkSQL?

edited Mar 31 '18 at 12:03

Kaushik Nayak

30,772
5
32
45

asked Mar 31 '18 at 11:32

Neha

https://stackoverflow.com/questions/37517371/update-query-in-spark-sql?utm_medium=organic&utm_source=google_rich_qa&utm_campaign=google_rich_qa – Kaushik Nayak Mar 31 '18 at 12:05
you can also create a new table with the updated column using CTAS. – jose_bacoy Apr 01 '18 at 01:10

Spark SQL - regexp_replace not updating the column value

0 Answers0