-1

How to convert dataframe to RDD and store it in Cassandra in Spark Scala. Consider an example like below,

emp_id| emp_city|emp_name

 1|Hyderabad| ram
 4| Banglore| deeksha

Here I am using only 3 columns as an example but I need to deal with 18 columns actually.

Learner
  • 1,170
  • 11
  • 21
shantha ramadurga
  • 75
  • 1
  • 2
  • 12

1 Answers1

1

You can directly save DF without converting into RDD.

df.write.format("org.apache.spark.sql.cassandra").options(Map( "table" -> "employee", "keyspace" -> "emp_data")).save()

Thanks.

Learner
  • 1,170
  • 11
  • 21
  • Hi,Thanks , i had tried this option it is working fine with simple tables like above, but i have few timestamp columns and null values and all. when i try to use this option ,its giving error saying like java.lang.IllegalArgumentException: requirement failed: Invalid row size: 18 instead of 17. – shantha ramadurga Aug 20 '18 at 10:18
  • Hi, please find the below link for more details about the question:https://stackoverflow.com/q/51929246/10246547 – shantha ramadurga Aug 20 '18 at 10:59