Copy from Table to Table (altering Primary Key)

Question

Context :

I need to change the order of my primarey key

PRIMARY KEY (a, b) --> PRIMARY KEY (b,a)

But in cassandra I can't just alter PRIMARY KEY like this : alter composite primary key in cassandra CQL 3.0

So, I just want to move data from the old table to a new table I've created. Since the number of rows is huge I need to use Spark.

Data "in" :

I get the data like this from the old table:

JavaRDD<CassandraRow> initialRDD;

Question :

Id it possible to directly save it back to the new table since the row itself didn't change?

What I managed to do :

I could do this but it seems absurd to map from CassandraRow to CassandraRow

javaFunctions(initialRDD).writerBuilder("targetKeyspace", "targetTable", mapToRow(CassandraRow.class)).saveToCassandra();

thus it's prompting me an error :

Some primary key columns are missing in RDD or have not been selected: ...

Do you really require to do it in Java client? It should be easier to write CQL migration script. — Patrik Třeštík, Jan 07 '16 at 11:56
if your data is less you can directly do it using COPY command in cql — undefined_variable, Jan 07 '16 at 12:46
@PatrikTřeštík It has more than a million rows, I can not just COPY so yes I would like to do it in Java — Fundhor, Jan 07 '16 at 13:50
I would still test COPY TO STDOUT piped to COPY FROM STDIN. I didn't tested it on bigger data, but piping should do the work. — Patrik Třeštík, Jan 07 '16 at 14:15

score 1 · Answer 1 · answered Jan 09 '16 at 22:04

I don't use spark very much, but I think you would need to create a second table with the new key order, then load the first table into an RDD using the spark connector:

val rdd = sc.cassandraTable("keyspace", "table1")

Then save it to the second table with the new key order:

rdd.saveToCassandra("keyspace", "table2", SomeColumns("b", "a"))

Copy from Table to Table (altering Primary Key)

Context :

Data "in" :

Question :

What I managed to do :

1 Answers1