1

We are trying to migrate our ScyllaDB cluster deployed on GCE machines to the GKE cluster in Google Cloud, we came across one approach of Cassandra migration and want to implement the same here in ScyllaDB migration. Below is the link for the same, can you please suggest if this is possible in Scylla ? or if Scylla hasn't introduced such a migration technique with the Scylla K8S operator ?

https://k8ssandra.io/blog/tutorials/cassandra-database-migration-to-kubernetes-zero-downtime/

Erick Ramirez
  • 13,964
  • 1
  • 18
  • 23
Aditya Bibave
  • 11
  • 1

2 Answers2

2

Adding a new "destination" DC to your existing cluster "source" DC, is a very common technic to migrate to a new DC.

  1. Add the new "destination" DC
  2. Change replication factor settings accordingly
  3. nodetool rebuild --> stream data from the "source" DC to the "destination" DC
  4. nodetool repair the new DC.
  5. Update your application clients to connect to the new DC once it's ready to serve (all data streamed + repaired)
  6. Decommission the "old" (source) DC

For the gory details see here:

If you prefer to go the full scan route. CQL reads on the source and CQL writes on the destination, with some ability for data manipulation and save points to resume from, than the Scylla Spark Migrator is a good option.

https://github.com/scylladb/scylla-code-samples/tree/master/spark-scylla-migrator-demo

You can also use the Scylla Spark migrator to migrate parquet files https://www.scylladb.com/2020/06/10/migrate-parquet-files-with-the-scylla-migrator/

Remember not to migrate Materialized views (MV), you can always re-create them post migration again from the base tables.

TomerSan
  • 1,576
  • 6
  • 12
1

We use an Apache Spark-based Migrator: https://github.com/scylladb/scylla-migrator

Here's the blog we wrote on how to do this back in 2019: https://www.scylladb.com/2019/02/07/moving-from-cassandra-to-scylla-via-apache-spark-scylla-migrator/

Though in this case, you aren't moving from Cassandra to ScyllaDB; just moving from one ScyllaDB instance to another. If this makes sense to you, it should be straight forward. If you have questions, feel free to join our Slack community to get more interactive assistance:

http://slack.scylladb.com/

Peter Corless
  • 781
  • 3
  • 12
  • 1
    Thank you Peter, for your suggestion. We are thinking to proceed with the spark migrator approach only. Just needed clarification from your side, the k8ssandra operator approach as I mentioned in my query, will this work with Scylla operator in kubernetes or not as this was proposed by the client. – Aditya Bibave Aug 03 '22 at 10:58