5

We are planning to upgrade our cluster which currently runs on 2.0.9 to 2.2.6. According to the documentation and some blogs people upgrade cassandra inplace ie remove a node from ring upgrade it and add it back again. We are skeptical of following this approach as things can go wrong(This is a high transaction database with a good number of QPS).

So we were planning to add a new datacenter to the cluster which shall have upgraded cassandra version(2.2). So the setup shall have two datacenter one old(2.0.9) and the other new (2.2.6)

This datacenter is just a backup. When the datacenter becomes stable we shall change the client connection to this datacenter and if it plays well then we shall go with this datacenter and close the old datacenter or else we can fall back to the old datacenter and debug what went wrong.

Is this process feasible enough or should we go for in place upgrade?

Can two cassandra version(2.0 and 2.2) exist across a datacenter.

Is there a downfall in this approach?

Martin Schröder
  • 4,176
  • 7
  • 47
  • 81
johri21
  • 153
  • 1
  • 9

2 Answers2

3

Can two cassandra version(2.0 and 2.2) exist across a datacenter.

No, they cannot.

Is this process feasible enough or should we go for in place upgrade?

You will need to perform an in-place upgrade. This is because Cassandra cannot stream across versions. Performing an in-place upgrade allows the new version to read the SSTables from the old version.

Is there a downfall in this approach?

As I mentioned, you will not be able to stream data from your existing nodes to the new 2.2 DC. So bootstrapping, rebuilding, and repairing are all out of the question.

The other issue you have, is that 2.2.6 is not "upgrade compatible" with 2.0.9. From this DataStax doc: Apache Cassandra versions requiring intermediate upgrades...

Apache Cassandra 2.2.x restrictions

  • Upgrade from Cassandra 2.1 versions later or equal to 2.1.9 directly to Cassandra 2.2.x.
  • Direct upgrade from Cassandra 2.0 and older versions is not supported.

You will first have to upgrade your entire cluster to Cassandra 2.1. Once the upgrade to 2.1 is complete, then you can upgrade your nodes to 2.2.6.

Aaron
  • 55,518
  • 11
  • 116
  • 132
  • Hi @Aaron, Thanks. Its very strange as to why two cassandra version cannot exist across a datacenter but can exist in a same datacenter(Cluster). Is there a particular reason for that? If cassandra cannot stream data across datacenter with different versions .. Can I first have the same version in new datacenter and then after bootstrapping, rebuilding and repairing I upgrade the version in new datacenter and after upgrade change the client connection? Is this feasible? ... I know even in this approach two datacenter are in two different versions. Will this work? Thanks. – johri21 Sep 28 '16 at 14:11
1

Cassandra is a master-less distributed datastore. For Cassandra there's no such thing as a "backup" datacenter. If you're going to add another DC running 2.2, you're opting-in for a mixed version cluster setup, just as you would by upgrading nodes individually. The only advantage I see is that performance issues should be less likely due to the added nodes. However, adding another DC will make your cluster setup more complex and may introduce issues that you don't know about yet, but won't have anything to do with running different versions. How would you bootstrap the new DC? How will taking down the old DC effect performance? The operational impact will be much bigger with this approach compared to updating individual nodes..

If you really don't want to do rolling upgrades, I'd suggest do setup the second DC as a separate cluster, import a backup and do some (load) testing. Also change your code to write into both clusters and eventually switch to the new one if you're satisfied. If you don't want to spend so much effort, just do the rolling upgrade.

Stefan Podkowinski
  • 5,206
  • 1
  • 20
  • 25
  • Hi Stefan, First a prelude I have a 4 node cluster with a running 2.0.9 cassandra version. Now I plan to set up different DC with 2.2 cassandra version. This shall be bootstraped from the old DC(2.0.9). I know this shall take some time(bootstapping). The advantage is the reads and writes from my application will be served from 2.0.9 which is working perfectly and all this while the reads and writes will be synced to the New DC(async). Then in a lean period(when traffic is low) I shall just change the contact point to my new DC(2.2). After everything works fine I shall close the old DC. – johri21 Sep 28 '16 at 12:07