5

I'm trying to evaluate a few distributed storage platforms and Cassandra is one of them. Our requirement is to save files between 1MB and 50MB of size and according to Cassandra's documentation http://docs.datastax.com/en/cql/3.3/cql/cql_reference/blob_r.html:

The maximum theoretical size for a blob is 2 GB. The practical limit on blob size, however, is less than 1 MB.

Does anyone have experience storing files in Cassandra as blobs? Any luck with it? Is the performance really bad with bigger file sizes?

Any other suggestion would also be appreciated!

Nikolay Dimitrov
  • 1,766
  • 1
  • 14
  • 23
  • 4
    Does anybody understand why there is the recommended (practical) limitation on 1 MB, vs the theoretically possible 2 GB? What exactly is causing problems if the blob size exceeds 1 MB? – Juergen Feb 11 '19 at 13:29

1 Answers1

0

Cassandra was not build for these type of job.

In Cassandra a single column value size can be: 2 GB ( 1 MB is recommended). So If you want to use use cassandra as object storage, split the big object into multiple small object and store them with object id as partition key and bucket id as clustering key.

It is best to use Distributed Object Storage System like OpenStack Object Storage ("Swift")

The OpenStack Object Store project, known as Swift, offers cloud storage software so that you can store and retrieve lots of data with a simple API. It's built for scale and optimized for durability, availability, and concurrency across the entire data set. Swift is ideal for storing unstructured data that can grow without bound.

Ashraful Islam
  • 12,470
  • 3
  • 32
  • 53
  • Thanks for the suggestion Ashraful, but we are not planning on using cloud solution. We need faster writes and reads in the same data centre. – Nikolay Dimitrov Jun 19 '17 at 06:22
  • Then you can use cassandra's solution. Here i make the object id as partition key so a single object read/write will be faster. Check the difference between partition key and clustering key key https://stackoverflow.com/a/24953331/2320144 – Ashraful Islam Jun 19 '17 at 06:33