22

I'm looking to for a service that is similar to Amazon S3, a simple service to store and retrieve arbitrary data (and meta-data), but one that runs locally in your own data center. Strictly speaking, I'm not sure whether you would call this a CDN or a lightweight CMS.

It must be horizontally scalable (both for storage and bandwidth) and fault tolerable. It must also support REST, preferably WS too, with a pluggable authentication and authorization system. Something built with Java EE would be preferable for more convenient integration and extensibility, but this is just a personal preference, and it not a requirement.

Suggestions?

Arjan Tijms
  • 37,782
  • 12
  • 108
  • 140
jnorris
  • 6,350
  • 8
  • 30
  • 33
  • If it's inside your data centre, why not just use some sort of SAN? – Toby Hede Apr 27 '09 at 04:28
  • The SAN is just the storage part (isn't it?). I'm looking for the storage and the APIs to go along with it: to store and retrieve arbitrary data and meta-data, security, etc. – jnorris Apr 27 '09 at 05:22

4 Answers4

17

Here are a few open source solutions I have come across that deserve further research:

  1. Apache Sling (JCR based CMS (JSR170, JSR283), RESTful interface).
  2. Apache Hadoop (Java based distributed data-store, map reduce functionality).
  3. HBase (built on top of Hadoop, provding Google Bigtable-like capabilities).
  4. CouchDB (Erlang based key/value DB with Map/Reduce functionality, RESTful interface).
  5. Dynomite (Erlang based, Amazon dynamo clone).
  6. Voldemort (Distributed key-value storage system).
  7. Cassandra (highly scalable, eventually consistent, distributed, structured key-value store).
  8. MongoDB (highly scalable, JSON document based storage).
jnorris
  • 6,350
  • 8
  • 30
  • 33
6

Walrus project (mostly s3 api compatible) . . .

http://open.eucalyptus.com/wiki/EucalyptusStorage_v1.4

tbond
  • 205
  • 1
  • 8
2

Park place is an S3 clone in Ruby.

1800 INFORMATION
  • 131,367
  • 29
  • 160
  • 239
  • I was trying to find the url for that one. – Toby Hede Apr 27 '09 at 04:27
  • This looks interesting. Is there really no drop-in JEE solution in this space? There has to be (not that I have anything against Ruby , in fact, I'll likely spike park place). Perhaps I'm just missing the correct classification and terminology for these types of solutions, so they don't turn up in my searching? – jnorris Apr 27 '09 at 05:59
  • There might be but I don't know of one. I just know about this one because a dead tree book I'm reading mentioned it – 1800 INFORMATION Apr 27 '09 at 07:57
-6

In addition to Park Place, the only other big player against S3 right now is Nirvanix. Nirvanix

Serapth
  • 7,122
  • 4
  • 31
  • 39