4

I'm looking to place a rather large HDF5 file online and am looking at Amazon S3 and/or EBS as a possible place to store it and possibly leverage an EC2 instance or two for back-end processing. I'm looking for practical ideas of how it can or has been done before and, if possible, a link to an actual example. I think it can be done in the same way that someone would access a flat-file on some other embedded data store, but would like to avoid reinventing the wheel if there's practical insight available.

Marc
  • 4,546
  • 2
  • 29
  • 45
  • I'm not familiar with HDF5, can you explain a little what that file is and how you plan to process it? That would influence my answer heavily. – Till May 22 '11 at 23:50

1 Answers1

1

you can just mount your S3 locally or on EC2 with FUSE:

http://code.google.com/p/s3fs/wiki/FuseOverAmazon

at that point you can use all the usual libraries to access HDF5.

Andrea Zonca
  • 8,378
  • 9
  • 42
  • 70