What is the recommended way of importing data from AWS glacier into ElasticSearch? The logstash s3 input plugin doesn't seem to have support of restoring data from glacier (obviously wouldnt expect it to), but is there an advised approach?
Asked
Active
Viewed 693 times
0
-
Your question is unclear, when you restore data from glacier, within 5 hours (that's [their SLA](http://docs.aws.amazon.com/AmazonS3/latest/dev/object-lifecycle-mgmt.html)) it becomes available and you can download it from S3 as you would fetch any object from any bucket. – Nir Alfasi Aug 24 '15 at 04:51
-
1There's an [ongoing discussion](https://discuss.elastic.co/t/snapshot-and-restore-s3-and-glacier/26337/14) about this very subject. [An issue](https://github.com/elastic/elasticsearch/issues/12500) has been opened recently as well. – Val Aug 24 '15 at 04:51
-
This is time-consuming, by design, and not something you would probably want to automate, since it could be expensive -- there are penalties for restoring too much, too quickly, as it's intended to be "cold storage." Objects in Glacier that were migrated by S3 have to be restored back to S3, and this is temporary (they become downloadable but also remain in Glacier, and the restored S3 copy will be purged after the number of days you specify) unless you deliberately un-archive the object by copying it onto itself in S3 after restoring it. See also http://stackoverflow.com/a/23804828/1695906. – Michael - sqlbot Aug 24 '15 at 15:52