Questions tagged [amazon-glacier]

Amazon Glacier is an online storage service intended to provide durable and secure long-term storage for the purposes of archiving and backup of large amounts of data. Glacier is lower cost than the other Amazon storage offerings, primarily because it's optimized for infrequent access.

Amazon Glacier is an online storage service offered as part of Amazon Web Services.

Glacier is intended to provide durable and secure long-term storage for the purposes of archiving and backing up of larger amounts of data, ultimately intended to compete with backup and storage systems that companies often must maintain.

It is differentiated from the other storage offerings from Amazon - S3 (Simple Storage Service) and EBS (Elastic Block Store) - in that it's designed for extremely infrequent access of the data stored inside its 'Vaults', whereas those services are designed such that objects stored inside them are highly available to be manipulated and/or served in real-time to Amazon EC2 servers (in the case of EBS), or web clients (in the case of S3).

Glacier is significantly lower cost than the other Amazon storage offerings, primarily because it's optimized for infrequent retrievals of data, however the delay period between requesting a data retrieval and the 'Vault' content becoming available makes it unsuitable for real-time use as a storage system.

331 questions
115
votes
8 answers

Backup strategies for AWS S3 bucket

I'm looking for some advice or best practice to back up S3 bucket. The purpose of backing up data from S3 is to prevent data loss because of the following: S3 issue issue where I accidentally delete this data from S3 After some investigation I…
Sergey Alekseev
  • 11,910
  • 11
  • 38
  • 53
81
votes
2 answers

Amazon S3 lifecycle retroactive application

Fairly straightforward question. Do Amazon S3 lifecycle rules that I set get applied to data retroactively? If so, what sort of delay might I see before older data begins to be archived or deleted? By way of example, let's say I have a bucket with…
DougW
  • 28,776
  • 18
  • 79
  • 107
66
votes
14 answers

How to restore folders (or entire buckets) to Amazon S3 from Glacier?

I changed the lifecycle for a bunch of my buckets on Amazon S3 so their storage class was set to Glacier. I did this using the online AWS Console. I now need those files again. I know how to restore them back to S3 per file. But my buckets have…
TylerW
  • 1,443
  • 2
  • 13
  • 25
32
votes
4 answers

Backup: Amazon S3 or Glacier - lots of little files?

I'm trying to understand the complicated Amazon Glacier pricing model. I don't want to store a huge amount of data, a few GB's say 10. I hope never to download the files and if I did need to I don't care how long it takes. Is there a cost per file I…
Markive
  • 2,350
  • 2
  • 23
  • 26
26
votes
2 answers

Permanently restore Glacier to S3

I'm wondering whether there is an easy way to permanently restore Glacier objects to S3. It seems that you can restore Glacier objects for the certain amount of time you provide when restoring to S3. So for example, we have now thousands of files…
Arman
  • 927
  • 3
  • 12
  • 32
20
votes
2 answers

Is it possible to move EC2 volumes to Amazon Glacier without having to download and upload it?

I am trying to reduce costs of my AWS system. I thought of moving some volumes I rarely use to Amazon Glacier, but I don't find any way I can do it inside AWS besides downloading the volume and uploading to Glacier, which sounds terrible. I am…
JordanBelf
  • 3,208
  • 9
  • 47
  • 80
19
votes
4 answers

Node reading file in specified chunk size

The goal: Upload large files to AWS Glacier without holding the whole file in memory. I'm currently uploading to glacier now using fs.readFileSync() and things are working. But, I need to handle files larger than 4GB and I'd like to upload multiple…
kjs3
  • 5,758
  • 8
  • 34
  • 49
17
votes
3 answers

Move files between amazon S3 to Glacier and vice versa programmatically using API

I am creating a PHP based web application using Amazon's S3 and glacier services. Now I want to give my site users a feature that they can choose any file and make it archive (means move file from S3 to Glacier) and unarchive (means move file from…
Irfan DANISH
  • 8,349
  • 12
  • 42
  • 67
15
votes
6 answers

How do I restore from AWS Glacier back to S3 permanently?

I have about 50Gb worth of files that was stored in S3. Yesterday I stupidly added a lifecycle rule to transfer files that were more than 30 days old from S3 to Glacier not realising that this will disable the public link to the original file. I…
Pete Dermott
  • 663
  • 1
  • 9
  • 20
15
votes
2 answers

Listing S3 bucket objects with specific storage class

It's very time consuming to get objects from Glacier so I decided to use S3 IA storage class instead. I need to list all the objects in my bucket that have Glacier storage class (I configured it via LifeCycle policy) and to change it to S3 IA. Is…
13
votes
4 answers

How to install awscli version 2 on raspberry pi

sudo apt install awscli installs currently version 1.16.113-1 The official website (https://docs.aws.amazon.com/cli/latest/userguide/install-cliv2-linux.html) supports aarch64 but not armhf. Is there another way or something that I am missing. The…
ECII
  • 10,297
  • 18
  • 80
  • 121
13
votes
3 answers

AWS S3 Sync --force-glacier-transfer

A few days back I was experimenting with S3 & Glacier and my data was archived so restoring it back I had to use their expedited service (which costs a lot). I want to move all of my content from one bucket to another bucket in the same region same…
ieatbytes
  • 516
  • 1
  • 6
  • 13
12
votes
3 answers

Amazon AWS Athena S3 and Glacier Mixed Bucket

Amazon Athena Log Analysis Services with S3 Glacier We have petabytes of data in S3. We are https://www.pubnub.com/ and we store usage data in S3 of our network for billing purposes. We have tab delimited log files stored in an S3 bucket. Athena…
12
votes
2 answers

AWS: Ways of keeping cost down while backing up S3 files to Glacier?

As part of our project, we have created quite a bushy folder/file tree on S3 with all the files taking up about 6TB of data. We currently have no backup of this data which is bad. We want to do periodic back ups. Seems like Glacier is the way to…
I Z
  • 5,719
  • 19
  • 53
  • 100
11
votes
3 answers

AWS S3 Glacier upload-archive taking a long time to finish execution - ways to check status or speed upload?

I am trying to upload some .tar.gz files to AWS S3 Glacier using the upload-archive command (docs). I ran the upload-archive command over one day ago on a 29 GB .tar.gz file like so: aws glacier upload-archive --vault-name my-vault --account-id -…
1
2 3
22 23