AWS: Append only mode for S3 bucket

Question

Context

I want to have a machine upload a file dump.rdb to s3/blahblahblah/YEAR-MONTH-DAY-HOUR.rdb on the hour.

Thus, I need this machine to have the ability to upload new files to S3.

However, I don't want this machine to have the ability to (1) delete existing files or (2) overwrite existing files.

In a certain sense, it can only "append" -- it can only add in new objects.

Question:

Is there a way to configure an S3 setup like this?

Thanks!

append to a file and creating new objects are not the same thing ... yeah ? — smartnut007, Jan 05 '15 at 02:34
I just answered a very closely related question here: https://stackoverflow.com/a/45767026/714112 — Sridhar Sarnobat, Aug 19 '17 at 02:25

score 5 · Answer 1 · edited May 23 '17 at 12:26

5

I cannot comment yet, so here is a refinement to @Viccari 's answer...

The answer is misleading because it only addresses #1 in your requirements, not #2. In fact, it appears that it is not possible to prevent overwriting existing files, using either method, although you can enable versioning. See here: Amazon S3 ACL for read-only and write-once access.

Because you add a timestamp to your file names, you have more or less worked around the problem. (Same would be true of other schemes to encode the "version" of each file in the file name: timestamps, UUIDs, hashes.) However, note that you are not truly protected. A bug in your code, or two uploads in the same hour, would result in an overwritten file.

edited May 23 '17 at 12:26

Community

1
1

answered Sep 03 '13 at 08:19

user4890

103
1
5

Use a UUID together with your filename and you'll be fine. – Ztyx Aug 22 '14 at 22:04
4

Using an UUID does not prevent a buggy expiration routine or a hacker deleting or overwriting the objects. – snap Feb 26 '15 at 20:16
I'm glad this post is not a comment. Have an upvote! – Flimm Feb 15 '17 at 16:46
Thanks for not adding a comment and adding a new response instead. Have a +1. – Viccari Feb 27 '17 at 15:50

Viccari · Accepted Answer · 2012-07-02T00:37:00.630

0

Yes, it is possible.
There are two ways to add permissions to a bucket and its contents: Bucket policies and Bucket ACLs. You can achieve what you want by using bucket policies. On the other hand, Bucket ACLs do not allow you to give "create" permission without giving "delete" permission as well.

1-Bucket Policies:
You can create a bucket policy (see some common examples here), allowing, for example, an specific IP address to have specific permissions.
For example, you can allow: s3:PutObject and not allow s3:DeleteObject.
More on S3 actions in bucket policies can be found here.

2-Bucket ACLs:
Using Bucket ACLs, you can only give the complete "write" permission, i.e. if a given user is able to add a file, he is also able to delete files.

edited Jul 02 '12 at 00:37

answered Jul 01 '12 at 03:15

Viccari

9,029
4
43
77

So basically, the answer is "No, this is not possible" because granting create rights imply delete rights? – Jul 01 '12 at 20:40
Sorry if I was not clear. The answer is "yes, it is possible". I am updating my answer to make it more clear. – Viccari Jul 01 '12 at 23:42
1

With apologies due to my S3 Ignorance, Is the following correct: with Bucket ACLs, it is NOT possible. However, there is a different mechanism called Bucket Policies, with which it is possible? – Jul 02 '12 at 00:07
1

Yes. Perfect. Sorry for the lack of clarity. – Viccari Jul 02 '12 at 00:35
4

It seems like `PutObject` will allow you to overwrite objects, so this isn't a perfect solution. – Flimm Feb 15 '17 at 16:58

score 0 · Answer 3 · answered May 22 '17 at 15:07

This is NOT possible! S3 is a key/value store and thus inherently doesn't support append only. The PUT/cp command to S3 can always overwrite a file. By enabling versioning on your bucket you are still safe in cause the account uploading the files gets compromised.

AWS: Append only mode for S3 bucket

Context

Question:

3 Answers3