7

I have an Amazon S3 bucket with about 300K objects in it and need to set the Cache-control header on all of them. Unfortunately it seems like the only way to do this, besides one at a time, is by copying the objects to themselves and setting the cache control header that way:

Is the documentation for the Amazon S3 CLI copy command but I have been unsuccessful setting the cache control header using it. Does anyone have an example command that would work for this. I am trying to set cache-control to max-age=1814400

Some background material:

Community
  • 1
  • 1
dpegasusm
  • 620
  • 2
  • 7
  • 20

1 Answers1

10

By default, aws-cli only copies a file's current metadata, EVEN IF YOU SPECIFY NEW METADATA.

To use the metadata that is specified on the command line, you need to add the '--metadata-directive REPLACE' flag. Here are some examples.

For a single file

aws s3 cp s3://mybucket/file.txt s3://mybucket/file.txt --metadata-directive REPLACE \
--expires 2100-01-01T00:00:00Z --acl public-read --cache-control max-age=2592000,public

For an entire bucket:

aws s3 cp s3://mybucket/ s3://mybucket/ --recursive --metadata-directive REPLACE \
--expires 2100-01-01T00:00:00Z --acl public-read --cache-control max-age=2592000,public

A little gotcha I found, if you only want to apply it to a specific file type, you need to exclude all the files, then include the ones you want.

Only jpgs and pngs

aws s3 cp s3://mybucket/ s3://mybucket/ --exclude "*" --include "*.jpg" --include "*.png" \
--recursive --metadata-directive REPLACE --expires 2100-01-01T00:00:00Z --acl public-read \
--cache-control max-age=2592000,public

Here are some links to the manual if you need more info:

Dan Williams
  • 3,769
  • 1
  • 18
  • 26
  • `REPLACE` makes problems: adding 'Cache-Control' now works, but this solution also breaks the 'ContentType' setting, changing it from existing, e.g. `image/png` to `binary/octet-stream`. – geekQ Nov 09 '16 at 08:16
  • @geekQ are you saying that by adding cache-control, it changes the 'ContectType' setting? Does it work if you specify 'ContentType' using the --content-type flag? The way the docs read, aws is guessing the content type, are your extentions '*.png'? – Dan Williams Nov 14 '16 at 15:54