0

I found a bunch of image files that has the wrong extension. Due to the way the website is made, they must have a .jpg extension. But some of them are png files.

So I made a quick list of fake JPEG files ls public/assets/image/*.jpg | xargs file --mime | grep -v jpeg and found https://stackoverflow.com/a/25825362/205696 that has an example of how to use the aws-cli tool to change the Cache-Control header (I want to change the Content-Type header).

According to the aws s3 sync help man page, the syntax is sync <LocalPath> <S3Uri> or <S3Uri> <LocalPath> or <S3Uri> <S3Uri>. Since I do not want to sync any files from my machine, I opt for the last syntax aws s3 sync <S3Uri> <S3Uri>.

However doing:

aws s3 sync s3://firefund-assets/underathens4.jpg s3://firefund-assets/underathens4.jpg --content-type image/png

does not seem to change anything. The underathens4.jpg file still has System defined Content-Type image/jpeg... and there is nothing written to stdout.

If I can change the meta-data for a file in S3, then I can simply do cat fakeJpegs.txt | sed 's/public\/assets\/image\///' | xargs -i aws s3 sync s3://firefund-assets/'{}' s3://firefund-assets/'{}' --content-type image/png

I do not want to re-upload any files. In that case, I could use s3.console.aws.amazon.com and do the change manually. I just want to have a little script to do this occasionally..

Change the default content type on multiple files that have been uploaded to a AWS S3 bucket is related but suggest uploading the files again.

dotnetCarpenter
  • 10,019
  • 6
  • 32
  • 54
  • 2
    `sync` operates on directories and shared prefixes. If you want to operate on a single file, use `cp`, including copying from s3 to s3 to modify the metadata. – Anon Coward Jan 20 '22 at 20:59
  • +1 to Anon Coward's comment. Also since the files' names are the same (I expect) you'll be creating a new version of the object – Oscar De León Jan 20 '22 at 21:07
  • Thanks! I have disabled versioning since we have a back up of the files elsewhere. So that is not an issue. – dotnetCarpenter Jan 21 '22 at 01:23

1 Answers1

1

You can copy an object over itself while specifying new information.

Use the cp command rather than the sync command:

aws s3 cp s3://firefund-assets/file.jpg s3://firefund-assets/file.jpg --content-type image/png
John Rotenstein
  • 241,921
  • 22
  • 380
  • 470
  • Something is wrong. After running `aws s3 cp ...` none of the image files are visible from our website. I get a 403 Forbidden code from AWS. Here is 2 examples - the first one I have changed the `Content-Type` with using your snippet above. https://firefund-assets.s3.amazonaws.com/t-27F.jpg and this one is unchanged https://firefund-assets.s3.amazonaws.com/100yil2.jpg – dotnetCarpenter Jan 21 '22 at 03:04
  • How are you making the objects public? The normal way would be to add a **Bucket Policy** on the Amazon S3 but that grants `GetObject` access. See: [Bucket policy examples - Amazon Simple Storage Service](https://docs.aws.amazon.com/AmazonS3/latest/userguide/example-bucket-policies.html#example-bucket-policies-use-case-2). Alternatively, it is possible to grant object-level permissions when objects are uploaded by specifying: `--acl public-read` – John Rotenstein Jan 21 '22 at 03:12
  • I sat the permission to public when I uploaded the files the first time. I think I should add the example you gave to my bucket policy, so that the files will stay public, regardless of meta-data changes. Is `s3:GetObjectVersion` really necessary? Thanks! – dotnetCarpenter Jan 21 '22 at 03:28
  • I guess `s3:GetObjectVersion` is not needed. It works now :) Thanks again! – dotnetCarpenter Jan 21 '22 at 03:29