34

On my website, all of a sudden I cannot upload photos. I get the following error:

##### RightAws::S3Interface returned an error: 400 Bad Request
<?xml version="1.0" encoding="UTF-8"?>
<Error><Code>InvalidBucketAclWithObjectOwnership</Code><Message>Bucket cannot have ACLs set with ObjectOwnership's BucketOwnerEnforced setting</Message><RequestId>REQUEST</RequestId><HostId>ID$
##### RightAws::S3Interface request: https://bucket.s3.amazonaws.com:443/ ####

RightAws::AwsError (InvalidBucketAclWithObjectOwnership: Bucket cannot have ACLs set with ObjectOwnership's BucketOwnerEnforced setting):

What's odd, is nothing has been changed on my end, when this used to work. AWS did send out this email, which seems related but was only supposed to apply to new buckets:

Hello,

We are reaching out to inform you that starting in April 2023 Amazon S3 will change the default security configuration for all new S3 buckets. For new buckets created after this date, S3 Block Public Access will be enabled, and S3 access control lists (ACLs) will be disabled.

The majority of S3 use cases do not need public access or ACLs. For most customers, no action is required. If you have use cases for public bucket access or the use of ACLs, you can disable Block Public Access or enable ACLs after you create an S3 bucket. In these cases, you may need to update automation scripts, CloudFormation templates, or other infrastructure configuration tools to configure these settings. To learn more, read the AWS News blog [1] and What's New announcement [2] on this change or visit our user guide for S3 Block Public Access [3] and S3 Object Ownership to disable ACLs [4]. Also, see our user guide for AWS CloudFormation on these settings [5][6].

I have seen similar issues, where people fixed it by using Edit Object Ownership to set Object Ownership to ACLs disabled (recommended). When I try this I get this error:

Bucket owner enforced cannot be applied because you have existing bucket ACLs If you want to apply the bucket owner enforced setting, you must remove access granted to other AWS accounts or groups from your bucket ACL and migrate these permissions to a bucket policy

I'm not sure how to get past this. I've seen people update the Bucket Policy, but mine is currently blank. I'm fairly new to AWS so I'm being careful not to mess with too much I don't understand.

This bucket should be public. This is the current ACL settings:
acls

I attempted to move over to using a Bucket Policy. I set my bucket policy as follows:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "Stmt1380877761162",
            "Effect": "Allow",
            "Principal": {
                "AWS": "*"
            },
            "Action": "s3:GetObject",
            "Resource": "arn:aws:s3:::bucketname/*"
        }
    ]
}

I removed the Bucket ACLs to put them back to the defaults.
removed

And I edited Object Ownership:
objectownership

And still I get the same error, which is particularly confusing because if I understand correctly, it now should not be using ACLs at all.

Mark Rotteveel
  • 100,966
  • 191
  • 140
  • 197
Sara Fuerst
  • 5,688
  • 8
  • 43
  • 86
  • "The majority of S3 use cases do not need public access or ACLs." in your case , you *might* (or might not) want public access granted. Look at your bucket ACL. What does it currently say? It would be useful information to add to the question – erik258 Apr 25 '23 at 01:33
  • @erik258 It should be public, I've updated the question with the info – Sara Fuerst Apr 25 '23 at 01:38
  • 1
    Right now you're granting public access with ACL. You should instead use a bucket policy, then you don't need an ACL and can remove it. https://stackoverflow.com/a/70212452/1726083 looks like good reference for appropriate bucket policy. Good luck, if it doesn't end up making sense after you think about it, come back here and we'll try to explain it! – erik258 Apr 25 '23 at 01:43
  • Thank you! Do you have any reference for removing ACL? I unchecked all the boxes under Bucket ACL and then tried to switch to ACLs disabled and got an error: Bucket owner enforced cannot be applied because you have existing bucket ACLs. Is there something else I need to do to get rid of the Bucket ACL? – Sara Fuerst Apr 25 '23 at 01:55
  • 1
    https://docs.aws.amazon.com/AmazonS3/latest/userguide/object-ownership-migrating-acls-prerequisites.html This is the doc you want, I think – erik258 Apr 25 '23 at 02:01
  • @erik258 That doc helped. I was able to switch to using a bucket policy, but I still seem to be getting the same error – Sara Fuerst Apr 25 '23 at 13:35
  • What tool are you using to upload. It looks like it's trying to set an ACL which you have disabled – Daniel Scott Apr 25 '23 at 21:42
  • @DanielScott This is a Rails 2.3.2 app that uses the Paperclip gem to upload – Sara Fuerst Apr 25 '23 at 22:43
  • Are you setting an ACL? https://www.rubydoc.info/gems/paperclip/Paperclip/Storage/S3#s3_permissions-instance_method (The "quickest" solution would probably be to revert the Object Ownership setting on the S3 bucket. The "correct" solution would probably be to avoid trying to set the ACL in paperclip and setting the bucket policy to ensure public access continues to work) – Daniel Scott Apr 26 '23 at 15:03

4 Answers4

29

TLDR To create a bucket nowadays (alternative to acl public-read option):

#!/bin/bash

bucket_name="my-unique-name"
aws s3api create-bucket --bucket "${bucket_name}" > /dev/null # 1
aws s3api put-public-access-block --bucket "${bucket_name}" --public-access-block-configuration "BlockPublicPolicy=false" # 2
aws s3api put-bucket-policy --bucket "${bucket_name}" --policy '{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "PublicReadGetObject",
            "Effect": "Allow",
            "Principal": "*",
            "Action": [
                "s3:GetObject"
            ],
            "Resource": [
                "arn:aws:s3:::'"${bucket_name}"'/*"
 
            ]
        }
    ]
}' # 3

What changed?

1 - Since 25.04.2023 Amazon changed default settings for newly created buckets. The ACL on buckets was considered as wrong practice (here is some nice post about that). To discourage using them the option BucketOwnerEnforced started to be the default one.

BucketOwnerEnforced - Access control lists (ACLs) are disabled and no longer affect permissions. The bucket owner automatically owns and has full control over every object in the bucket. The bucket only accepts PUT requests that don't specify an ACL or bucket owner full control ACLs, such as the bucket-owner-full-control canned ACL or an equivalent form of this ACL expressed in the XML format.

$ aws s3api get-bucket-ownership-controls --bucket "${bucket_name}"

{
    "OwnershipControls": {
        "Rules": [
            {
                "ObjectOwnership": "BucketOwnerEnforced"
            }
        ]
    }
}

2 - Without BlockPublicPolicy we won't be able to set public access to the bucket. In case of acl authenticated-read (not 100% sure thought) make sure RestrictPublicBuckets is set to false as well.

3 - For public or write acl, policy needs to be changed accordingly (PutObject)

Of course nothing stops you for using ACL, what you need to do is set BucketOwnerPreferred or ObjectWriter for your bucket.

$ aws s3api put-bucket-ownership-controls --bucket "${bucket_name}" --ownership-controls="Rules=[{ObjectOwnership=BucketOwnerPreferred}]"
$ aws s3api put-bucket-acl --bucket "${bucket_name}" --acl public-read
kosciej16
  • 6,294
  • 1
  • 18
  • 29
  • 2 questions, 1. Any ideas why this change affected an already existing bucket? and 2. How do I make these changes through the AWS Management Console? – Sara Fuerst Apr 25 '23 at 14:14
  • What exactly do you try to do with existing bucket? My guess is your upload function tries to create a bucket if doesn't exist. Will answer second question a bit later. – kosciej16 Apr 25 '23 at 14:23
  • When a user uploads a photo it puts the photo in this specific S3 bucket. It doesn't try to create a new one, it has a specific one it needs to go in. – Sara Fuerst Apr 25 '23 at 14:26
  • I need to see a code that is doing a real upload. – kosciej16 Apr 26 '23 at 08:29
  • According to my observations, step #2 sets all four of those boolean flags in `PublicAccessBlockConfiguration` to `false`, so setting `RestrictPublicBuckets` to `false` as well doesn't seem required. – ssc Jul 10 '23 at 11:37
  • @ssc Indeed, I rephrased that point a bit. Thanks! – kosciej16 Jul 10 '23 at 11:45
14

I have a similar issue. Based on previous answers on that question I did these changes :

resources:
  Resources:
    TasksAPIBucket:
      Type: AWS::S3::Bucket
      Properties:
        BucketName: ${self:custom.bucketName}
        ### No more AccessControl
        # AccessControl: PublicRead
        ###  replace it with...
        PublicAccessBlockConfiguration:
          BlockPublicAcls: false
        OwnershipControls:
          Rules:
            - ObjectOwnership: ObjectWriter
        ###
        WebsiteConfiguration:
          IndexDocument: index.html
          ErrorDocument: error.html

See the AWS::S3::Bucket PublicAccessBlockConfiguration and OwnershipControls for more details.

Costin
  • 2,699
  • 5
  • 25
  • 43
5

Please enable ACLs and choose ObjectWriter.

If you use CloudFormation you have to remove AccessControl

Type: AWS::S3::Bucket
Properties:
  # AccessControl: PublicRead << Remove
  OwnershipControls:  # << Add
    Rules:
      - ObjectOwnership: ObjectWriter
Truong
  • 101
  • 3
  • 3
    Can you add an explanation why `AccessControl` should be removed and what `OwnershipControls` is doing? – Jesusaur Apr 27 '23 at 00:14
1

The solution provided by @kosciej16 works however for those of you who use an S3 bucket to store an FE app which requires support of paths (for react router, for instance) this is the policy we had to use in our CloudFormation template to make it work:

     FrontEndBucketPolicy: {
        Type: 'AWS::S3::BucketPolicy',
        Properties: {
          Bucket: 'front-end-bucket',
          PolicyDocument: {
            Statement: [
              {
                Action: ['s3:GetObject'],
                Effect: 'Allow',
                Principal: '*',
                Resource: {
                  'Fn::Join': [
                    '',
                    [{ 'Fn::GetAtt': ['FrontEndBucket', 'Arn'] }, '/*'],
                  ],
                },
              },
              {
                Action: ['s3:ListBucket'],
                Effect: 'Allow',
                Principal: '*',
                Resource: { 'Fn::GetAtt': ['FrontEndBucket', 'Arn'] },
              },
            ],
          },
        },
      },

The important bit is the s3:ListBucket action inside the policy statement.

What I find strange is that the first time you create the stack with CloudFormation it does not allow you to use the PublicRead ACL however on the second attempt you can actually enable it back, which is sort of what was achieved in the previous answer by using the cli.

peterstarling
  • 533
  • 4
  • 12
  • Thanks! s3:ListBucket really solved my issue while keeping all "PublicAccessBlockConfiguration" to true. – snibbo Jun 20 '23 at 21:39