How to force delete a non empty S3 bucket with versioning enabled?

Question

I have this code block in my CDK file:

const artifactBucket = new s3.Bucket(this, "MyBucket", {
  bucketName: "cdk-cws-pipeline-artifacts",
  removalPolicy: cdk.RemovalPolicy.DESTROY,
  versioned: true, // a bucket used as a source in CodePipeline must be versioned
});

// code omitted for brevity

const pipeline = new codepipeline.Pipeline(this, "Pipeline", {
  artifactBucket: artifactBucket,
  // code omitted for brevity
});

Everything was working fine until I try to delete the stack. I got this error:

Are you sure you want to delete: cdk-staging-CwsPipeline (y/n)? y
cdk-staging-CwsPipeline: destroying...
6:11:42 PM | DELETE_FAILED        | AWS::S3::Bucket             | MyBucketF68F3FF0
The bucket you tried to delete is not empty. You must delete all versions in the bucket. (Service: Amazon S3; Status Code: 409; Error Code: BucketNotEmpty; R
equest ID: 1P2N8WFPFQ8R9J4W; S3 Extended Request ID: uQWJgUnsgxVAXoEM95L28jm8p0qGbX+F0oNzMt8BH6Bg5G4Vd6mbNenJdDmPIgtrhXJFUbveSpk=)

    new Bucket (/Users/zulh/hg/cws-cdk/node_modules/@aws-cdk/aws-s3/lib/bucket.ts:1049:22)
    \_ new CiCdStack (/Users/zulh/hg/cws-cdk/lib/ci-cd-stack.ts:128:32)
    \_ Object.<anonymous> (/Users/zulh/hg/cws-cdk/bin/cws-cdk.ts:99:24)
    \_ Module._compile (internal/modules/cjs/loader.js:1200:30)
    \_ Module.m._compile (/Users/zulh/hg/cws-cdk/node_modules/ts-node/src/index.ts:858:23)
    \_ Module._extensions..js (internal/modules/cjs/loader.js:1220:10)
    \_ Object.require.extensions.<computed> [as .ts] (/Users/zulh/hg/cws-cdk/node_modules/ts-node/src/index.ts:861:12)
    \_ Module.load (internal/modules/cjs/loader.js:1049:32)
    \_ Function.Module._load (internal/modules/cjs/loader.js:937:14)
    \_ Function.executeUserEntryPoint [as runMain] (internal/modules/run_main.js:71:12)
    \_ main (/Users/zulh/hg/cws-cdk/node_modules/ts-node/src/bin.ts:227:14)
    \_ Object.<anonymous> (/Users/zulh/hg/cws-cdk/node_modules/ts-node/src/bin.ts:513:3)
    \_ Module._compile (internal/modules/cjs/loader.js:1200:30)
    \_ Object.Module._extensions..js (internal/modules/cjs/loader.js:1220:10)
    \_ Module.load (internal/modules/cjs/loader.js:1049:32)
    \_ Function.Module._load (internal/modules/cjs/loader.js:937:14)
    \_ Function.executeUserEntryPoint [as runMain] (internal/modules/run_main.js:71:12)
    \_ /Users/zulh/.nvm/versions/node/v14.4.0/lib/node_modules/npm/node_modules/libnpx/index.js:268:14

6:11:42 PM | DELETE_FAILED        | AWS::CloudFormation::Stack  | cdk-staging-CwsPipeline
The following resource(s) failed to delete: [MyBucketF68F3FF0].

 ❌  cdk-staging-CwsPipeline: destroy failed Error: The stack named cdk-staging-CwsPipeline is in a failed state: DELETE_FAILED (The following resource(s) failed to delete: [MyBucketF68F3FF0]. )
    at Object.waitForStack (/Users/zulh/.nvm/versions/node/v14.4.0/lib/node_modules/aws-cdk/lib/api/util/cloudformation.ts:266:11)
    at processTicksAndRejections (internal/process/task_queues.js:97:5)
    at Object.destroyStack (/Users/zulh/.nvm/versions/node/v14.4.0/lib/node_modules/aws-cdk/lib/api/deploy-stack.ts:365:28)
    at CdkToolkit.destroy (/Users/zulh/.nvm/versions/node/v14.4.0/lib/node_modules/aws-cdk/lib/cdk-toolkit.ts:251:9)
    at main (/Users/zulh/.nvm/versions/node/v14.4.0/lib/node_modules/aws-cdk/bin/cdk.ts:270:16)
    at initCommandLine (/Users/zulh/.nvm/versions/node/v14.4.0/lib/node_modules/aws-cdk/bin/cdk.ts:185:9)
The stack named cdk-staging-CwsPipeline is in a failed state: DELETE_FAILED (The following resource(s) failed to delete: [MyBucketF68F3FF0]. )

Is there any way to force delete the S3 bucket using CDK TypeScript code when running cdk destroy stack-name?

score 13 · Answer 1 · answered Mar 30 '21 at 14:14

13

Now, you can delete the bucket as well as the contents of the Bucket which got created with the help of CloudFormation (i.e. cdk deploy)

You just have to add autoDeleteObjects: true parameter while creating the S3 object.

Here is the Sample TypeScript Code:

new s3.Bucket(this, 'MyFirstBucket', {
  versioned: true,
  removalPolicy: cdk.RemovalPolicy.DESTROY,
  autoDeleteObjects: true
});

Here is the reference link from the official AWS Documentation: https://docs.aws.amazon.com/cdk/api/latest/docs/aws-s3-readme.html#bucket-deletion

answered Mar 30 '21 at 14:14

raj

141
2
4

2

If the bucket is already created as OP said, this answer doesn't add any value. – Praneeth Peiris Mar 31 '21 at 12:17
@PraneethPeiris Yes, so one can destroy that previous deployment and add the parameter and deploy again if whosoever wants the feature – raj Apr 01 '21 at 15:34

score 7 · Answer 2 · answered Jul 02 '20 at 11:04

Unfortunately you can't do this easily. You can't delete a non-empty bucket using CloudFormation or CDK, as you already experienced.

If you want your stack to continue deletion, rather than fail on the bucket, the easiest option is to add RemovalPolicy of RETAIN. This will simply skip the bucket. Stack will get deleted, but bucket will remian. Later you can use CLI or console to delete it.

If you really want to do everything from CDK or CloudFormation you would have to develop a custom solution for that. This can be done using custom-resource. The resource would be based on a lambda function which you would have to develop and deploy. The lambda would be able to empty the bucket when stack is deleted. Example code of how to delete versioned objects in a bucket using boto3 here.

Hope this helps.

score 6 · Answer 3 · answered Feb 10 '21 at 06:39

6

CDK now allows for removal of objects on delete:

https://docs.aws.amazon.com/cdk/api/latest/docs/aws-s3-readme.html#bucket-deletion

answered Feb 10 '21 at 06:39

Court Schuett

145
1
3

score -4 · Answer 4 · answered Feb 10 '21 at 07:30

To delete an S3 bucket

Sign in to the AWS Management Console and open the Amazon S3 console at https://console.aws.amazon.com/s3/.

In the Buckets list, select the option next to the name of the bucket that you want to delete, and then choose Delete at the top of the page.

On the Delete bucket page, confirm that you want to delete the bucket by entering the bucket name into the text field, and then choose Delete bucket.

Note If the bucket contains any objects, empty the bucket before deleting it by selecting the empty bucket configuration link in the This bucket is not empty error alert and following the instructions on the Empty bucket page. Then return to the Delete bucket page and delete the bucket.

You can delete it on AWS CLI, SDK or like i posted it... interface console https://docs.aws.amazon.com/AmazonS3/latest/userguide/delete-bucket.html

How to force delete a non empty S3 bucket with versioning enabled?

4 Answers4