Amazon S3 delete sub-folder recursively

Question

I have a S3 bucket with nested folder and file-structures. I understand that S3 has a flat concept but bear with me as I state my problem clearly.

I'd like to do this:

delete bucket/do-not-delete-folder/delete-this-folder-recursively

I've already poured over the S3 docs and followed the answers given on python - Amazon S3 boto - how to delete folder? - Stack Overflow.

I've tried two different solutions and they produce two different outputs, you guessed it right, none of them doing the thing I want it to do.

Function 1:

This deletes the "do-not-delete-folder" and all its contents recursively. I only want it to delete the sub-folder and not the top-level folder.

I call this with the following params:

DeleteAWSBucketObjects(my-bucket, do-not-delete-folder/delete-this-folder/)

The function is:

def DeleteAWSBucketObjects(bucketName, prefixStr):

    s3 = boto3.resource('s3')
    bucket = s3.Bucket(bucketName)

    try:
        client = boto3.client('s3')
        response = client.list_objects_v2(Bucket=bucketName, Prefix=prefixStr)
        if 'Contents' in response:
            for item in response['Contents']:
                while response['KeyCount'] == 1000:
                    response = client.list_objects_v2(Bucket=bucketName,
                                StartAfter=response['Contents'][0]['Key'],)
                    for item in response['Contents']:
                        client.delete_object(Bucket=bucketName, Key=item['Key'])
    except ClientError as e:
        error_code = int(e.response['Error']['Code'])
        print ("DeleteAWSBucketObjects : hit an exception")
        print (error_code)

    return

Function 2:

Taken from the link mentioned above. I call it with the same params as mentioned above.

Strangely enough, this results in an Access-denied error. I'm running it from within the same code-file in Lambda and same AWS accounts, so I'm not sure why one version of the function deletes everything and the other does not have the access to delete anything.

DeleteAWSBucketObjectsV2(my-bucket, do-not-delete-folder/delete-this-folder/)

Function:

def DeleteAWSBucketObjectsV2(bucketName, prefixStr):

    s3 = boto3.resource('s3')
    bucket = s3.Bucket(bucketName)
    objects_to_delete = s3.meta.client.list_objects(Bucket=bucketName, Prefix=prefixStr)

    delete_keys = {'Objects' : []}
    delete_keys['Objects'] = [{'Key' : k} for k in [obj['Key'] for obj in objects_to_delete.get('Contents', [])]]
    for i in delete_key:
        print("delete_key_array !!!")
        print(str(i))
    try:
        s3.meta.client.delete_objects(Bucket="MyBucket", Delete=delete_keys)
    except ClientError as e:
        error_code = int(e.response['Error']['Code'])
        print ("DeleteAWSBucketObjects : hit an exception")
        print (error_code)

Instead of a Python function, you could use the [AWS Command-Line Interface (CLI)](http://aws.amazon.com/cli/). This command would do it for you: `aws s3 rm s3://MyBucket/do-not-delete-folder/delete-this-folder/ --recursive` — John Rotenstein, May 31 '18 at 05:47

score 0 · Answer 1 · answered May 31 '18 at 03:56

What i understand from your question is , you want to delete your sub folder named delete-this-folder, so here is an easy way to do so :

import boto3
s3 = boto3.resource('s3')
bucket=s3.Bucket('mausamrest');
obj = s3.Object('mausamrest','do-not-delete-folder/delete-this-folder/')
counter=-1


for obj in bucket.objects.filter(Prefix='do-not-delete-folder/delete-this-folder/'):
    print(obj.key)
    try:
        obj.delete()
        counter=counter+1
    except:
        print("error occurred while deleting")

if(counter==-1):
    print("subfolder doesn't exists")
if(counter!=-1):
    print("1 subfolder and "+str(counter)+" items in it deleted")

here mausamrest is the bucketname and do-not-delete-folder/delete-this-folder/ is the sub directories

Amazon S3 delete sub-folder recursively

1 Answers1