8

I have a directory in my s3 bucket 'test', I want to delete this directory. This is what I'm doing

s3 = boto3.resource('s3')
s3.Object(S3Bucket,'test').delete()

and getting response like this

{'ResponseMetadata': {'HTTPStatusCode': 204, 'HostId': '************', 'RequestId': '**********'}}

but my directory is not getting deleted!

I tried with all combinations of '/test', 'test/' and '/test/' etc, also with a file inside that directory and with empty directory and all failed to delete 'test'.

MikA
  • 5,184
  • 5
  • 33
  • 42
  • What is `S3Bucket`? `s3.Object()` returns a key object. Don't you just need `s3.Bucket('test').delete()`? – AChampion Oct 13 '15 at 14:05
  • S3Bucket is my s3 bucket name – MikA Oct 13 '15 at 14:11
  • 2
    Then I misunderstand what you mean by directory, there are no such things as directories, just buckets and objects (keys). Object names can have structure, e.g. `/this/is/my/object`, but `/this/is/my` doesn't exist independent of the object. How are you testing for existence? – AChampion Oct 13 '15 at 14:17

2 Answers2

36

delete_objects enables you to delete multiple objects from a bucket using a single HTTP request. You may specify up to 1000 keys.

https://boto3.readthedocs.io/en/latest/reference/services/s3.html#S3.Bucket.delete_objects

import boto3

s3 = boto3.resource('s3')
bucket = s3.Bucket('my-bucket')

objects_to_delete = []
for obj in bucket.objects.filter(Prefix='test/'):
    objects_to_delete.append({'Key': obj.key})

bucket.delete_objects(
    Delete={
        'Objects': objects_to_delete
    }
)
Daniel Levinson
  • 391
  • 3
  • 3
  • 4
    This should be accepted answer as it is better than from David Morales – baldr Jan 07 '17 at 21:13
  • 3
    `objects_to_delete = [{'Key': obj.key} for obj in bucket.objects.filter(Prefix='test', Marker='test/path/to/dir')]` – colllin Feb 09 '18 at 01:03
  • [This extension](https://stackoverflow.com/a/43436769/3679900) of the same approach appears to be more comprehensive – y2k-shubham Dec 12 '18 at 19:20
16

NOTE: See Daniel Levinson's answer for a more efficient way of deleting multiple objects.

In S3, there are no directories, only keys. If a key name contains a / such as prefix/my-key.txt, then the AWS console groups all the keys that share this prefix together for convenience.

To delete a "directory", you would have to find all the keys that whose names start with the directory name and delete each one individually. Fortunately, boto3 provides a filter function to return only the keys that start with a certain string. So you can do something like this:

s3 = boto3.resource('s3')
bucket = s3.Bucket('my-bucket-name')
for obj in bucket.objects.filter(Prefix='test/'):
    s3.Object(bucket.name, obj.key).delete()
David Morales
  • 870
  • 7
  • 14