Get only file names from s3 bucket folder

Question

I have a s3 bucket named 'Sample_Bucket' in which there is a folder called 'Sample_Folder'. I need to get only the names of all the files in the folder 'Sample_Folder'.

I am using the following code to do so -

import boto3
s3 = boto3.resource('s3', region_name='us-east-1', verify=False)
    bucket = s3.Bucket('Sample_Bucket')
    for files in bucket.objects.filter(Prefix='Sample_Folder):
        print(files)

The variable files contain object variables which has the filename as key.

s3.ObjectSummary(bucket_name='Sample-Bucket', key='Sample_Folder/Sample_File.txt')

But I need only the filename. How do I extract that? Or is there any other way to do it?

Does this answer your question? [Listing contents of a bucket with boto3](https://stackoverflow.com/questions/30249069/listing-contents-of-a-bucket-with-boto3) — ZF007, Dec 07 '19 at 13:02

score 12 · Answer 1 · answered Jul 19 '20 at 19:22

12

Here you go.

import boto3


bucket = "Sample_Bucket"
folder = "Sample_Folder"
s3 = boto3.resource("s3")
s3_bucket = s3.Bucket(bucket)
files_in_s3 = [f.key.split(folder + "/")[1] for f in s3_bucket.objects.filter(Prefix=folder).all()]

answered Jul 19 '20 at 19:22

rob

249
3
3

Thanks. I love using boto3.resource when possible. – cdabel May 05 '21 at 18:40

score 5 · Accepted Answer · edited Aug 29 '23 at 12:13

5

You should use list_object_v2 which gives you the list from the defined prefix used.

... snippet ...

filenames = []

def get_filenames(s3):
    result = s3.list_objects_v2(Bucket=bucket, Prefix=prefix)
    for item in result['Contents']:
        files = item['Key']
        print("file: ", files)
        filenames.append(files)   #optional if you have more filefolders to got through.
    return filenames

get_filenames(my_bucketfolder)

edited Aug 29 '23 at 12:13

Sergi

11
1
7

answered Dec 07 '19 at 12:57

ZF007

3,708
8
29
48

3

That can get max 1000 objects. It is unbelievable that they hardcoded this limit and you cant change. – gench Jul 17 '20 at 08:29

score 1 · Answer 3 · answered Dec 30 '20 at 13:20

for myself, i made a function that you might find helpful:

import boto3


s3_client = boto3.client('s3')


def list_objects_without_response_metadata(**kwargs):
    ContinuationToken = None
    while True:
        if ContinuationToken:
            kwargs["ContinuationToken"] = ContinuationToken
        res = s3_client.list_objects_v2(**kwargs)
        for obj in res["Contents"]:
            yield obj
        ContinuationToken = res.get("NextContinuationToken", None)
        if not ContinuationToken:
            break


file_names = [obj["Key"] for obj in list_objects_without_response_metadata(Bucket='Sample_Bucket', Prefix='Sample_Folder')]

score 0 · Answer 4 · answered Jul 06 '21 at 14:33

If you don't want to use boto3.client and prefer boto3.resource you can use this snippet to list all the directory names within a directory.

import boto3

s3 = boto3.resource('s3')
bucket = s3.Bucket("Sample_Bucket")
res = bucket.meta.client.list_objects(Bucket=bucket.name, Delimiter='/', Prefix = "Sample_Folder/"')
for o in res.get('CommonPrefixes'):
    print(o.get('Prefix'))

Get only file names from s3 bucket folder

4 Answers4