Unable to download files from S3 bucket

Question

I want to download all files from S3 bucket which are in this path All Buckets /abc/or/uploads

I tried with this snippet but only managed to get files in All Buckets /abc

Changing the path in bucket_list = bucket.list('or/uploads') this line is not working? why?

import boto
import sys, os
import logging

bucket_name = 'abc'
conn = boto.connect_s3('XXXXXXXXXXXXXX', 'XXXXXXXXXXXXXXXXX+XXXXXXXXXXXXXX')
bucket = conn.get_bucket(bucket_name)
bucket_list = bucket.list('or/uploads')

for key in bucket.list():
    try:
        res = key.get_contents_to_filename(key.name)
        print "done"
    except:
        logging.info(key.name + ":" + "FAILED")

What do you mean, it is not working? What error are you getting? You're not using `bucket_list` anywhere in the code you posted. — magni-, Oct 07 '16 at 06:48
@magni- No errors, the program just runs and finish execution but no files gets downloaded when i use `bucket_list ` in for loop and when i use the above code files in this path only gets downloaded `All Buckets /abc` — prashantitis, Oct 07 '16 at 06:53
where exactly you are saying? in which line `bucket_list = bucket.list('or/uploads')` I think i have already used this path — prashantitis, Oct 07 '16 at 06:59

prashantitis · Accepted Answer · 2016-10-07T07:52:54.460

Amazon S3 does not have folders/directories. It is a flat file structure. To maintain the appearance of directories, path names are stored as part of the object Key (filename).

Make sure the place where you are downloading the files, have exact same structure i.e.. /abc/or/uploads

snippet

for key in bucket.list('or/uploads'):
    try:
        res = key.get_contents_to_filename(key.name)
        print "done"
    except:
        logging.info(key.name + ":" + "FAILED")

Source

Unable to download files from S3 bucket

1 Answers1