Download S3 Files with Boto

Question

I am trying to set up an app where users can download their files stored in an S3 Bucket. I am able to set up my bucket, and get the correct file, but it won't download, giving me the this error: No such file or directory: 'media/user_1/imageName.jpg' Any idea why? This seems like a relatively easy problem, but I can't quite seem to get it. I can delete an image properly, so it is able to identify the correct image.

Here's my views.py

def download(request, project_id=None):
    conn = S3Connection('AWS_BUCKET_KEY', 'AWS_SECRET_KEY')
    b = Bucket(conn, 'BUCKET_NAME')
    k = Key(b)
    instance = get_object_or_404(Project, id=project_id)
    k.key = 'media/'+str(instance.image)
    k.get_contents_to_filename(str(k.key))
    return redirect("/dashboard/")

quick tip: since you're using django you could use [django-storages](http://django-storages.readthedocs.io/en/latest/) it'll be easier — pleasedontbelong, May 25 '16 at 16:11

John Rotenstein · Accepted Answer · 2016-05-25T18:33:12.300

13

The problem is that you are downloading to a local directory that doesn't exist (media/user1). You need to either:

Create the directory on the local machine first
Just use the filename rather than a full path
Use the full path, but replace slashes (/) with another character -- this will ensure uniqueness of filename without having to create directories

The last option could be achieved via:

k.get_contents_to_filename(str(k.key).replace('/', '_'))

See also: Boto3 to download all files from a S3 Bucket

edited May 25 '16 at 18:33

answered May 25 '16 at 17:03

John Rotenstein

241,921
22
380
470

1

Could you possibly give me a simple example. I relatively new to Boto and AWS. If not, that's totally fine. – pepper5319 May 25 '16 at 17:33
This answer should have more up vote. simplest solution – Roshan Oct 07 '16 at 19:38
You can use [`os.makedirs()`](https://docs.python.org/3/library/os.html#os.makedirs) to create a directory, including its parent directories if needed. – jkdev Nov 04 '16 at 05:03

score 3 · Answer 2 · answered Jul 06 '18 at 18:43

Downloading files using boto3 is very simple, configure your AWS credentials at system level before using this code.

client = boto3.client('s3')

// if your bucket name is mybucket and the file path is test/abc.txt
// then the Bucket='mybucket' Prefix='test'

resp = client.list_objects_v2(Bucket="<your bucket name>", Prefix="<prefix of the s3 folder>") 

for obj in resp['Contents']:
    key = obj['Key']
    //to read s3 file contents as String
    response = client.get_object(Bucket="<your bucket name>",
                         Key=key)
    print(response['Body'].read().decode('utf-8'))

    //to download the file to local
    client.download_file('<your bucket name>', key, key.replace('test',''))

replace is to locate the file in your local with s3 file name, if you don't replace it will try to save as 'test/abc.txt'.

This was very helpful, but the C-style comments in Python code are unnerving. — user1717828, Mar 11 '19 at 19:25

score 0 · Answer 3 · answered Feb 06 '20 at 09:30

import os
import boto3
import json

s3 = boto3.resource('s3', aws_access_key_id="AKIAxxxxxxxxxxxxJWB",
                    aws_secret_access_key="LV0+vsaxxxxxxxxxxxxxxxxxxxxxry0/LjxZkN")
my_bucket = s3.Bucket('s3testing')

# download file into current directory
for s3_object in my_bucket.objects.all():
    # Need to split s3_object.key into path and file name, else it will give error file not found.
    path, filename = os.path.split(s3_object.key)
    my_bucket.download_file(s3_object.key, filename)

Download S3 Files with Boto

3 Answers3

Linked