1

I have an HTML form (implemented in Flask) for uploading files. And I want to store the uploaded files directly to S3.

The relevant part of the Flask implementation is as follows:

@app.route('/',methods=['GET'])
def index():
    return '<form method="post" action="/upload" enctype="multipart/form-data"><input type="file" name="files" /><button>Upload</button></form>'

I then use boto3 to upload the file to S3 as follows:

file = request.files['files']
s3_resource = boto3.resource(
    's3',
     aws_access_key_id='******',
     aws_secret_access_key='*******')

bucket = s3_resource.Bucket('MY_BUCKET_NAME')

bucket.Object(file.filename).put(Body=file)

file is a werkzeug.datastructures.FileStorage object.

But I get the following error when uploading the file to S3:

botocore.exceptions.ClientError: An error occurred (BadDigest) when calling the PutObject operation (reached max retries: 4): The Content-MD5 you specified did not match what we received.

How can I upload the file to S3?

John Rotenstein
  • 241,921
  • 22
  • 380
  • 470
Saurabh Sharma
  • 804
  • 6
  • 16
  • 42

1 Answers1

4

Since you are using Flask web framework, the file variable is of type werkzeug.datastructures.FileStorage.

I think the problem is that the put() method expects a byte sequence or a file-like object as its Body argument. So, it does not know what to do with the Flask's FileStorage object.

A possible solution could be to use file.read() to get a sequence of bytes and then pass it to put():

bucket.Object(file.filename).put(Body=file.read())
SergiyKolesnikov
  • 7,369
  • 2
  • 26
  • 47
  • Yuppp!!! its working. Now file is getting stored in to S3 bucket. but when I open that its blank.... image is not appearing, it seems like file not written properly. and its size is showing `0B` – Saurabh Sharma Feb 15 '20 at 13:38
  • This may be helpful https://stackoverflow.com/questions/40336918/how-to-write-a-file-or-data-to-an-s3-object-using-boto3 – SergiyKolesnikov Feb 16 '20 at 08:03
  • @SaurabhSharma where you able to solve this issue, i am facing the same situation. In my case the size of image is altered as i am not able to read the content of file storage object properly. – Utkarsh Apr 18 '23 at 12:32
  • Hi @Utkarsh, Make sure you add `fileObj.seek(0)` before uploading file into S3 `filename = secure_filename(fileObj.filename) key_path = os.path.join(filePath,filename) s3_resource = boto3.resource( 's3', aws_access_key_id=app.config['S3_ACCESS_KEY'], aws_secret_access_key=app.config['S3_SECRET_KEY'] ) bucket = s3_resource.Bucket('bucket-name') fileObj.seek(0) bucket.upload_fileobj(fileObj, Key=key_path)` – Saurabh Sharma Apr 19 '23 at 03:02
  • @SaurabhSharma the image is of type filestorage and when we do f.read() we are reading the image content along with the metadata, so when the image is getting stored in the S3 bucket the size of image is not matching the actual size. – Utkarsh Apr 20 '23 at 10:59