0

I download a file from S3 like this:

s3 = boto3.client('s3')
s3.download_file('testunzipping','DataPump_10000838.zip','/tmp/DataPump_10000838.zip')

For now it always works. However, I wanted to add some sort of error handling. How can I check or get an error message if the download fails. How would I know that there's something wrong?

Does boto3 offer any error handling functions?

I read this: Check if S3 downloading finish successfully but I am looking for alternatives as well.

x89
  • 2,798
  • 5
  • 46
  • 110
  • See [How to handle errors with boto3?](https://stackoverflow.com/questions/33068055/how-to-handle-errors-with-boto3) – jarmod Oct 22 '21 at 17:08

2 Answers2

3

This is just to improve the answer of @balderman, to actually check in exception what caused your BOTO request to fail.

def download_and_verify(Bucket, Key, Filename):
  try:
    os.remove(Filename)
    s3 = boto3.client('s3')
    s3.download_file(Bucket,Key,Filename)
    return os.path.exists(Filename)
  except botocore.exceptions.ClientError as error:
    print(error.response['Error']['Code']) #a summary of what went wrong
    print(error.response['Error']['Message']) #explanation of what went wrong
    return False
Zain Ul Abidin
  • 2,467
  • 1
  • 17
  • 29
  • I get this error ```ClientError: An error occurred (404) when calling the HeadObject operation:```. Which libraries do I need to import for this apart from boto3? I used ```import botocore import os``` – x89 Oct 25 '21 at 09:46
  • did you get this error as a response error message or as an exception in the code itself? – Zain Ul Abidin Oct 27 '21 at 05:35
  • exception for the code itself I think. A normal try catch block works but not with the botocore exception – x89 Oct 27 '21 at 06:50
  • not always true in my case the boto exception system works just fine i am also using botocore to connect with s3 – Zain Ul Abidin Oct 27 '21 at 06:51
1

You can have something like the below. Download and make sure it was created.

import boto3 
import os


def download_and_verify(Bucket, Key, Filename):
  try:
    os.remove(Filename)
    s3 = boto3.client('s3')
    s3.download_file(Bucket,Key,Filename)
    return os.path.exists(Filename)
  except Exception: # should narrow the scope of the exception
    return False
balderman
  • 22,927
  • 7
  • 34
  • 52
  • The download worked individually without this function but when I try to use the function, It gives the exception and later an error that ```[ERROR] FileNotFoundError:``` – x89 Oct 25 '21 at 09:51