Error while downloading file from S3 using boto3

Question

I am encountering an error while downloading a file from S3 programmatically. The code is give below.

The error is: An error occurred (404) when calling the HeadObject operation: Not Found

import csv
import json
import boto3

def get_file(bucket_name, file_name, filename):
    try:
        s3 = boto3.client('s3')
        print("Inside get_file")
        with open(filename, 'w') as data:
            s3.download_file(bucket_name, file_name, filename)
    except Exception as e:
        print(str(e))
        
toName_cfg = 'seo_input_config.cfg'
bucket = 'my-landing-dev'
internal_folder = 'input'
directory_remote = 'Seo'

get_file(bucket, f'{internal_folder}/{directory_remote}/{toName_cfg}', f"{toName_cfg}")

The S3 directory structure is:

my-landing-dev/input/Seo/seo_input_config.cfg

I tried finding a solution by searching the net, but couldn't get any conclusive answer. Please help.

Thanks

Answer 1

Error is most likely because the file you're trying to access doesn't exist in S3.

Change the print statement to: print(f"Inside get_file. bucket_name: {bucket_name}, file_name: {file_name}, filename: {filename}") and confirm. To be absolutely certain that this is the problem call head_object() before trying to download it.

I.e. s3.head_object(Bucket=bucket_name, Key=file_name)

Also you are mixing download to an open fd or to a path.

download to a file by path:

import boto3

s3 = boto3.client('s3')
s3.download_file('BUCKET_NAME', 'OBJECT_NAME', 'FILE_NAME')

download into an already open fd:

s3 = boto3.client('s3')
with open('FILE_NAME', 'wb') as f:
    s3.download_fileobj('BUCKET_NAME', 'OBJECT_NAME', f)

See official examples.