1

Couldn't comment on the initial thread where I adapted this code (Track download progress of S3 file using boto3 and callbacks) so hopefully someone can help me here. I was able to use this code to show a progress bar for file uploads, now I need the do the same thing for file downloads from AWS S3. Any help would be GREATLY APPRECIATED!

I know I need to get the size of the file from S3 instead of from the local file system. I'm sure there is some silly code I need to adjust to make this work. Hopefully, someone can shed some light. :)

def upload_to_aws(local_file, bucket, s3_file):
    s3 = boto3.client('s3', aws_access_key_id=s3ak,
                      aws_secret_access_key=s3sk)

    statinfo = os.stat(local_file)
    up_progress = progressbar.progressbar.ProgressBar(maxval=statinfo.st_size)
    up_progress.start()
    def upload_progress(chunk):
        up_progress.update(up_progress.currval + chunk)

    try:
        s3.upload_file(local_file, bucket, s3_file, Callback=upload_progress)
        up_progress.finish()
        print("Upload Successful")
        return True
    except FileNotFoundError:
        print("The file was not found")
        return False
    except NoCredentialsError:
        print("Credentials not available")
        return False
Kirk Holmes
  • 59
  • 1
  • 5

1 Answers1

3
import os
import boto3
import progressbar
from botocore.exceptions import NoCredentialsError


def download_from_s3(bucket, s3_file, local_file):
    s3 = boto3.client('s3')
    response = s3.head_object(Bucket=bucket, Key=s3_file)
    size = response['ContentLength']
    up_progress = progressbar.progressbar.ProgressBar(maxval=size)
    up_progress.start()

    def upload_progress(chunk):
        up_progress.update(up_progress.currval + chunk)

    try:
        s3.download_file(bucket, s3_file, local_file, 
Callback=upload_progress)
        up_progress.finish()
        print("Download Successful")
        return True
    except FileNotFoundError:
        print("The file was not found")
        return False
    except NoCredentialsError:
        print("Credentials not available")
        return False


resp = download_from_s3(bucket, s3_file, local_file)
lucky
  • 31
  • 4