0

I'm trying to create a python script to download images directly from a url to my AWS bucket. What I'm trying to do is akin to the code below:

import logging
import boto3
from botocore.exceptions import ClientError
import os


def url_to_bucket(url, bucket, object_name):
    """Take the image from the URL and put it in a S3 bucket

    :param url: url to save image from
    :param bucket: Bucket to upload to
    :param object_name: S3 object name
    :return: True if file was uploaded, else False
    """

    s3_client = boto3.client('s3')
    try:
        response = s3_client.download_from_url(url, bucket, object_name)
        # I'm mainly trying to get answers on how to do the line above
    except ClientError as e:
        logging.error(e)
        return False
    return True

But in the documentation for boto3, I couldn't find anything close to it except for the how to upload files.

The reason I want a method to do this is because I have a large csv filled with image urls from web scraping and I want to store them in my bucket. But I have McDonald's WiFi quality and as such, want to directly store the images in my AWS bucket instead of downloading it all on my PC and uploading it into my AWS bucket.

Thanks!

  • @AnonCoward This is a valid answer to my question, yes, thank you! Not the ideal option I was looking for, but I could run this on the cloud and bypass my own crappy WiFi. Thanks @AnonCoward! – Nikhil Melgiri Sep 10 '22 at 01:16

0 Answers0