How to download an image from s3 and store it as a variable instead of storing it in a directory?

Question

I have several images in a bucket on S3 and my task is to generate thumbnails for the same and upload them back to another directory in the same bucket. I shall be using AWS Lambda as I want to auto execute my python script for thumbnail generation if any new images are uploaded.

This is what I have so far:

from botocore.exceptions import NoCredentialsError, ClientError
from os import listdir
from PIL import Image
import boto3
import botocore

def get_img_list():

    s3 = boto3.client('s3')
        response = s3.list_objects_v2(
            Bucket='imagesforgreendub',
            Prefix='photo_test/',
            )

    for val in response['Contents']:
        if val['Key'] != 'photo_test/':
            temp = val['Key']
            img_name = temp.split('/')
            if img_name[1] != '':
                thumb_name = 'thumb_' + img_name[1]
                print(thumb_name)
                dld_img(val['Key'], thumb_name)

The above function just gets the names of images from my S3 bucket.

The next function is to download the image and where I am facing trouble. The download_file requires me to specify a name/directory for storing the image. The problem is I plan to run it on AWS lambda and I don't think it will let me store the image. So I wish to retrieve the image and store it in a variable that I can pass onto my thumbnail generation function.

I found this thread (Retrieve S3 file as Object instead of downloading to absolute system path) that said it retrieves the content as string but how will that work when I have an image file and I am using PIL/Pillow to generate the thumb so i need to pass an image as the parameter.

def dld_img(key_name, thumb_name):
    s3 = boto3.resource('s3')

    try:
        local_img = s3.Bucket('imagesforgreendub').download_file(key_name, 'my_local_image.jpg')


    except botocore.exceptions.ClientError as e:
        if e.response['Error']['Code'] == "404":
            print("The object does not exist.")
        else:
            raise

def img_thumb(img, thumb_name):

    MAX_SIZE = (100, 100)

    img.thumbnail(MAX_SIZE)
    s3 = boto3.client('s3')
    s3.upload_file(img, 'imagesforgreendub', '%s/%s' % ('thumb_test', thumb_name))

So to sum it up: How do I download an image from s3 and store it in a variable that I can then pass of to my img_thumb function?

Answer 1

Can try the following to read image from s3 as image in Pillow:

import os

import boto3

from PIL import Image

s3 = boto3.resource('s3')

def image_from_s3(bucket, key):

    bucket = s3.Bucket(bucket)
    image = bucket.Object(key)
    img_data = image.get().get('Body').read()

    return Image.open(io.BytesIO(img_data))

To call it you can use:

 img = image_from_s3(image_bucket, image_key)

Where img will be Pillow' Image which you should be able to pass to your img_thumb.