2

I am trying to train a neural network where I pass in a series of images. I want to create a generator which passes each image in as a numpy array

from skimage import io
image_array = io.imread(url)

But this is only for a specific amazon aws url. I know the standard way using the boto library is something like this: 

s3 = boto3.resource('s3')
s3.meta.client.download_file('mybucket', 'hello.txt', '/tmp/hello.txt')

But again here, it seems like you are pointing towards a specific resource

I want something like this: 

def my_generator():
    for object in s3_bucket():     # does an s3_bucket() iterator like this exist?
        image_array = io.imread(object)
        yield image_array

How could I do this?

BigBoy1337
  • 4,735
  • 16
  • 70
  • 138

1 Answers1

8

You can do something like:

for key in bucket.objects.all():
    print(key.key)
kardaj
  • 1,897
  • 19
  • 19