1

I'm currently trying to fit a model using a training and testing data generator, flowed from two directories. 

train_datagen = ImageDataGenerator(rescale = 1./255, horizontal_flip = True)
# output: Found 440 images belonging to 2 classes.
test_datagen = ImageDataGenerator(rescale=1./255)
# output: Found 554 images belonging to 2 classes.

training_generator = train_datagen.flow_from_directory('dir', batch_size=20, class_mode='categorical')
testing_generator = test_datagen.flow_from_directory('dir', batch_size=30, class_mode='categorical')

When I try to run :

history = model.fit_generator(training_generator, epochs=150, validation_data=testing_generator, callbacks= [early_stopping_monitor, model_checkpoint])

I get to the last batch and receive one of two StopIteration errors.

For a batch_size of 20, I get:

Epoch 1/150
21/22 [==================>..] - ETA: 0s - loss: 0.8482 - acc: 0.5619
Traceback (mostrecent call last):
...
File "...\keras\engine\\training_generator.py", line 309, in evaluate_generator
    generator_output=next(output_generator)

File "..\keras\utils\data_utils.py", line 583, in get
    six.raise_from(StopIteration(e), e)

StopIteration: tile cannot extend outside image

For a batch_size of 24, I get:

Epoch 1/150
18/19 [==================>..] - ETA: 0s - loss: 0.8482 - acc: 0.5619
Traceback (mostrecent call last):
...
File "...\keras\engine\\training_generator.py", line 309, in evaluate_generator
    generator_output=next(output_generator)

File "..\keras\utils\data_utils.py", line 583, in get
    six.raise_from(StopIteration(e), e)

StopIteration: -2

I don't understand. I saw this : StopIteration: generator_output = next(output_generator), but that seemed to be a person making their own datagenerator, so the coder is responsible for making the generator continuously able to provide data. But I'm using flow_from_directory, so shouldn't that be smart enough to figure out how to always provide data?

I also read that perhaps the issue might be with PIL trying to access out of bounds parts of my images. I wrote a script to go through my image directories and open and verify them with PIL according to: How to check if a file is a valid image file?, but all the directories don't raise an exception.

What am I missing here?

Hair of Slytherin
  • 397
  • 4
  • 21
  • Since there are ~1000 images, you can open the folder containing the images and manually scan through the images in thumbnail view to see if there are any ill-formed images. – Manoj Mohan Mar 29 '19 at 05:27
  • Is that what you suppose is happening? Is there someway I can put a callback into the fit_generator function to output the file it's choking on? – Hair of Slytherin Mar 29 '19 at 17:56

0 Answers0