getting error ran out of data in Tensor flow 2.0

Question

I am training a multi class classifier on a dataset : Total Dataset: 1197 images Training Dataset: 599 images Validation/Test Dataset: 598 images

My code is given below -

import tensorflow as tf
import keras_preprocessing
from keras_preprocessing import image
from keras_preprocessing.image import ImageDataGenerator

TRAINING_DIR = "/content/drive/My Drive/solar_images/solar_images/train/"
training_datagen = ImageDataGenerator(
      rescale = 1./255,
        rotation_range=40,
      width_shift_range=0.2,
      height_shift_range=0.2,
      shear_range=0.2,
      zoom_range=0.2,
      horizontal_flip=True,
      fill_mode='nearest')

VALIDATION_DIR = "/content/drive/My Drive/solar_images/solar_images/test/"
validation_datagen = ImageDataGenerator(rescale = 1./255)

train_generator = training_datagen.flow_from_directory(
    TRAINING_DIR,
    target_size=(150,150),
    class_mode='categorical',
  batch_size=126
)

validation_generator = validation_datagen.flow_from_directory(
    VALIDATION_DIR,
    target_size=(150,150),
    class_mode='categorical',
  batch_size=126
)


model = tf.keras.models.Sequential([
    # Note the input shape is the desired size of the image 150x150 with 3 bytes color
    # This is the first convolution
    tf.keras.layers.Conv2D(64, (3,3), activation='relu', input_shape=(150, 150, 3)),
    tf.keras.layers.MaxPooling2D(2, 2),
    # The second convolution
    tf.keras.layers.Conv2D(64, (3,3), activation='relu'),
    tf.keras.layers.MaxPooling2D(2,2),
    # The third convolution
    tf.keras.layers.Conv2D(128, (3,3), activation='relu'),
    tf.keras.layers.MaxPooling2D(2,2),
    # The fourth convolution
    tf.keras.layers.Conv2D(128, (3,3), activation='relu'),
    tf.keras.layers.MaxPooling2D(2,2),
    # Flatten the results to feed into a DNN
    tf.keras.layers.Flatten(),
    tf.keras.layers.Dropout(0.5),
    # 512 neuron hidden layer
    tf.keras.layers.Dense(512, activation='relu'),
    tf.keras.layers.Dense(3, activation='softmax')
])


model.summary()

model.compile(loss = 'categorical_crossentropy', optimizer='rmsprop', metrics=['accuracy'])

history = model.fit(train_generator, 
                    epochs=25, 
                    steps_per_epoch=20, 
                    validation_data = validation_generator, 
                    verbose = 1, 
                    validation_steps=3)

model.save("solar_images_weight.h5")

But I am getting error -

Epoch 1/25
 5/20 [======>.......................] - ETA: 42:21 - loss: 2.3594 - accuracy: 0.3306WARNING:tensorflow:Your input ran out of data; interrupting training. Make sure that your dataset or generator can generate at least `steps_per_epoch * epochs` batches (in this case, 40 batches). You may need to use the repeat() function when building your dataset.
 5/20 [======>.......................] - 2378s 476s/step - loss: 2.3594 - accuracy: 0.3306 - val_loss: 1.1150 - val_accuracy: 0.3069

I am using the below code, but still not working.

batch_size=126
history = model.fit(train_generator, 
                    epochs=25, 
                    steps_per_epoch=int(20/batch_size), 
                    validation_data = validation_generator, 
                    verbose = int(1/batch_size), 
                    validation_steps=3)

I am mainly follow this link How to use repeat() function when building data in Keras?

Any help?

score 0 · Accepted Answer · answered May 31 '20 at 17:48

The problem, as it is well answered in the link you provided, is that steps_per_epoch and validation_steps need to be equal to the total number of data points divided to the batch_size

Try this

history = model.fit(train_generator, 
                    epochs=25, 
                    steps_per_epoch=int(599/batch_size), 
                    validation_data=validation_generator,
                    validation_steps=int(598/batch_size) 
                    )

getting error ran out of data in Tensor flow 2.0

1 Answers1