Generator not being recognized when passing validation data to .fit in Keras Sequential

Question

The exact error:

ValueError: When passing validation_data, it must contain 2 (x_val, y_val) or 3 (x_val, y_val, val_sample_weights) items, however it contains 39 items

I literally cannot find that error anywhere except the source code.

model.fit(  train_x
            , train_y
            , epochs=1
            , validation_data=validation_data_flow
            , callbacks=[checkpointer]
        )

validation_data is a DirectoryIterator, by flow_from_directory

validation_data_flow = ImageDataGenerator().flow_from_directory(
        validation_data_dir,
        target_size = (img_width, img_height),
        batch_size = batch_size,
        class_mode = 'categorical')

You may want to check out `fit_generator` from https://keras.io/models/sequential/ . — Sayandip Dutta, Sep 25 '19 at 07:47
fit_generator only accept generators for both train and validation. Here, I am passing train_x and train_y as numpy arrays, and validation data as a generator. Is that what's causing the problem? — Faraaz, Sep 25 '19 at 07:50
I am not sure, but it feels that way. `fit` does not work with generators. — Sayandip Dutta, Sep 25 '19 at 07:53
Yes, that is causing the problem, both train and validation data have to be provided with the same format (numpy arrays or generators). — Dr. Snoopy, Sep 25 '19 at 11:34

score 1 · Answer 1 · answered Sep 25 '19 at 19:47

It's not exactly true that you have to train and validate on the same kind data, such as array or generator, but you can't do that with the same function.

You could train on your array, and validate on your generator but this would take 2 different function calls, and means you won't get validation metrics after every epoch if you're using multiple epoch. You can work around this, with something like:

for i in range(epochs):
    model.fit(train_x, train_y, epochs=1, callbacks=[checkpointer])
    loss = model.evaluate_generator(validation_data_flow)

    print("Validation loss for epoch %s was %s" % (i, loss))

score 0 · Accepted Answer · answered Sep 25 '19 at 18:26

Validation and training data need to be the same type, either both generators or both ndarrays. To fix this, you need to convert one to the other type. Look at this answer for how to convert a generator to an ndarray. To convert an ndarray to a generator, use ImageDataGenerator.flow().

Generator not being recognized when passing validation data to .fit in Keras Sequential

2 Answers2