Keras tuner Bayesian Optmization graph error

Question

I am trying to optimize a convolutional neural network with Bayesian Optimization algorithm provided in keras tuner library.

When I perform the line: tuner_cnn.search(datagen.flow(X_trainRusReshaped,Y_trainRusHot), epochs=50, batch_size=256) I encounter this error: InvalidArgumentError: Graph execution error

One-Hot-Encode y_train and y_test as the following:

y_train = to_categorical(y_train)
y_test = to_categorical(y_test)

X_trainShape = X_train.shape[1]*X_train.shape[2]*X_train.shape[3]
X_testShape = X_test.shape[1]*X_test.shape[2]*X_test.shape[3]
X_trainFlat = X_train.reshape(X_train.shape[0], X_trainShape)
X_testFlat = X_test.reshape(X_test.shape[0], X_testShape)
# One-hot-encoding
Y_trainRusHot = to_categorical(Y_trainRus, num_classes = 2)
Y_testRusHot = to_categorical(Y_testRus, num_classes = 2)

I defined my model builder like that:

datagen = ImageDataGenerator(
    featurewise_center=True,
    featurewise_std_normalization=True,
    rotation_range=180,
    horizontal_flip=True,vertical_flip = True)

def model_builder(hp):
  model = Sequential()
  #model.add(Input(shape=(50,50,3)))
  for i in range(hp.Int('num_blocks', 1, 2)):
    hp_padding=hp.Choice('padding_'+ str(i), values=['valid', 'same'])
    hp_filters=hp.Choice('filters_'+ str(i), values=[32, 64])

    model.add(Conv2D(hp_filters, (3, 3), padding=hp_padding, activation='relu', kernel_initializer='he_uniform', input_shape=(50, 50, 3)))
    model.add(MaxPooling2D((2, 2)))
    model.add(Dropout(hp.Choice('dropout_'+ str(i), values=[0.0, 0.1, 0.2])))
    model.add(Flatten())

    hp_units = hp.Int('units', min_value=25, max_value=150, step=25)
    model.add(Dense(hp_units, activation='relu', kernel_initializer='he_uniform'))
    model.add(Dense(10,activation="softmax"))
    hp_learning_rate = hp.Choice('learning_rate', values=[1e-2, 1e-3])
    hp_optimizer=hp.Choice('Optimizer', values=['Adam', 'SGD'])

    if hp_optimizer == 'Adam':
      hp_learning_rate = hp.Choice('learning_rate', values=[1e-2, 1e-3])

    elif hp_optimizer == 'SGD':
      hp_learning_rate = hp.Choice('learning_rate', values=[1e-2, 1e-3])
      nesterov=True
      momentum=0.9

    model.compile(loss=keras.losses.binary_crossentropy, optimizer=tf.keras.optimizers.Adam(learning_rate=hp_learning_rate), metrics=['accuracy'])
    return model

perform the tuner search:

tuner_cnn = kt.tuners.BayesianOptimization(
    model_builder,
    objective='val_loss',
    max_trials=100,
    directory='.',
    project_name='tuning-cnn')

tuner_cnn.search(datagen.flow(X_trainRusReshaped,Y_trainRusHot), epochs=50, batch_size=256)

I also tried to do:

tuner_cnn.search(X_trainRusReshaped, Y_trainRusHot, epochs=80, validation_data=(X_testRusReshaped, Y_testRusHot), callbacks=[stop_early])

But it does not work neither. Any idea?

On the last line I have this error: `ValueError Traceback (most recent call last) in () ----> 1 tuner_cnn.search(X_trainRusReshaped, Y_trainRusHot, epochs=80, validation_data=(X_testRusReshaped, Y_testRusHot), callbacks=[stop_early])` and `ValueError: `logits` and `labels` must have the same shape, received ((None, 10) vs (None, 2)).` — user979974, Jun 24 '22 at 12:51
yes a regular fit call works: `H = model.fit_generator(datagen.flow(X,Y,batch_size=256),steps_per_epoch=len(X_trainRusReshaped) / batch_size,epochs=epochs,validation_data=(X_testRusReshaped, Y_testRusHot),callbacks=[early_stopping_monitor])` — user979974, Jun 24 '22 at 12:53

score 0 · Accepted Answer · answered Jun 24 '22 at 13:21

0

From the full error message I was able to narrow down where the issue is coming from. The issue is that your last Dense layer has 10 units, which means you expect 10 classes (you even chose the correct activation function given the number of units). However you have Binary CrossEntropy as loss.

So you either have 10 classes and use either categorical or sparse categorical CrossEntropy or you have 2 classes and so the loss is indeed Binary CrossEntropy.

answered Jun 24 '22 at 13:21

DPM

845
7
33

You are totally right. numclass=2, I don't know why I put 10. So the line: `tuner_cnn.search(X_trainRusReshaped, Y_trainRusHot, epochs=80, validation_data=(X_testRusReshaped, Y_testRusHot), callbacks=[stop_early])` works without any errors. I just have a question the datagen cannot be applied inside the `tuner_cnn.search` ? – user979974 Jun 24 '22 at 13:31
Well, theoretically you should be able to pass generators to the search method without a problem. However, when I did it, by chance I (and the stack overflow user that helped me) ended up finding a bug on keras tuner code which is explained in this post: https://stackoverflow.com/questions/72145506/keras-tuning-cant-find-callback. So if you have a similar issue, rest assured it is not your fault. – DPM Jun 24 '22 at 13:52

Keras tuner Bayesian Optmization graph error

1 Answers1