NaN for Keras Tuner score for RandomSearch

Question

I am trying out Keras (2.8.0) autotuner for a regression problem. Here is my code:

import pandas as pd
from tensorflow import keras
from keras import layers, losses
from keras_tuner.tuners import RandomSearch

df = pd.read_csv('./datasets/data.csv')

X = df.iloc[:, :-1]
y = df.iloc[:, -1]


def build_model(hp):
    model = keras.Sequential()
    for i in range(hp.Int('num_layers', 2, 20)):
        model.add(layers.Dense(units=hp.Int('units_' + str(i),
                                            min_value=32,
                                            max_value=512,
                                            step=32),
                               activation='relu'))
    model.add(layers.Dense(1, activation='linear'))
    model.compile(
        optimizer=keras.optimizers.Adam(
            hp.Choice('learning_rate', [1e-2, 1e-3, 1e-4])),
        loss=losses.MeanAbsoluteError(reduction="auto", name="mean_absolute_error"),
        metrics=['mean_absolute_error'])
    return model


tuner = RandomSearch(
    build_model,
    objective='val_mean_absolute_error',
    max_trials=5,
    executions_per_trial=3,
    directory='project',
    project_name='TunerTest')

model_summary = tuner.search_space_summary()

print(model_summary)

from sklearn.model_selection import train_test_split

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, random_state=0)

tuner.search(X_train, y_train,
             epochs=5,
             validation_data=(X_test, y_test))

tuner.results_summary()

Code runs without any errors (so to assume) but in the results, for all the iterations, the Score is nan. Below is the output of one of the runs:

Showing 10 best trials
<keras_tuner.engine.objective.Objective object at 0x7fef945235d0>
Trial summary
Hyperparameters:
num_layers: 12
units_0: 128
units_1: 224
learning_rate: 0.01
units_2: 32
units_3: 32
units_4: 32
units_5: 32
units_6: 32
units_7: 32
units_8: 32
units_9: 32
units_10: 32
units_11: 32
Score: nan

What could I be doing wrong here?

score 2 · Accepted Answer · answered Apr 25 '22 at 08:06

It might be due to your data having nan values or values very close to zero. Another issue could be the variance in your data. Maybe try normalizing it to avoid exploding gradients or even applying gradient clipping. When using random data, your code works and the result is not nan:

import pandas as pd
from tensorflow import keras
from keras import layers, losses
from keras_tuner.tuners import RandomSearch
import numpy as np

X = np.random.random((500, 5))
y = np.random.random((500, 1))

def build_model(hp):
    model = keras.Sequential()
    for i in range(hp.Int('num_layers', 2, 20)):
        model.add(layers.Dense(units=hp.Int('units_' + str(i),
                                            min_value=32,
                                            max_value=512,
                                            step=32),
                               activation='relu'))
    model.add(layers.Dense(1, activation='linear'))
    model.compile(
        optimizer=keras.optimizers.Adam(
            hp.Choice('learning_rate', [1e-2, 1e-3, 1e-4])),
        loss=losses.MeanAbsoluteError(reduction="auto", name="mean_absolute_error"),
        metrics=['mean_absolute_error'])
    return model


tuner = RandomSearch(
    build_model,
    objective='val_mean_absolute_error',
    max_trials=5,
    executions_per_trial=3,
    directory='project',
    project_name='TunerTest')

model_summary = tuner.search_space_summary()

from sklearn.model_selection import train_test_split

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, random_state=0)

tuner.search(X_train, y_train,
             epochs=5,
             validation_data=(X_test, y_test))

tuner.results_summary()

...
Trial summary
Hyperparameters:
num_layers: 20
units_0: 512
units_1: 384
learning_rate: 0.01
units_2: 32
units_3: 32
units_4: 32
units_5: 32
units_6: 32
units_7: 32
units_8: 32
units_9: 32
units_10: 32
units_11: 32
units_12: 32
units_13: 32
units_14: 32
units_15: 32
units_16: 32
units_17: 32
units_18: 32
units_19: 32
Score: 0.24690431356430054

Sorry. I acted early. Even with your code as it is, the Score is still NaN — rishi, Apr 25 '22 at 14:51
I used X = np.random.random((500, 5)) and y = np.random.random((500, 1)). I also removed NaN from my data and tried on that data as well — rishi, Apr 25 '22 at 14:54
So you ran the exact same code as the answer? @rishi..because as you can see.. it is running without any problems in google colab — AloneTogether, Apr 25 '22 at 14:55
Does that mean there is some issue with the version or installation? — rishi, Apr 25 '22 at 14:57
Very likely..can you run the code in colab from the answer and let me know? — AloneTogether, Apr 25 '22 at 14:57

score 0 · Answer 2 · answered May 13 '22 at 16:13

0

If I am not wrong this was an example from Krish Naik's Dl videos; I tried the exact same code and encountered the following issue. I just replaced the null values in the dataset and it worked fine.

answered May 13 '22 at 16:13

Anish Johnson

23
3

You are correct. I think there is a problem with my local env. It worked fine on Colab – rishi May 24 '22 at 14:36

NaN for Keras Tuner score for RandomSearch

2 Answers2