1

I am getting two very different results from the model coded in Sequential and Function APIs. What could go wrong with my code?

I started my code with the following:

np.random.seed(42)
tf.random.set_seed(42)
random.seed(42)

Then, I coded the following model in sequential API for 18 input features and 7 output:

model = keras.models.Sequential([
    Input(shape=18),
    Dense(1000, activation = "relu"),
    BatchNormalization(),
    Dense(1000, activation = "relu"),
    BatchNormalization(),
    Dense(7)
])
model.compile(optimizer=keras.optimizers.Adam(learning_rate=1e-4), loss=keras.losses.mse)
history = model.fit(X_tr, Y_tr, epochs=1000, validation_data=(X_val, Y_val))

I got the overall minimum validation MSE of 0.417.

However, then I had to go for Functional API to find the MSE separately for each output. So I tried to code the same above model as follows:

targets = ('output1', 'output2', 'output3', 'output4', 'output5', 'output6', 'output7')

inp = Input(shape=18)
hidden1 = Dense(1000, activation = "relu")(inp)
hidden2 = BatchNormalization()(hidden1)
hidden3 = Dense(1000, activation = "relu")(hidden2)
hidden4 = BatchNormalization()(hidden3)
out = [Dense(1, name = nn)(hidden4) for nn in targets ]
modelF = keras.Model(inputs = [inp], outputs = [out])

modelF.compile(optimizer=keras.optimizers.Adam(learning_rate=1e-4), loss=keras.losses.mse)
history = modelF.fit(X_tr, Y_tr, epochs=1000, validation_data=(X_val, Y_val))

But with this Functional API, I am getting the minimum MSE more than 1284 for each output. What is wrong with the code? How can I modify the Functional API code to get the same model if these aren't the same but still get 7 MSEs instead of just 1?

MM Khan
  • 203
  • 1
  • 2
  • 7
  • 1
    These are not the same models, its not the same to have one output Dense of 7 neurons, than 7 output Dense with 1 neuron each, specially due to the loss. – Dr. Snoopy Dec 08 '22 at 08:08
  • @Dr.Snoopy So could you please help in how to modify the code of Functional API to make it the same model as that of the Sequential API but which gives 7 separate output MSEs instead of just one? I will update the question accordingly and you may give an answer to that. Thanks. – MM Khan Dec 09 '22 at 02:04
  • @Dr.Snoopy I am getting exactly the same number of parameters from both models as follows: Total params: 1,035,007 Trainable params: 1,031,007 Non-trainable params: 4,000 – MM Khan Dec 09 '22 at 02:26
  • The loss is non-linear, so changing the 7x1 outputs vs 1x7 output is not the same thing. – Dr. Snoopy Dec 09 '22 at 02:30
  • @Dr.Snoopy I do not understand how the performance got so worse in the Functional API. I mean from MSE of 0.417 to the MSE of at least 1284. That is strange! – MM Khan Dec 09 '22 at 02:48

1 Answers1

0

  1. There is a logical error behind getting so high MSE in the Functional API code. Just replace the following line

    modelF = keras.Model(inputs = [inp], outputs = [out])

with the following:

modelF = keras.Model(inputs = [inp], outputs = out)

  1. The loss will be provided for all the outputs separately in compilation stage i.e.,

    model.compile(optimizer=keras.optimizers.Adam(learning_rate=1e-4), loss=["mse", "mse", "mse", "mse", "mse", "mse", "mse"])

  2. The output labels will also be provided separately for all stages of train, validation, and test stages. For the train and validation stages, you will write the following:

    historyF = modelF.fit(x = X_tr, y = [Y_tr[:,0], Y_tr[:,1], Y_tr[:,2], Y_tr[:,3], Y_tr[:,4], Y_tr[:,5], Y_tr[:,6] ], epochs=1000, validation_data=(X_val, [Y_val[:,0], Y_val[:,1], Y_val[:,2], Y_val[:,3], Y_val[:,4], Y_val[:,5], Y_val[:,6] ] ) )

This will give the normal MSEs similar to that of the Sequential API code.

MM Khan
  • 203
  • 1
  • 2
  • 7