How to build a model having multiple inputs and a single output using Keras

Question

I am trying to use the functional api of Keras to build a model having multiple inputs and a single output. The goal is to combine each row of each input to predict the corresponding output (either 1 or 0).
for example concatenate(inputs_1[0], and inputs_2[0]) and predict output outputs[0]

My data structure looks like this :

inputs_1 = [[[-18.73, 8.98, 0.29, 0.23],[58.50, 28.31, 45.89, -1.62], [48.70, 21.31, 25.89, 1.62]], 
           [[-18.73, 8.98, 0.29, 0.65],[58.50, 28.31, 45.89, -1.62], [48.70, 21.31, 25.89, 1.62]],
           [[-18.73, 8.98, 0.29, 9,3],[58.50, 28.31, 45.89, -1.62], [48.70, 21.31, 25.89, 1.62]],
            ...
            [[-18.73, 8.98, 0.29, 8.93],[58.50, 28.31, 45.89, -1.62], [48.70, 21.31, 25.89, 1.62]]]
            

inputs_2 = [[[0.29, 0.23], [28.31, -1.62]], 
           [[8.98, 0.65], [21.31, 1.62]],
           [[18.50, -1.62], [25.89, 1.62]],
            ...
            [[-48.73, 8.98], [48.70, 1.62]]]

outputs = [1,
           1, 
           0, 
           ...
           0]

I am having some difficulties building the model, the first one arises when I want to reshape the data.

from sklearn.preprocessing import MinMaxScaler
scaler = MinMaxScaler()

# scale our data set so that every observation is between 0 and 1
training_data = scaler.fit_transform(inputs_1.reshape(-1, 1))

But list object has no attribute reshape

I have read this functional api doc but it didn't help me much. However, I now know that I will merge all available features into a single large vector via concatenation. How to do so with these nested arrays?
Another difficulty was to split the data into train, validation, and test. Articles I have found do it based on a single input data. Is there a way to spit multiple inputs of data?

How could I define the layer, for this case to build the model?
How could I use the api to build my model?
Any hints or a model's skeleton will be welcome.
Thank you in advance.

Answer 1

There are a ton of questions you are asking which is usually not inline with SO guidelines. It would be better to tackle (search first, ask later if not found) each question separately.

Still, just to help you get started, I'll try to answer them in order you asked. First, there are few issues with your code -

1. You have to first convert inputs_1 and inputs_2 into a numpy array before using reshape. Use inputs_1 = np.array(inputs_1) and same for input_2.

2. Next, you want to apply min max scaler, but you use reshape(-1,1). This doesn't make sense, since min-max scaling is for each feature independent to the other. I have shown how you can reshape for proper min-max scaling.

3. You also ask about train-test split. You can simply use sklearn's train_test_split the way you usually use it with more inputs.

4. Lastly, you ask about the multi-input Keras functional API. The documentation is really really well done (in fact that is the philosophy of the author of keras as well, to make deep learning easy to learn and implement). I have added an example below -

#Dummy data (USE YOUR OWN DATA HERE AS NUMPY ARRAYS
import numpy as np
X1 = np.random.random((1000, 3, 4))
X2 = np.random.random((1000, 2, 2))
y = np.random.randint(0, 2, (1000,))

Scaling using min-max scaler

#Scaling individual features by respective min max for 3D tensors
from sklearn.preprocessing import MinMaxScaler

#Have separate scaler objects for each input data
scaler1 = MinMaxScaler()
scaler2 = MinMaxScaler()

#Ensure that in reshape to 2D matrix, you keep the number of features separate
#as min-max scaler works on each feature's respective min-max values
#Then, reshape it back to the 3D dataset

X1_scaled = scaler1.fit_transform(X1.reshape(-1,X1.shape[-1])).reshape(X1.shape)
X2_scaled = scaler1.fit_transform(X2.reshape(-1,X2.shape[-1])).reshape(X2.shape)

print(X1_scaled.shape, X2_scaled.shape)

(1000, 3, 4) (1000, 2, 2)

Train test split for multiple inputs/outputs

from sklearn.model_selection import train_test_split

X1_train, X1_test, X2_train, X2_test, y_train, y_test = train_test_split(X1_scaled, X2_scaled, y, test_size=0.2)

[i.shape for i in (X1_train, X1_test, X2_train, X2_test, y_train, y_test)]

[(800, 3, 4), (200, 3, 4), (800, 2, 2), (200, 2, 2), (800,), (200,)]

Keras functional API with 2 inputs and 1 output

from tensorflow.keras import layers, Model, utils

inp1 = layers.Input((3,4))
inp2 = layers.Input((2,2))
x1 = layers.Flatten()(inp1)
x2 = layers.Flatten()(inp2)
x = layers.concatenate([x1, x2])
x = layers.Dense(32)(x)
out = layers.Dense(1, activation='sigmoid')(x)

model = Model([inp1, inp2], out)

utils.plot_model(model, show_layer_names=False, show_shapes=True)

Training the multi-input model

model.compile(loss='binary_crossentropy', optimizer='adam')
model.fit([X1_train, X2_train], y_train, epochs=4)

Epoch 1/4
25/25 [==============================] - 0s 674us/step - loss: 0.7310
Epoch 2/4
25/25 [==============================] - 0s 753us/step - loss: 0.7198
Epoch 3/4
25/25 [==============================] - 0s 842us/step - loss: 0.7147
Epoch 4/4
25/25 [==============================] - 0s 2ms/step - loss: 0.7079