Keras Neural Network model for binary classification with 3D tensor time series as input

Question

I am building a NN model using keras and tensorflow for time series binary classification. This is how my input looks like, with a shape of (124,4,591):

      |      Col 1      |    Col 2        |    Col 3        |    Col 4        |
Row 1 | [x1, ..., x591] | [x1, ..., x591] | [x1, ..., x591] | [x1, ..., x591] |
Row 2 | [x1, ..., x591] | [x1, ..., x591] | [x1, ..., x591] | [x1, ..., x591] |
Row 3 | [x1, ..., x591] | [x1, ..., x591] | [x1, ..., x591] | [x1, ..., x591] |

I split my data into X_train, X_test, y_train and y_test. I also encoded my labels from ['True', 'False'] into [0, 1] using LabelEncoder() and OneHotEncoder().

x = np.stack((np.vstack(x[:,0]),np.vstack(x[:,1]),np.vstack(x[:,2]),np.vstack(x[:,3])))
x = x.reshape((124,4,591))
y = table_raw_ptpt['Binding Known']
X_train, X_test, y_train, y_test = train_test_split(x, y, test_size = 0.3, random_state = 0)

X_train.shape returns (86, 4, 591). Encoding for labels:

label_encoder = LabelEncoder()
integer_encoded_train = label_encoder.fit_transform(array(y_train))
integer_encoded_test = label_encoder.fit_transform(array(y_test))

onehot_encoded_y_train = OneHotEncoder(sparse=False)
integer_encoded_train = integer_encoded_train.reshape(len(integer_encoded_train), 1)
onehot_encoded_y_train = onehot_encoded_y_train.fit_transform(integer_encoded_train)

onehot_encoded_y_test = OneHotEncoder(sparse=False)
integer_encoded_test = integer_encoded_test.reshape(len(integer_encoded_test), 1)
onehot_encoded_y_test = onehot_encoded_y_test.fit_transform(integer_encoded_test)

onehot_encoded_y_train.shape returns (86, 2).

Here is my NN:

model = Sequential()
model.add(Dense(86, activation='relu', input_shape=(4,591)))
model.add(Dense(43, activation='relu'))
model.add(Dense(20, activation='relu'))
model.add(Dense(1, activation='sigmoid'))

model.compile(loss = 'binary_crossentropy',
              optimizer = 'adam',
              metrics = ['accuracy'])
model.summary()

It works. But when I try to fit the X_train I get an error:

Error when checking target: expected dense_227 to have shape (1,) but got array with shape (2,)

The mentioned layer is the output layer. As far as I understood, the output shape is incorrect. I tried using Flattern() in between the layers, and even tried a Reshape(1,). But since I'm a beginner at this, I don't fully understand what I have to add to control my data shape in the NN in order to obtain the output I need.

I managed to make it work with softmax, but I need the sigmoid to work too so I can have a final prediction afterward (True or False / 1 or 0).

Thank you.

Can you show us the part of the code where you define `y_train`, `y_test`? — Milo Lu, Aug 29 '18 at 09:46
@IsabellaNavarro Since you are doing binary classification, the label for each sample should be either zero or one and not `[0, 1]` or `[1, 0]`. In other words, the `y_train` should have a shape of `(86,)` and not `(86,2)`. — today, Aug 29 '18 at 11:19
Oh I see the point there. But how can I do the encoding for my labels then? Any tip? — Bella, Aug 29 '18 at 11:21
@IsabellaNavarro If `y_train` consists of `True` and `False` values then you don't need to change it. Just pass it to the `fit` method. Plus, use "@[user_name]" at the beginning of your comment when you are replying to a specific user. Otherwise he/she will not be notified of your comment. — today, Aug 29 '18 at 11:23

ralf htp · Answer 1 · 2018-08-29T15:08:11.560

The problem is how you handle your ground truth data y . The following model is working with random data with your real data you have to encode y as one-hot with 4 values and assign this one-hot encoding the ground truth vector y which has to have the shape (124,4,1) note the 4 ( i use 124 samples here )

So y has to have 4 dimensions y1, y2, y3, y4. In the following is shown how your ground truth logical values 00, 01, 10,11 that are pairs have to be encoded and assigned to the 4 dimensional target vector y:

0 0 ->      0001  -> y1=0, y2=0, y3=0, y4=1
0 1 ->      0010  -> y1=0, y2=0, y3=1, y4=0
1 0 ->      0100  -> y1=0, y2=1, y3=0, y4=0
1 1 ->      1000  -> y1=1, y2=0, y3=0, y4=0

this handling of logical values can be illustrated with a Karnaugh-Veitch map ( https://en.wikipedia.org/wiki/Karnaugh_map ), each square in the map is equivalent to one one-hot encoded logical value.

you have to write a function that translates the combination of your logical values 00,01,10,11 into a one-hot encoded feature vector of dimension 4 like above

with y encoded like this the model can be built like

from random import randint
from random import seed
import math
import numpy as np
from keras.models import Sequential
from keras.layers import Dense


# generating random data, replace this with your data

X = np.random.rand(124,4,591)

y = np.random.randint(2,size=(124,4)) # replace this with your ground truth feature vector

y=y.reshape(124,4,1)



# https://stackoverflow.com/questions/44704435/error-when-checking-model-input-expected-lstm-1-input-to-have-3-dimensions-but


model = Sequential()
model.add(Dense(86, activation='relu', input_shape=(4,591)))
model.add(Dense(43, activation='relu'))
model.add(Dense(20, activation='relu'))
model.add(Dense(1, activation='sigmoid'))
model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])
model.summary()

model.fit(X, y, epochs=1, batch_size=10)

Critical is that y has the same second dimension like X here namely 4, this can be achieved by encoding the logical values of your ground truth like above

this is not done in the code above because i do not know your data, in the code random data of correct dimensionality is used

Keras Neural Network model for binary classification with 3D tensor time series as input

1 Answers1