Predict a variable that is not in the input sequences with LSTM-Keras

Question

Assume that I have five columns in my dataset (A,B,C,D,E) and I want to build an LSTM model by training just on A,B,D,E (i.e. I want to exclude C)

My problem is that I still want to use this model to predict C. Is it possible if I didn't train my model with this variable? How can I do that?

EDIT 1
I'm working with categorical and numerical data modeled as time series. In this specific case, C is a categorical time series (given in a one-hot representation).

Answer 1

Yes, you can! However, there needs to be a correlation between field C and the other columns. If not, then the predictions will be close to random.

• Train the model using A,B,D,E as Input (x)
• Make C to be the (y)

Divide the dataset into Train, Test and Validate.

To answer your other question (Is it possible also if I didn't train my model with this variable?)

• No, because how would the model learn to map 4 input fields into an Output Field which in this case it will be (C).

To understand this problem, compare your approach to the Boston Housing dataset.

import pandas as pd
import numpy as np

# Read dataset into X and Y
df = pd.read_csv('YOURDATASET.csv', delim_whitespace=True, header=None)

dataset = df.values


# for example, your dataset is all loaded into a matrix (aka an array with rows of data, and each Index representing those features mentioned A B C D E)


X = dataset[:, 0:1] + dataset[:, 3:4]
Y = dataset[:, 2]


#print "X: ", X
#print "Y: ", Y


# Define the neural network
from keras.models import Sequential
from keras.layers import Dense

def build_nn():
    model = Sequential()
    model.add(Dense(20, input_dim=5, init='normal', activation='relu'))
    # No activation needed in output layer (because regression)
    model.add(Dense(1, init='normal'))

    # Compile Model
    model.compile(loss='mean_squared_error', optimizer='adam')
    return model


# Evaluate model (kFold cross validation)
from keras.wrappers.scikit_learn import KerasRegressor

# sklearn imports:
from sklearn.cross_validation import cross_val_score, KFold
from sklearn.preprocessing import StandardScaler
from sklearn.pipeline import Pipeline

# Before feeding the i/p into neural-network, standardise the dataset because all input variables vary in their scales
estimators = []
estimators.append(('standardise', StandardScaler()))
estimators.append(('multiLayerPerceptron', KerasRegressor(build_fn=build_nn, nb_epoch=100, batch_size=5, verbose=0)))

pipeline = Pipeline(estimators)

kfold = KFold(n=len(X), n_folds=10)
results = cross_val_score(pipeline, X, Y, cv=kfold)

print "Mean: ", results.mean()
print "StdDev: ", results.std()

Answer 2

I'd say that one way for it to do this is for your network to simply predict C or have C as the label
I have been seeing this again and again. Don't confuse a NN with something more than it actually is. You simply approximate the output Y given an input X by learning a function F. That is your NN.
In your case the output could very easily be C + Other_Output Depending on what that other output is, your network could converge and have good results. It could very well not, so your question is simply at this point incomplete.
You have to ask yourself some questions like:

1. Does C + Ohter_Output make sense for the give input.
2. Is there a good way for me serialize the C + Other_Output ? Like having the first K out of N output array elements describing C and the rest N-K describing Other_Output ?
3. Is C a multiclass problem and if so is Other_Output a different kind of problem or could potentially be turned into a multiclass of the same kind, problem that could converge along with C or make them both multilabel problem ?

These are at least some of the questions you need to ask yourself before even choosing the architecture.
That being said, no, unless you train your network to learn about patterns between A B D and C it will not be able to predict a missing input.

Good luck,
Gabriel