Best approach for multiple time series prediction in Tensor Flow Js

Question

EDIT: This answer helps very much, however, I would really like some actual tensor flow javascript code to show how to implement this. How to train a RNN with LSTM cells for time series prediction

Other questions I found do not cover normalization or my specific goal of prediction beyond 1 point into the future, where locality is kept in mind i.e. if 1 is predicted at T+1, then T+2, is more likely to be 0.

Total data sample (actually have 132 samples)

const Data = [
    [0,45,0,0],
    [1,40,0,0],
    [0,10,3,0],
    [0,0,0,0],
    [2,30,0,1], 
    [2,20,3,1]
 ];

Array representations [X1,X2,X3,Y]

*Note X2 requires normalization --- not sure how to handle this if we need to predict X2 into future values in order to predict Y while doing all of this in a single network

Ultimate goal

Using this data (with many more samples) predict the next 5 Y values in the most accurate way possible while weighing previous patterns as in a time series prediction where a sample in the past could be important 25 samples from then.

Current progress

Have predicted 25 values forward by training on 25 [X1,X2,X3] arrays only to discover that input position 1 ([X1,X2,X3] array 1) could affect all the next 25 Y values but position 2 ([X1,X2,X3] array 2) could only affect the 2-25 of the set and so on and so on --- position 25 can only affect the last value of the 25 predicted values --- so I was likely not even really predicting the next 25 as far as the network was concerned.

Current approach

Desired prediction would be the next 3 Y values (for example) after training on

Input:

[
    [0,45,0],
    [1,40,0],
    [0,10,3]
]

Output

[
    [0],
    [1],
    [1]
]

Attempted model so far

var model = tf.sequential();

model.add(tf.layers.batchNormalization({
    inputShape:[null,6],
    axis: 2,
    units:10,
    //returnSequences: true,
    //kernelInitializer: 'VarianceScaling'
}));

model.add(tf.layers.lstm({
    //inputShape:[null,7],
    units: 10,
    activation: 'relu',
    returnSequences: true,
    //kernelInitializer: 'VarianceScaling'
}));

model.add(tf.layers.lstm({
    //inputShape:[null,7],
    units: 6, 
    activation: 'relu',
    returnSequences: true,
    //kernelInitializer: 'VarianceScaling'
}));

//modelHelper(model);
const optimizer = tf.train.adam (.05);
//optimizer: 'sgd', loss: 'binaryCrossentropy', lr: 0.1
model.compile({
    loss:tf.losses.meanSquaredError,
    optimizer:optimizer
});

When using multiple values per time location (X1,X2,X3), what is the best approach to predict the next 4 or 5 or 25 single (Y) values of a time series rather than just the next value?

*After browsing for 5 years created an account, so lost on this one.

score 1 · Accepted Answer · edited Feb 28 '19 at 19:13

There are so many questions here.

What is the best approach to predict the next 4 or 5 or 25 single (Y) values of a time series rather than just the next value

You simply need to return the sequence of the lstm. If you want to predict the next 4 values, then the units of the last lstm layer should be 4 with return returnSequences set to be true.

If you want to predict either 1 or 0 depending on your series, then you can use a binaryCrossEntropy loss with a softmax activation for the last layer to compute a probability. As for what is most likely, the network will figure it out if the data is much consistent with your observation, ie if 1 is predicted in T then 0 is likely to be predicted next at T+i.

X2 requires normalization --- not sure how to handle this if we need to predict X2 into future values in order to predict Y while doing all of this in a single network

This is not specific to your use case, it is a best practice to keep all data within the same range. For data with high variance will tend to influence highly the model influencing the convergence. You can normalize the x2 feature before feeding your model. Here is a function that will normalize the data across all features

// standardize a tensor data by computing (data - mean(data) ) / std(data)

function standardize(data) {
  let means = []
  let variances = []
  for ( let axe = 0 ; axe < axes; axe++) {
    const {mean, variance} = tf.moments(data.gather([axe], 1), undefined, true )
    means.push(mean)
    variances.push(variances)
  }

  return data.sub(tf.concat(means).reshape([axes])).div(tf.concat(variances).reshape([axes]))
}

Best approach for multiple time series prediction in Tensor Flow Js

1 Answers1