0

I am trying to do a timeseries forecasting and the training is going smoothly but passing the same dataset to predict function I'm getting the following error.

InvalidArgumentError: Specified a list with shape [1,1] from a tensor with shape [32,1] [[node sequential/lstm/TensorArrayUnstack/TensorListFromTensor ]] [Op:__inference_predict_function_55827] Function call stack: predict_function

I'm using a Statefull Lstm and same code and model works fine in tensorflow v1.14 but not in tensorflow v2.4.

my X_train.shape,y_train.shape is ((6191, 10, 1), (6191, 1)), X_test.shape=(6191, 10, 1) and batch_size=1

model = Sequential()
model.add(LSTM(10,batch_input_shape=(batch_size, int(i_shape[0]), int(i_shape[1])), 
               activation=activation,stateful=True,
               kernel_regularizer=L1L2(0.01,0.001)))


Model: "sequential_6"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
lstm_11 (LSTM)               (1, 10)                   480       
_________________________________________________________________
dense_4 (Dense)              (1, 1)                    11        
=================================================================
Total params: 491
Trainable params: 491
Non-trainable params: 0
_________________________________________________________________
None

Let me know if any additional information is required.

manvall
  • 19
  • 1
  • 3

2 Answers2

0

I encountered the same error. In my case, I used a Bidirectional wrapper around LSTM.

I resolved the problem by predicting one time step at a time.

  1. Create a function that splits the data into X and Y. (I think you already have one)
import numpy as np

def split_sequence(sequence, n_steps):
    X, y = list(), list()
    for i in range(len(sequence)):
        # find the end of this pattern
        end_ix = i + n_steps
        # check if we are beyond the sequence
        if end_ix > len(sequence)-1:
            break
        # gather input and output parts of the pattern
        seq_x, seq_y = sequence[i:end_ix], sequence[end_ix]
        X.append(seq_x)
        y.append(seq_y)
    return np.array(X), np.array(y)
  1. Create a function that will loop through X and predict.
def get_predictions(model, np_input, look_back):
    X = list()
    for i in range(len(np_input)):
        #print(np_input[i])
        
        n_features = 1
        testX = np_input[i].reshape((1, look_back, n_features))
        
        #this returns one prediction that has 2 dimensions so we need to flatten
        testPredict = model.predict(testX)
        X.append(testPredict.flatten())
        
    return np.array(X)

Test split_sequence:

import numpy as np
from keras.models import load_model

raw_seq = np.array([0, 0, 0, 0, 0, 0,
                 0, 0, 0, 0, 0, 0,
                 0, 0, 0, 0, 0, 0,
                 0, 0, 0, 0, 0, 0, 1])

n_steps = 24
X, y = split_sequence(raw_seq, n_steps)

for i in range(len(X)):
    print(X[i], y[i])

n_features = 1
testX = X.reshape((X.shape[0], X.shape[1], n_features))

model = load_model("<your_model_file>")

testPredict = model.predict(testX)
print("===Prediction===")
print(testPredict)

As in this example, I have 24 timesteps (the model, of course, was created with 24 time steps), and predict the 25th element. So, the sample input (raw_seq) has a total of 25 elements.

You'll notice that if you add one element to raw_seq, the error will come back. This means that model.predict can only do one prediction at a time.

Test split_sequence and get_predictions:

raw_seq = np.array([0, 0, 0, 0, 0, 0,
                 0, 0, 0, 0, 0, 0,
                 0, 0, 0, 0, 0, 0,
                 0, 0, 0, 0, 0, 1, 1,
                   
                 0, 1, 0, 0, 0, 0,
                 0, 0, 0, 0, 0, 0,
                 0, 0, 0, 0, 0, 0,
                 0, 0, 0, 0, 0, 1, 2
                   
                   ])

look_back = 24
X, y = split_sequence(raw_seq, look_back)

preds = get_predictions(model, X, look_back)
print(preds)

Running the code will give 26 predictions preds.shape = (26, 1)

As expected, predictions on a whole dataset will take a long time time.

apvilla
  • 1
  • 1
  • thanks for the answer. Does this work for a stateful lstm where the batch size is defined in the model? Because you the batchsize in model.predict() is 1 but the batch_input_shape defined during training the model maybe different. – manvall May 28 '21 at 13:05
  • I'm doing something similar to what you have mentioned, my use case is comparing the model predicitons when trained on whole dataset vs online training. So the problem only occurs during using the whole dataset. – manvall May 28 '21 at 13:08
  • @manvall, from [this](https://stackoverflow.com/questions/66298721/tensorflow-python-framework-errors-impl-invalidargumenterror-specified-a-list-w) answer, seems that batch size when creating a model SHOULD match batch size when predicting. So, ``` model.predict(data,batch_size=) ``` – apvilla Mar 09 '22 at 03:53
  • @manvall I was able to resolve the issue by adding the batch size argument to predict. Thus, model.predict(testX, batch_size=1) should work. I wouldn't recommend the previous solution I had because it takes up too much memory and processor power. – apvilla Mar 09 '22 at 04:03
0

the reason for this error is that you are choosing batch_size not split over the number of TrainSets

the batch size should be chosen in a way so that the number of samples is divisible by the batch size. See also here: this problem solve here

Chafik Boulealam
  • 516
  • 5
  • 10