4

I am trying to implement a many to one model where inputs are word vectors of size d . I need to output a vector of size d at the end of LSTM .

enter image description here

In this question , it is mentioned to use(for a many to one model)

model = Sequential()
model.add(LSTM(1, input_shape=(timesteps, data_dim)))

I am still doubtful about how to specify input and more importantly output dimensions in the keras model

MysticForce
  • 1,253
  • 1
  • 16
  • 27

2 Answers2

1

The code you provided is actually for one-to-many scenario - but for the output of size 1. If you want to have an output of size d you need to try the following code:

model = Sequential()
model.add(LSTM(d, input_shape=(len_seq, d)))

Moreover - with word vectors I strongly advise you to use Embedding layer which is designed for your use case. Use it as an input to your LSTM then.

MysticForce
  • 1,253
  • 1
  • 16
  • 27
Marcin Możejko
  • 39,542
  • 10
  • 109
  • 120
  • I think the code I gave is valid for many-to-one case where at each timestep input is an integer and LSTM outputs one and only one output which is at the end . so shouldn't it be many-to-one code? – MysticForce Apr 08 '17 at 04:10
  • @MysticForce did you come up with a solution? struggling with the same problem. – Marc Juchli Jun 21 '17 at 13:54
0

Input is a 3D tensor of shape (batch_size, timesteps, input_dims)

So it will look something like this:

batch_input = np.empty([batch_size] + [timesteps] + [d])

There will be one output (the last sequence) because 'return_state=False' in keras by default.

The output shape will be:

(batch_size, 1)

Because you only have one unit in your LSTM. If you want an output with 'd' dimensions then you have to change LSTM to 

LSTM(d, input_shape=(timesteps, data_dim))
bigdata2
  • 999
  • 1
  • 11
  • 17