0

In order to do "Inference" with a trained RNN/LSTM model for sentences one by one. I think I need to set the History input parameter of a RNN/LSTM cell to Zero at the beginning of each sentence.

So how can it be done in Tensorflow?

J Tang
  • 11
  • 3

1 Answers1

0

Based on this answer I believe that the state is set back to whatever state you pass in with the initial_state argument to tf.nn.rnn (or the other RNN creation functions).

Community
  • 1
  • 1
Davis Yoshida
  • 1,757
  • 1
  • 10
  • 24
  • First, thanks. However, I think I don't fully understand the `initial_state`. I'm not sure this set whether or not both effect the parameter of History and the parameter of current Input. – J Tang Oct 02 '16 at 11:24
  • If you look at the code here, https://github.com/tensorflow/tensorflow/blob/master/tensorflow/python/ops/rnn_cell.py, it doesn't appear that the LSTMCells have a history parameter. – Davis Yoshida Oct 02 '16 at 18:48
  • If you mean the `state` argument to when they are called, that is handled by the `rnn` function (or again one of the related functions). – Davis Yoshida Oct 02 '16 at 18:49
  • So...This means that each time I want to Inference a sentence, the only thing I should do is to set the `initial_state`, which means I don't need to reload the model parameter from the checkpoint(only reload for the first time), is it right? – J Tang Oct 03 '16 at 23:49
  • I'm fairly sure that you don't even need to do that. If you set the `initial_state` argument to be constant with value zero, you should just be able to have the session run your example. – Davis Yoshida Oct 04 '16 at 02:12
  • Just sentence by sentence is OK? Will context of previous sentence affect current one? – J Tang Oct 05 '16 at 09:56
  • Thanks, you're right! I did a tiny experiment. However, I don't why it seems not logical to me... – J Tang Oct 05 '16 at 10:45
  • They probably made that the default since it's most convenient and training time. You can save the state by replacing the initial state with a variable. – Davis Yoshida Oct 05 '16 at 20:24