5

I'm working off the LSTM language model tutorial discussed here.

With language models, it's common to use the model to generate a new sentence from scratch after training (i.e. sample from the model).

I'm new to TensorFlow but I'm trying to use my trained model to generate new words until the end-of-sentence marker.

My initial attempt:

x = tf.zeros_like(m.input_data)
state = m.initial_state.eval()
for step in xrange(m.num_steps):
    state = session.run(m.final_state,
                               {m.input_data: x,
                                m.initial_state: state})
    x = state

It fails with error:

ValueError: setting an array element with a sequence.

Shazam
  • 689
  • 2
  • 9
  • 17

1 Answers1

8

The issue here seems to be the m.input_data: x mapping in the feed_dict passed session.run(). In this case, TensorFlow expects that x is a numpy array (or some object that can be implicitly converted to a numpy array), but the value is a TensorFlow Tensor (the result of tf.zeros_like()).

Fortunately, the solution is simple. Replace x = tf.zeros_like(m.input_data) with the following:

x = tf.zeros_like(m.input_data).eval()

...which ensures that x is converted to a numpy array.

(Note that a more direct way to achieve this would be to construct the initial x as a numpy array of the appropriate size.)

mrry
  • 125,488
  • 26
  • 399
  • 400
  • Thanks! that makes a lot of sense. – Shazam Dec 16 '15 at 01:16
  • Running into a different error: W tensorflow/core/common_runtime/executor.cc:1076] 0x7fcc3575bc70 Compute status: Invalid argument: You must feed a value for placeholder tensor 'model_1/Placeholder_2' with dtype int32 and shape dim { size: 1 } dim { size: 1 } – Shazam Dec 16 '15 at 01:47
  • It looks like you are missing a feed. I'm not 100% sure, but there could be a dependency on `m.targets` as well, in which case you might need to refactor the code slightly (or feed a dummy value for `m.targets`). – mrry Dec 16 '15 at 01:58
  • Thanks again. m.targets is a placeholder. Trying to run: target = session.run(m.targets, {m.input_data: x, m.targets: targets, m.initial_state: state}) gives me the error: Invalid argument: model_1/Placeholder_3:0 is both fed and fetched. I realize at this point I need to refactor the model to not have target as a placeholder – Shazam Dec 16 '15 at 02:00
  • Nevermind. Targets are the correct labels and final_state would be the predicted label. – Shazam Dec 16 '15 at 02:14
  • @Shazam I'm also trying to use the RNN tutorial to first train the network and then just run it when the training is done. I tried your code (with this answer's fix) but get the same error as your second comment here. How did you fix that error? Right now I think that I need to change the model in order to not be dependent to feed the "target" placeholder, how did you do this? Did you simply change it into a tf.Variable but with same dimension? Thank you for all the help! – Jacob Hagstedt Feb 11 '16 at 11:52