I have an interesting, non-novel but seemingly generally unsolved, at least in sources I could find online, problem. The issue is basically this:
I want to persist the states of my RNN in between calls/invokations to it via the TF Serving API.
I have found quite a few people online talking about this problem, such as in the following links:
https://github.com/tensorflow/serving/issues/724
https://groups.google.com/a/tensorflow.org/forum/#!topic/discuss/00tipdqxRZk
Tensorflow Serving - Stateful LSTM
Unfortunetly, there seems to be some discussion of the problem and discussion of a built in fix being added to TF but not being there yet, but no actual guides/examples for how to work around this problem.
The closest I have been able to find a guide for how to do this was the following in the groups.google link:
"As background for future readers, there are two broad cases to consider. The first is where you want to run an entire sequence computation, with multiple steps, each with state to preserve for the next step. This case is addressed with dynamic unrolling."
I am not really sure how to go about implementing this at all though. I dont really want to share my code here just because it is really quite long for a SO post and don't expect anyone to read through it all and tell me what to do.
Just any general tips would be awesome.
I have two files right now that I am using to deploy and use my RNN in the serving api. They are called rnn_save_model.py and rnn_client.py. rnn_save_model just creates the actual model in code and then uses SavedModelBuilder to save it to file. rnn_client then passes in some parameters to the model when I have it loaded via the following command:
tensorflow_model_server --port=9000 --model_name=rnn --model_base_path=/tmp/rnn_model/ I am just not even sure where to add code to get the model to load a state stored to file or something as the model itself is "created" once in rnn_save_model and I do not see how to clearly pass in states via the rnn_client file that are then added to the graph as the file in its current state is just using a model loaded in memory and interfacing with it VS actually editing the model like would be needed to load previous state from file or something.
I really appreciate any help!
