14

Now we have used TensorFlow to train and export an model. We can implement the inference service with this model just like how tensorflow/serving does.

I have a question about whether the tf.Session object is thread-safe or not. If it's true, we may initialize the object after starting and use the singleton object to process the concurrent requests.

tobe
  • 1,671
  • 2
  • 23
  • 38

1 Answers1

18

The tf.Session object is thread-safe for Session.run() calls from multiple threads.

Before TensorFlow 0.10 graph modification was not thread-safe. This was fixed in the 0.10 release, so you can add nodes to the graph concurrently with Session.run() calls, although this is not advised for performance reasons; instead, it is recommended to call sess.graph.finalize() before using the session from multiple threads, to prevent accidental memory leaks.

Graham
  • 7,431
  • 18
  • 59
  • 84
mrry
  • 125,488
  • 26
  • 399
  • 400
  • 1
    Great. Thanks a lot. Is it necessary to add this in document? Any one uses TensorFlow may implement the inference service for their models and it is important to do it on the right way. – tobe Aug 03 '16 at 04:02
  • Can a minimal example using Keras be provided. – Adrian Hood Sr Apr 12 '19 at 22:27