0

My model performs 100 times of inference and one time of training alternately, so I want the inference to be as quick as possible. I see many webpages that discuss how to freeze a saved graph in Tensorflow, but I'd like to do a quick inference without saving the weights to a file... just like PyTorch, to which I'm more accustomed. Saving the weights to a file and freezing the graph in the file takes a while, and I want to avoid that. In PyTorch, using volatile=True made inference twice faster, so I expect the same speedup in Tensorflow by freezing a graph.

So, could anyone tell me what the volatile and requires_grad counterparts in Tensorflow are? If they don't exist, what would be the recommended way to achieve my objective? Does using tf.stop_gradient or tf.estimator.ModeKeys.PREDICT solve my issue?

Math.StackExchange
  • 199
  • 1
  • 9
  • 2
    You can see [my answer to this question](https://stackoverflow.com/questions/45466020/how-to-export-keras-h5-to-tensorflow-pb/45466355#45466355) about how to "freeze" a graph without going through a file (you get a `GraphDef` that you can store in a file if you want, but you can also make a new `Graph` from it. About speed, I don't know the impact this will have. You can read more about performance in the [Performance Guide](https://www.tensorflow.org/performance/performance_guide). – jdehesa Jan 09 '18 at 14:12
  • Thanks! Unfortunately, I've verified that through various methods, TF didn't accelerate inference :( Maybe because it already accelerated like volatile does. – Math.StackExchange Jan 09 '18 at 19:06

0 Answers0