Tensorflow model import to Java

Question

I have been trying to import and make use of my trained model (Tensorflow, Python) in Java.

I was able to save the model in Python, but encountered problems when I try to make predictions using the same model in Java.

Here, you can see the python code for initializing, training, saving the model.

Here, you can see the Java code for importing and making predictions for input values.

The error message I get is:

Exception in thread "main" java.lang.IllegalStateException: Attempting to use uninitialized value Variable_7
     [[Node: Variable_7/read = Identity[T=DT_FLOAT, _class=["loc:@Variable_7"], _device="/job:localhost/replica:0/task:0/cpu:0"](Variable_7)]]
    at org.tensorflow.Session.run(Native Method)
    at org.tensorflow.Session.access$100(Session.java:48)
    at org.tensorflow.Session$Runner.runHelper(Session.java:285)
    at org.tensorflow.Session$Runner.run(Session.java:235)
    at org.tensorflow.examples.Identity_import.main(Identity_import.java:35)

I believe, the problem is somewhere in the python code, but I was not able to find it.

I used [this](https://github.com/tensorflow/tensorflow/blob/master/tensorflow/java/src/main/java/org/tensorflow/examples/LabelImage.java) as inspiration — szi, Apr 20 '17 at 13:50

score 12 · Accepted Answer · edited May 23 '17 at 12:17

The Java importGraphDef() function is only importing the computational graph (written by tf.train.write_graph in your Python code), it isn't loading the values of trained variables (stored in the checkpoint), which is why you get an error complaining about uninitialized variables.

The TensorFlow SavedModel format on the other hand includes all information about a model (graph, checkpoint state, other metadata) and to use in Java you'd want to use SavedModelBundle.load to create session initialized with the trained variable values.

To export a model in this format from Python, you might want to take a look at a related question Deploy retrained inception SavedModel to google cloud ml engine

In your case, this should amount to something like the following in Python:

def save_model(session, input_tensor, output_tensor):
  signature = tf.saved_model.signature_def_utils.build_signature_def(
    inputs = {'input': tf.saved_model.utils.build_tensor_info(input_tensor)},
    outputs = {'output': tf.saved_model.utils.build_tensor_info(output_tensor)},
  )
  b = saved_model_builder.SavedModelBuilder('/tmp/model')
  b.add_meta_graph_and_variables(session,
                                 [tf.saved_model.tag_constants.SERVING],
                                 signature_def_map={tf.saved_model.signature_constants.DEFAULT_SERVING_SIGNATURE_DEF_KEY: signature})
  b.save()

And invoke that via save_model(session, x, yhat)

And then in Java load the model using:

try (SavedModelBundle b = SavedModelBundle.load("/tmp/mymodel", "serve")) {
  // b.session().run(...)
}

Hope that helps.

Warning: While this will work in Java, TF currently does not support loading a SavedModel in Android. Found this out the hard way. :/ — Keilaron, Feb 13 '18 at 21:35
Instead, see issues #12750 or #13079, or see https://www.tensorflow.org/mobile/prepare_models — Keilaron, Feb 13 '18 at 21:41
does this work with loading models from gcs bucket ?? i am not able to use savedmodelbundle to load models and variables froma gcs bucket — user179156, Apr 19 '18 at 08:16

score 2 · Answer 2 · answered Aug 03 '17 at 18:21

2

Fwiw, Deeplearning4j lets you import models trained on TensorFlow with Keras 1.0 (Keras 2.0 support is on the way).

https://deeplearning4j.org/model-import-keras

We also built a library called Jumpy, which is a wrapper around Numpy arrays and Pyjnius that uses pointers instead of copying data, which makes it more efficient than Py4j when dealing with tensors.

https://deeplearning4j.org/jumpy

answered Aug 03 '17 at 18:21

racknuf

444
3
12

Thanks! I was about building a GAN in Tensorflow, and as far as I know DL4J doesn't support GANs in general, that's why I was looking for a workaround being able to use a GAN trained with Tensorflow and ran on JVM. – szi Aug 07 '17 at 08:03
when will keras support be released? i see the PR is in. It'd be super convenient to get the conv1d layer support – Tucker Mar 15 '18 at 02:19
2

RIP Jumpy. Link is broken. – Gili May 16 '20 at 15:43

score 1 · Answer 3 · answered Apr 20 '17 at 14:27

Your python-model will certainly fail at this:

sess.run(init) #<---this will fail
save_model(sess)
error = tf.reduce_mean(tf.square(prediction - y))

#accuracy = tf.reduce_mean(tf.cast(error, 'float'))
print('Error:', error)

init is not defined in the model - I'm unsure what you want achieve at this place, but that should give you a starting point

Tensorflow model import to Java

3 Answers3

Linked