Update: "AttributeError: 'AdamOptimizer' object has no attribute 'name'" might be an alternative title for this question. If this can be solved then the whole thing might work right.
Have a new jupyter notebook that has NO model saving code. The trained model was already saved using another notebook like this. The iris dataset was used here and the model is all trained up:
#This might be the way:
#https://www.tensorflow.org/guide/eager#object_based_saving
#https://stackoverflow.com/questions/47852516/tensorflow-eager-mode-how-to-restore-a-model-from-a-checkpoint
savePrefix = "/tmp/iris"
root = tfe.Checkpoint(optimizer=optimizer, model=model, optimizer_step=global_step)
restorePrefix = root.save(savePrefix)
# '/tmp/iris-1'
print(restorePrefix)
print(root)
print("The model was saved to {}\nRestore the model using {}".format(savePrefix, restorePrefix))
# Try loading this in a different notebook to prove it worked.
>/tmp/iris-1
><tensorflow.contrib.eager.python.checkpointable_utils.Checkpoint object at 0x7fb308799e80>
>The model was saved to /tmp/iris
>Restore the model using /tmp/iris-1
I grabbed the output path from the above and then tried to load the model in a new notebook using tf.contrib.eager code but it fails:
s = tfe.Saver([optimizer, model, global_step])
s.restore(file_prefix="/tmp/iris-1")
>NameError: name 'optimizer' is not defined
So what is an actually WORKING use case code to load a previously developed model with the tf.contrib.eager api (not the session api) WHEN THE CODE IN THE MODEL-LOADING NOTEBOOK DOES NOT SAVE THE MODEL AND DOES NOT HAVE THE MODEL'S PARTS IN MEMORY ALREADY like optimizer, graph definition, and global_state?
The TF docs always choose to demo a pointless example of loading a model we already have in memory. I can't tell if "With the TF.contrib.eager API you have to have explicit model creation code and optimizer creation code and global step code right in your notebook because TFE cannot load this stuff from disk" or "its a new API and some features are missing", or "you have to also use session and graph coding api along with tf.contrib.eager api" or "just use Microsoft cntk which actually works with imperative code and they didnt forget critical parts of the API".
Thanks if you know something. If speculating, please state.
I suspect it's some superset and subset combination of the following if it's going to work. (Scraped from SO posts on the subject.)
tfe.restore_variables_on_create
tf.train.import_meta_graph()
new_saver.restore()
tf.train.latest_checkpoint()
tfe.save_network_checkpoint()
tfe.restore_network_checkpoint()
Update - I added code to first manually recreate the model, optimizer, etc into memory, and then ran the restore() code. Error - optimizer does not have a name. But, oddly, it actually does have a name attribute according to the documentation:
#OK I'll just try to use code to create the model artifacts first.
# I'd rather load it all from disk but maybe at least it might work.
optimizer = tf.train.AdamOptimizer() #ga
global_step = tf.train.get_or_create_global_step() # what is a global_step?
model = tf.keras.Sequential([
tf.keras.layers.Dense(10, activation=tf.nn.relu, input_shape=(4,)), # input shape required
tf.keras.layers.Dense(10, activation=tf.nn.relu, kernel_initializer='glorot_uniform'),
tf.keras.layers.Dense(3)
])
s = tfe.Saver([optimizer, model, global_step])
s.restore(file_prefix="/tmp/iris-1")
INFO:tensorflow:Restoring parameters from /tmp/iris-1
AttributeError: 'AdamOptimizer' object has no attribute 'name'
The documentation says:
__init__(
learning_rate=0.001,
beta1=0.9,
beta2=0.999,
epsilon=1e-08,
use_locking=False,
name='Adam'
)
It sure looks like Adam optimizer has a name! That's funny. What's the problem then?
Maybe the TF error is really saying that an optimizer and a model and a global_state variable cannot be restored. So then what would be restored from a checkpoint -- specifically what variables would go in the save and corresponding restore? Thanks if you know anything.
