tensorflow restore only some variables from checkpoint

Question

After inspecting a checkpoint (let's call it model 1) I obtained the following list of variable names in it (shortened for simplicity):

var_list = ["ex1_model/fc2/b",
"ex1_model/fc2/b/Adam",
"ex1_model/fc2/b/Adam_1",
"ex1_model/fc2/w",
"ex1_model/fc2/w/Adam"]

Suppose I have a much bigger model 2 and want to initialize parts of it with the values from model 1.

Obtain variables from names as described here (because I didn't find an easy way to do it):

def get_vars_by_name(names):
    return [v for v in tf.global_variables() if v.name in names]

Build model 2 and a saver to restore:

logits = build_model(inputs)
saver = tf.train.Saver(var_list=get_vars_by_name(var_list))

At

saver.restore(sess, tf.train.latest_checkpoint(checkpoints_dir))

I receive the error:

"ex1_model/fc2/w/Adam" [...] raise ValueError("No variables to save")

Please help me find the mistake I made. I'd also be grateful for a simpler way because this is terrible. Thank you.

score 0 · Answer 1 · answered May 02 '18 at 08:31

An simple way to solve is this issue would be to guess if the variable should be restored or not.

def ignore_name(name):
    if name.endswith('/Adam') or name.endswith('/Adam_1'):
        return True
    return False

You should be able to use this idea directly via

def get_vars_by_name(names):
    return [v for v in tf.global_variables() if v.name in names and not ignore_name(v.name)]

This even allows to train the model using ADAM and later switch to SDG or vice versa.

Thank you for your time. The Adam is not the problem. If I remove it from the list I get the same error on "ex1_model/fc2/w" and so on. — Gerry, May 02 '18 at 08:36

1 Answers1