1

I am working with the Audioset/VGGish and attempting to convert the checkpoint file they provide into a .pb file. The problem is that the only two items that is provided for the trained model are the ckpt file (linked above) and an npz file.

This is my third pass at attempting this and have spent a few hours now trying to find the best tool to do this. So far I have found several solutions but they all seem to require more information than just the ckpt file. Keep in mind that the ckpt file and Audioset in general needs to use TensorFlow < 2.

Examples:

• freeze_graph: I always end up with an error of ValueError: You need to supply the name of a node to --output_node_names regardless of what value I put in there. The example uses softmax but the problem is that I can't seem to figure out how to extract node names from the ckpt file so it seems I can't add a valid value without knowing them.

• Logged GitHub Issues: Following the code of the OP, I receive the error ValueError: No variables to save.

• Stack Overflow questions: This one seems to be a solid answer but the GitHub repository doesn't provide a .ckpt.meta file. I assume that that meta information is generally going to be needed in some of these cases? I looked up to see if there is any way to extract meta from a ckpt file to create a meta file and then run the information since it appears that the meta file is the structure or graph of the ckpt file without the values (from this answer: Tensorflow : What is the relationship between .ckpt file and .ckpt.meta and .ckpt.index , and .pb file) but I may be misunderstanding this.

One of the reason I figured there is a way to extract a meta file is because of this issue someone logged on the MMdnn GitHub: Convert Audioset VGG from tensorflow to pytorch. Although not converting to .pb, they have a ckpt.meta file in their command. That file is not linked in their description and a Google search for "vggish_model.ckpt.meta" turns up nothing but that GitHub issue. I have messaged the OP on that issue to see if they could shed some light on where that file came from.

• Previous article (2018) with a conversion script: This is a relatively old article. I can get the script to run but also get the error ValueError: No variables to save.

It would be great if someone could point me in the right direction; I've begun to exhaust my options. It seems that there are some good solutions out there that I'm trying but I may just be missing a step or two (or a file or two) in order to get this to successfully convert.

Thanks for your help!

MillerMedia
  • 3,651
  • 17
  • 71
  • 150

1 Answers1

1

I hope its not too late for this reply, but i managed to generate the .pb files by using the inference code provided in the repository.

Obs: Im using tensorflow 1.4.1 because of my GPU, so this probably wont work on a newer version or should need some changes.

The inference demo loads the graph and the checkpoint data into a session. From there i could use a function to save the session and the graph. Here is a sample of my code:

import vggish_input
from tensorflow.python.tools import freeze_graph
def save(sess, directory, filename, saver):
    """
    This function saves a checkpoint, based on the current session
    """
    if not os.path.exists(directory):
        os.makedirs(directory)
    filepath = os.path.join(directory, filename)
    saver.save(sess, filepath)
    return filepath

def save_as_pb(sess, directory, filename, saver):
    """
    This function saves a checkpoint, then writes the graph in a pbtxt, and then              makes a frozen graph with the chekpoint and the pbtxt
    """

    # Save checkpoint to freeze graph later
    ckpt_filepath = save(sess, directory=directory, filename=filename, saver=saver)
    pbtxt_filename = filename + '.pbtxt'
    pbtxt_filepath = os.path.join(directory, pbtxt_filename)
    pb_filepath = os.path.join(directory, filename + '.pb')

    # This will only save the graph but the variables will not be saved.
    tf.train.write_graph(graph_or_graph_def=sess.graph_def, logdir=directory, name=pbtxt_filename, as_text=True)

    # Freeze graph, combining the checkpoint and 
    freeze_graph.freeze_graph(input_graph=pbtxt_filepath, input_saver='', input_binary=False, input_checkpoint=ckpt_filepath, output_node_names=vggish_params.OUTPUT_TENSOR_NAME.split(':')[0], restore_op_name='save/restore_all', filename_tensor_name='save/Const:0', output_graph=pb_filepath, clear_devices=True, initializer_nodes='')

    return pb_filepath

Then i inserted save_as_pb right after loading the model from the checkpoint in the vggish_inference_demo.py file:

  config = tf.ConfigProto()
  config.gpu_options.allow_growth=True
  with tf.Graph().as_default(), tf.Session(config=config) as sess:
    # Define the model in inference mode, load the checkpoint, and
    # locate input and output tensors.
    vggish_slim.define_vggish_slim(training=False)
    vggish_slim.load_vggish_slim_checkpoint(sess, checkpoint)
    features_tensor = sess.graph.get_tensor_by_name(
        vggish_params.INPUT_TENSOR_NAME)
    embedding_tensor = sess.graph.get_tensor_by_name(
        vggish_params.OUTPUT_TENSOR_NAME)
    saver = tf.train.Saver()
    save_as_pb(sess, './saved_vggish/', 'vggish', saver)
Pedropva
  • 11
  • 2