Error Loading Keras Model Second Time. Feed_dict Error

Question

I used Keras' model.save to save my trained model in h5py format. I then use it in my flask app. At the line 'model = tf.keras.models.load_model('soundmodel.k')' under 'print(loading model', the whole operation runs smoothly the first time it runs, allowing me to play back generated audio to my browser.

@app.route('/api/generate', methods=['POST'])
def generate():
    block_size = 2700
    seq_len = 215

    print(request.form['firstGen'], 'hi\n')
    upl_str = request.form['filePath']
    is_first_gen = request.form['firstGen']
    if is_first_gen == "1":
        print(is_first_gen, '\n')

    x_data, y_data = make_tensors(upl_str, seq_len, block_size)


    #if is_first_gen == "1":

    print('loading model')
    model = tf.keras.models.load_model('soundmodel.k')
    masterpiece = compose(model, x_data)
    masterpiece = convert_sample_blocks_to_np_audio(masterpiece[0])

    masterpiece = write_np_as_wav(masterpiece, sample_rate=44100, filename='new.wav')
    print('wrote np as wav')
    wpath = os.path.join(os.getcwd(), 'new.wav')

    response = {
        'wavPath': wpath
    }

    K.clear_session()

    return jsonify(response)

But the second time onward, when I run it, I get this error:

loading model
[2019-12-26 13:30:30,274] ERROR in app: Exception on /api/generate [POST]
Traceback (most recent call last):
  File "C:\Users\Iman\.conda\envs\pyenv3\lib\site-packages\tensorflow\python\client\session.py", line 1040, in _run
    subfeed, allow_tensor=True, allow_operation=False)
  File "C:\Users\Iman\.conda\envs\pyenv3\lib\site-packages\tensorflow\python\framework\ops.py", line 3339, in as_graph_element
    return self._as_graph_element_locked(obj, allow_tensor, allow_operation)
  File "C:\Users\Iman\.conda\envs\pyenv3\lib\site-packages\tensorflow\python\framework\ops.py", line 3418, in _as_graph_element_locked
    raise ValueError("Tensor %s is not an element of this graph." % obj)
ValueError: Tensor Tensor("Placeholder:0", shape=(2700, 10800), dtype=float32) is not an element of this graph.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\Iman\.conda\envs\pyenv3\lib\site-packages\flask\app.py", line 2446, in wsgi_app
    response = self.full_dispatch_request()
  File "C:\Users\Iman\.conda\envs\pyenv3\lib\site-packages\flask\app.py", line 1951, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "C:\Users\Iman\.conda\envs\pyenv3\lib\site-packages\flask_cors\extension.py", line 161, in wrapped_function
    return cors_after_request(app.make_response(f(*args, **kwargs)))
  File "C:\Users\Iman\.conda\envs\pyenv3\lib\site-packages\flask\app.py", line 1820, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "C:\Users\Iman\.conda\envs\pyenv3\lib\site-packages\flask\_compat.py", line 39, in reraise
    raise value
  File "C:\Users\Iman\.conda\envs\pyenv3\lib\site-packages\flask\app.py", line 1949, in full_dispatch_request
    rv = self.dispatch_request()
  File "C:\Users\Iman\.conda\envs\pyenv3\lib\site-packages\flask\app.py", line 1935, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "C:\Users\Iman\code\music-generator\app.py", line 426, in generate
    model = tf.keras.models.load_model('soundmodel.k')
  File "C:\Users\Iman\.conda\envs\pyenv3\lib\site-packages\tensorflow\python\keras\engine\saving.py", line 232, in load_model
    load_weights_from_hdf5_group(f['model_weights'], model.layers)
  File "C:\Users\Iman\.conda\envs\pyenv3\lib\site-packages\tensorflow\python\keras\engine\saving.py", line 802, in load_weights_from_hdf5_group
    K.batch_set_value(weight_value_tuples)
  File "C:\Users\Iman\.conda\envs\pyenv3\lib\site-packages\tensorflow\python\keras\backend.py", line 2719, in batch_set_value
    get_session().run(assign_ops, feed_dict=feed_dict)
  File "C:\Users\Iman\.conda\envs\pyenv3\lib\site-packages\tensorflow\python\client\session.py", line 877, in run
    run_metadata_ptr)
  File "C:\Users\Iman\.conda\envs\pyenv3\lib\site-packages\tensorflow\python\client\session.py", line 1043, in _run
    'Cannot interpret feed_dict key as Tensor: ' + e.args[0])
TypeError: Cannot interpret feed_dict key as Tensor: Tensor Tensor("Placeholder:0", shape=(2700, 10800), dtype=float32) is not an element of this graph.

I have tried deleting my model after using it, and clearing my session using K.clear_session() to try to load the model as it did on the first run, but this seemed to not make any difference. I also tried adding a flag to stop it from loading the model a second time, but then the model will simply not be defined, resulting in an error. So I think flask clears out those variables after the function returns.

The error occurs when I call the load model function, so it did not try to run the prediction operation before it fails. I am using tensorflow 1.1 and keras 2.0.6. My python version is Python 3.5.4

score 0 · Answer 1 · 2020-04-25T16:38:10.290

There are a couple of workarounds, depending on various context:

Using clear_session() function:
```
from keras import backend as K
```
Then do following at the beginning or at the end of the function, after predicting all the data:
```
K.clear_session()
```
Calling _make_predict_function():

After you load your trained model call:
```
model._make_predict_function()
```
See explanation

Error Loading Keras Model Second Time. Feed_dict Error

1 Answers1