3

I am trying to load a model with two custom objects and am getting this error in the title.

This is where i import/define my functions, and where i allow keras to reference them by name.

from tensorflow.keras.utils import get_custom_objects
from tensorflow.python.keras.layers import LeakyReLU
from tensorflow.keras.layers import Activation
from tensorflow.keras.backend import sigmoid


def swish(x, beta=1):
    return x * sigmoid(beta * x)


get_custom_objects().update({'swish': Activation(swish)})
get_custom_objects().update({'lrelu': LeakyReLU()})

I load the model with this part

from tensorflow.keras.models import load_model
model = load_model('model.h5', custom_objects={'swish': Activation(swish), 'lrelu': LeakyReLU()}, compile=False)

I get the error below:

Traceback (most recent call last):
  File "C:\Users\Ben\PycharmProjects\untitled\trainer.py", line 102, in load_items
    model = load_model(data_loc + 'model.h5', custom_objects={'swish': Activation(swish), 'lrelu': LeakyReLU()}, compile=False)
  File "C:\Users\Ben\PycharmProjects\untitled\venv\lib\site-packages\tensorflow_core\python\keras\saving\save.py", line 146, in load_model
    return hdf5_format.load_model_from_hdf5(filepath, custom_objects, compile)
  File "C:\Users\Ben\PycharmProjects\untitled\venv\lib\site-packages\tensorflow_core\python\keras\saving\hdf5_format.py", line 168, in load_model_from_hdf5
    custom_objects=custom_objects)
  File "C:\Users\Ben\PycharmProjects\untitled\venv\lib\site-packages\tensorflow_core\python\keras\saving\model_config.py", line 55, in model_from_config
    return deserialize(config, custom_objects=custom_objects)
  File "C:\Users\Ben\PycharmProjects\untitled\venv\lib\site-packages\tensorflow_core\python\keras\layers\serialization.py", line 102, in deserialize
    printable_module_name='layer')
  File "C:\Users\Ben\PycharmProjects\untitled\venv\lib\site-packages\tensorflow_core\python\keras\utils\generic_utils.py", line 191, in deserialize_keras_object
    list(custom_objects.items())))
  File "C:\Users\Ben\PycharmProjects\untitled\venv\lib\site-packages\tensorflow_core\python\keras\engine\sequential.py", line 369, in from_config
    custom_objects=custom_objects)
  File "C:\Users\Ben\PycharmProjects\untitled\venv\lib\site-packages\tensorflow_core\python\keras\layers\serialization.py", line 102, in deserialize
    printable_module_name='layer')
  File "C:\Users\Ben\PycharmProjects\untitled\venv\lib\site-packages\tensorflow_core\python\keras\utils\generic_utils.py", line 193, in deserialize_keras_object
    return cls.from_config(cls_config)
  File "C:\Users\Ben\PycharmProjects\untitled\venv\lib\site-packages\tensorflow_core\python\keras\engine\base_layer.py", line 594, in from_config
    return cls(**config)
  File "C:\Users\Ben\PycharmProjects\untitled\venv\lib\site-packages\tensorflow_core\python\keras\layers\core.py", line 361, in __init__
    self.activation = activations.get(activation)
  File "C:\Users\Ben\PycharmProjects\untitled\venv\lib\site-packages\tensorflow_core\python\keras\activations.py", line 321, in get
    identifier, printable_module_name='activation')
  File "C:\Users\Ben\PycharmProjects\untitled\venv\lib\site-packages\tensorflow_core\python\keras\utils\generic_utils.py", line 180, in deserialize_keras_object
    config, module_objects, custom_objects, printable_module_name)
  File "C:\Users\Ben\PycharmProjects\untitled\venv\lib\site-packages\tensorflow_core\python\keras\utils\generic_utils.py", line 165, in class_and_config_for_serialized_keras_object
    raise ValueError('Unknown ' + printable_module_name + ': ' + class_name)
ValueError: Unknown activation: Activation

Also might be worth noting that i am trying to save and load the model in different projects with different environments. Both are using tf 2.0.0 gpu. The imports should all be the same.

Ben Arnao
  • 492
  • 5
  • 11
  • I saw this question before, did you delete your previous question? – Dr. Snoopy Jan 16 '20 at 14:03
  • Yea, i didn't get any answers so i wanted to repost and maybe add some more information – Ben Arnao Jan 16 '20 at 14:26
  • You should not do that, reposting or asking the same question more than once is not allowed. You can always edit your own questions. – Dr. Snoopy Jan 16 '20 at 14:28
  • Also I don't get why you are passing the activation as Activation(swish), you should just pass the function, not a layer. – Dr. Snoopy Jan 16 '20 at 14:58
  • I've seen that your supposed to wrap in an Activation Layer. https://stackoverflow.com/questions/43915482/how-do-you-create-a-custom-activation-function-with-keras – Ben Arnao Jan 16 '20 at 15:06
  • If you look at the first comment of that answer, you see that its not really necessary – Dr. Snoopy Jan 16 '20 at 15:10
  • I know you can pass directly to layer but because i am doing hyper parameter optimization i need to be able to pass by reference – Ben Arnao Jan 16 '20 at 15:49
  • I am not sure what you are talking about, the problem is literally that you are passing the activations in the wrong way. – Dr. Snoopy Jan 16 '20 at 16:52
  • I want to be able to do the following but for my custom activation "model.add(Activation('tanh'))". I know this is possible because i add the reference by name and my custom activations work fine when i build the model. It's only when i load the model i get issues. I am following the guide here for swish https://www.bignerdranch.com/blog/implementing-swish-activation-function-in-keras/. The link i posted also shows that you should be reference the activation layer in the custom objects, not just a plain function – Ben Arnao Jan 16 '20 at 17:05
  • The tutorial is actually wrong, see my answer for details and the correct way. – Dr. Snoopy Jan 17 '20 at 14:28

2 Answers2

9

You should not blindly believe every tutorial in the internet. As I said in the comments, the problem is passing an activation function as a Layer (Activation to be precise), which works but it is not correct, as you get problems during model saving/loading:

def swish(x, beta = 1):
    return (x * K.sigmoid(beta * x))

get_custom_objects().update({'swish': Activation(swish)})

model = Sequential()
model.add(Dense(10, input_shape=(1,), activation="swish"))

This code above is NOT the correct way, an activation inside a layer should not be another layer. With this code I get errors during model.save with keras and tf.keras in TensorFlow 1.14. The correct way is to:

def swish(x, beta = 1):
    return (x * K.sigmoid(beta * x))

get_custom_objects().update({'swish': swish})

model = Sequential()
model.add(Dense(10, input_shape=(1,), activation="swish"))

Then you will be able to load and save the model correctly. If you need to add an activation as a layer, you should do:

model.add(Activation("swish"))

Which will also allow model save/load just fine.

Dr. Snoopy
  • 55,122
  • 7
  • 121
  • 140
1

I just use the following line of code and it works great!

activation=tf.nn.swish
Devin
  • 11
  • 1