Keras - LeakyReLU has no attribute name error when saving model

Question

I'm using a LeakyReLU activation in my model. I'm able to train it. But when I train to save the model,

discriminator_model.save(os.path.join(output_folder_path, 'discriminator_model_{0}.h5'.format(iteration_no)))

I get the below error

AttributeError: 'LeakyReLU' object has no attribute '__name__'

I'm using keras-gpu 2.2.4 with tensorflow-gpu 1.12.0 backend. This is my model:

discriminator_model = Sequential()
discriminator_model.add(Conv2D(64, 5, strides=2, input_shape=(28, 28, 1), padding='same', activation=LeakyReLU(alpha=0.2)))
discriminator_model.add(Dropout(0.4))
discriminator_model.add(Conv2D(128, 5, strides=2, padding='same'))
discriminator_model.add(LeakyReLU(alpha=0.2))
discriminator_model.add(Dropout(0.4))
discriminator_model.add(Conv2D(256, 5, strides=2, padding='same'))
discriminator_model.add(LeakyReLU(alpha=0.2))
discriminator_model.add(Dropout(0.4))
discriminator_model.add(Conv2D(512, 5, strides=2, padding='same'))
discriminator_model.add(LeakyReLU(alpha=0.2))
discriminator_model.add(Dropout(0.4))
discriminator_model.add(Flatten())
discriminator_model.add(Dense(1))
discriminator_model.add(Activation('sigmoid'))
discriminator_model.summary()

Initially, I was using

discriminator_model.add(Conv2D(128, 5, strides=2, padding='same', activation=LeakyReLU(alpha=0.2)))

But it was suggested here and here to add LeakyReLU as a separate activation layer. No luck even after trying that.

Full stack trace:

Traceback (most recent call last):
  File "/opt/PyCharm/pycharm-community-2018.3.3/helpers/pydev/pydevd.py", line 1741, in <module>
    main()
  File "/opt/PyCharm/pycharm-community-2018.3.3/helpers/pydev/pydevd.py", line 1735, in main
    globals = debugger.run(setup['file'], None, None, is_module)
  File "/opt/PyCharm/pycharm-community-2018.3.3/helpers/pydev/pydevd.py", line 1135, in run
    pydev_imports.execfile(file, globals, locals)  # execute the script
  File "/opt/PyCharm/pycharm-community-2018.3.3/helpers/pydev/_pydev_imps/_pydev_execfile.py", line 18, in execfile
    exec(compile(contents+"\n", file, 'exec'), glob, loc)
  File "..../Workspace/src/v01/MnistTrainer.py", line 100, in <module>
    main()
  File "..../Workspace/src/v01/MnistTrainer.py", line 92, in main
    mnist_trainer.train(train_steps=100, log_interval=1, save_interval=1)
  File "..../Workspace/src/v01/MnistTrainer.py", line 56, in train
    self.save_models(output_folder_path, i + 1)
  File "..../Workspace/src/v01/MnistTrainer.py", line 69, in save_models
    os.path.join(output_folder_path, 'discriminator_model_{0}.h5'.format(iteration_no)))
  File "..../.conda/envs/e0_270_ml_3.6/lib/python3.6/site-packages/keras/engine/network.py", line 1090, in save
    save_model(self, filepath, overwrite, include_optimizer)
  File "..../.conda/envs/e0_270_ml_3.6/lib/python3.6/site-packages/keras/engine/saving.py", line 382, in save_model
    _serialize_model(model, f, include_optimizer)
  File "..../.conda/envs/e0_270_ml_3.6/lib/python3.6/site-packages/keras/engine/saving.py", line 83, in _serialize_model
    model_config['config'] = model.get_config()
  File "..../.conda/envs/e0_270_ml_3.6/lib/python3.6/site-packages/keras/engine/sequential.py", line 278, in get_config
    'config': layer.get_config()
  File "..../.conda/envs/e0_270_ml_3.6/lib/python3.6/site-packages/keras/layers/convolutional.py", line 493, in get_config
    config = super(Conv2D, self).get_config()
  File "..../.conda/envs/e0_270_ml_3.6/lib/python3.6/site-packages/keras/layers/convolutional.py", line 226, in get_config
    'activation': activations.serialize(self.activation),
  File "..../.conda/envs/e0_270_ml_3.6/lib/python3.6/site-packages/keras/activations.py", line 176, in serialize
    return activation.__name__
AttributeError: 'LeakyReLU' object has no attribute '__name__'

Can you add the imports? Are you mixing keras and tf.keras? A self-containing example that reproduces the issue would be the best — Dr. Snoopy, Mar 16 '19 at 09:49

alift · Accepted Answer · 2019-03-16T20:45:47.357

4

Edited part( Thanks @NagabhushanSN for mentioning the remaining issue)

There is a line of the code where we still have discriminator_model.add(Conv2D(64, 5, strides=2, input_shape=(28, 28, 1), padding='same', activation=LeakyReLU(alpha=0.2))) , it is the second line of the code.

If we modify that line, the final corrected code should be like this:

discriminator_model = Sequential()
discriminator_model.add(Conv2D(64, 5, strides=2, input_shape=(28, 28, 1), padding='same'))
discriminator_model.add(LeakyReLU(alpha=0.2))
discriminator_model.add(Dropout(0.4))
discriminator_model.add(Conv2D(128, 5, strides=2, padding='same'))
discriminator_model.add(LeakyReLU(alpha=0.2))
discriminator_model.add(Dropout(0.4))
discriminator_model.add(Conv2D(256, 5, strides=2, padding='same'))
discriminator_model.add(LeakyReLU(alpha=0.2))
discriminator_model.add(Dropout(0.4))
discriminator_model.add(Conv2D(512, 5, strides=2, padding='same'))
discriminator_model.add(LeakyReLU(alpha=0.2))
discriminator_model.add(Dropout(0.4))
discriminator_model.add(Flatten())
discriminator_model.add(Dense(1))
discriminator_model.add(Activation('sigmoid'))
discriminator_model.summary()

And this one should work well on the most recent version of tensroflow, I tested on 1.8.0 and it works fine. However, if check your code with older version like tesnorflow1.1.0, we get the same error.

For that case, I suggest update tensorflow to a higher version

To check the current tensorflow version python is using, do as here.
To update tensorflow, this post seems good enough to show how to do it.

edited Mar 16 '19 at 20:45

answered Mar 16 '19 at 07:13

alift

1,855
2
13
28

I'm using tensorflow 1.12.0 `$ python -c 'import tensorflow as tf; print(tf.__version__)' 1.12.0` – Nagabhushan S N Mar 16 '19 at 07:16
Okay. Thank you. I'll check – Nagabhushan S N Mar 16 '19 at 07:23
TF 1.12 is newer than 1.8, there is no upgrade there. – Dr. Snoopy Mar 16 '19 at 09:48
@MatiasValdenegro Thanks for the notice! You are right, I misread it as 1.2! – alift Mar 16 '19 at 10:12
@NagabhushanSN I deleted my previous comments, and i put this new one : Mine is working with 1.8.0, If yorus is not working, one suggestion might be to downgrade to 1.8.0 to see if your problem will be resolved. I highly recommend to create a virutal environment, and try your tnesorflow 1.8 there! – alift Mar 16 '19 at 10:14
@alift Thanks for your time. Actually, I had made a small mistake. In the first layer `discriminator_model.add(Conv2D(64, 5, strides=2, input_shape=(28, 28, 1), padding='same', activation=LeakyReLU(alpha=0.2)))`, the activation LeakyReLU is specified here only, not as a separate layer. I moved it to a separate layer. It worked. You can update your answer. I'll accept it. – Nagabhushan S N Mar 16 '19 at 12:01
1

@NagabhushanSN Oh! you re right, I did not pay attention to it. I have just corrected the code on my local machine. Thanks for letting me know, let me edit the answer – alift Mar 16 '19 at 20:39
1

Note that the error is on the keras side, if you change the version of tensorflow and it fixes something, that means you are probably mixing tf.keras and keras, which you shouldn't do. – Dr. Snoopy Mar 16 '19 at 20:49

score -1 · Answer 2 · edited Sep 16 '20 at 13:51

-1

leakyrelu_alpha = 0.2    

gen5 = Conv2D(filters=256, kernel_size=3, strides=1, padding='same')(gen5)
gen5 = LeakyReLU(alpha=leakyrelu_alpha)(gen5)#Activation('relu')'or #LeakyReLU(alpha=0.3)

use this, it will solve your problem

edited Sep 16 '20 at 13:51

AlexisG

2,476
3
11
25

answered Sep 16 '20 at 04:32

anas elmasry

1

1

Hi, kindly format your code properly. And, how does your answer differ from the accepted one? Does your answer add anything extra to the existing solution? You'll also need to explain what change and why that change needs to be done. – Nagabhushan S N Sep 16 '20 at 07:00

Keras - LeakyReLU has no attribute name error when saving model

2 Answers2