Keras-rl2 error AttributeError: 'Sequential' object has no attribute '_compile_time_distribution_strategy'

Question

I am getting this error AttributeError: 'Sequential' object has no attribute '_compile_time_distribution_strategy' with keras-rl2, when using the below code.

I have searched the whole internet but could not find a solution.

import gym
import tensorflow
print("Import Done")

env = gym.make("CartPole-v0")
states = env.observation_space.shape[0]
print(env.observation_space.shape)
actions = env.action_space.n
print(actions)

print(states)

print(env.observation_space)
print(env.action_space)



def build_model(nstates, nactions):
    model = tensorflow.keras.models.Sequential()
    model.add(tensorflow.keras.layers.Flatten(input_shape=(1, states)))
    model.add(tensorflow.keras.layers.Dense(24, activation='relu'))
    model.add(tensorflow.keras.layers.Dense(24, activation='relu'))
    model.add(tensorflow.keras.layers.Dense(actions, activation='linear'))
    return model


model = build_model(states, actions)
# print(model.summary())

from rl.agents import DQNAgent
from rl.policy import BoltzmannQPolicy
from rl.memory import SequentialMemory


def build_agent(model, actions):
    policy = BoltzmannQPolicy()
    memory = SequentialMemory(limit=50000, window_length=1)
    dqn = DQNAgent(model=model, memory=memory, policy=policy, nb_actions=actions, nb_steps_warmup=10,
                   target_model_update=1e-2)
    return dqn


dqn = build_agent(model, actions)
dqn.compile(tensorflow.keras.optimizers.Adam(learning_rate=1e-3), metrics=['mae'])
dqn.fit(env, nb_steps=50000, visualize=False, verbose=1)

The full error is below

Traceback (most recent call last):
  File "D:\Python Project\reinforceMent_Leran\main.py", line 64, in <module>
    dqn.compile(tensorflow.keras.optimizers.Adam(learning_rate=1e-3), metrics=['mae'])
  File "D:\Python Project\rl_learn\lib\site-packages\rl\agents\dqn.py", line 167, in compile
    self.target_model = clone_model(self.model, self.custom_model_objects)
  File "D:\Python Project\rl_learn\lib\site-packages\rl\util.py", line 16, in clone_model
    clone.set_weights(model.get_weights())
  File "D:\Python Project\rl_learn\lib\site-packages\tensorflow\python\keras\engine\training_v1.py", line 175, in get_weights
    self._compile_time_distribution_strategy)
AttributeError: 'Sequential' object has no attribute '_compile_time_distribution_strategy'

Answer 1

This will occur when you construct your model and then import from rl.* afterwards.

Reverse the order to this, and it will work:

from rl.agents import DQNAgent
from rl.policy import BoltzmannQPolicy
from rl.memory import SequentialMemory

model = build_model(states, actions)
# print(model.summary())

Answer 2

One quick fix would be to delete the model and then create it again.