6

When I try to train an agent with a batch_size greater than 1 it gives me an exception. Where is my issue?

lr = 1e-3
window_length = 1
emb_size = 10
look_back = 6

# "Expert" (regular dqn) model architecture

inp = Input(shape=(look_back,))
emb = Embedding(input_dim=env.action_space.n+1, output_dim = emb_size)(inp) 
rnn = Bidirectional(LSTM(5))(emb)
out = Dense(env.action_space.n, activation='softmax')(rnn)
expert_model = Model(inputs = inp, outputs = out)
expert_model.compile(loss='categorical_crossentropy', optimizer= Adam(lr))

print(expert_model.summary())
# memory
memory = PrioritizedMemory(limit=1000000,  window_length=window_length)

# policy
policy = BoltzmannQPolicy()

# agent
dqn = DQNAgent(model=expert_model, nb_actions=env.action_space.n, policy=policy, memory=memory, 
               enable_double_dqn=False, enable_dueling_network=False, gamma=.9, batch_size = 100, #Here
               target_model_update=1e-2, processor = RecoProcessor())

Im printing some values directly from the code of keras-rl and it gives me this output:

State[array([0., 0., 0., 0., 0., 0.])]
Batch: [[[0. 0. 0. 0. 0. 0.]]]

But also this exception:

ValueError: Error when checking input: expected input_1 to have 2 dimensions, but got array with shape (1, 1, 6)

I could put the code of the processor class, and I think that there is the key of this, but first I want to make sure that there's nothing wrong here.

Angelo
  • 575
  • 3
  • 18
  • 1
    try to use `numpy.reshape` on the input data before feeding it on the model and see if that works. – Abhinav Anand Apr 01 '19 at 12:00
  • Hey, may I know what libraries you have used for `PrioritizedMemory()`, `BoltzmannQPolicy()` and `DQNAgent`. Thanks – Abhinav Anand Apr 04 '19 at 10:10
  • 1
    @AbhinavAnand Keras RL, but the implementation for the `PrioritizedMemory()` has been done in [this Post](https://towardsdatascience.com/lunar-landings-from-demonstrations-27b553b00ce2) (the github branch of the library is at the end) – Angelo Apr 04 '19 at 10:28
  • thanks, I have been looking for building deep q agent for NLP tasks, but not much luck so far. – Abhinav Anand Apr 04 '19 at 12:06
  • 1
    Are you using Theano or Tensorflow Backend? If I interpret that error message right, it means that the first (Input) layer expects a one dimensional Input but for some reason your Input has shape (1,6). I recommend the following modification: inp = Input(shape=(1, look_back,)) Please let me know if this helps – Marcus Apr 06 '19 at 10:48
  • 1
    I would suggest you post standalone code that reproduces the problem (I couldn't find the implementation of PrioritizedMemory() where you pointed to). – Yishai E Apr 06 '19 at 17:42

0 Answers0