integer scalar arrays can be converted to a scalar index

Question

I am attempting to create a Q table following this blog post that I found on Medium.com. In my step function (python class) for creating a custom open AI Gym environment, my action space self.action_space would be 3 possible actions and the possible observations would be a values 50 to 150 in increments of 1 self.observation_space.

    #possible actions from more_heat less heat functions
    self.action_space = np.array([ 0,  1,  2])

    #possible deviation values from temp - setpoint
    self.observation_space = np.arange(50,150,1)

The blog post that I am following creates the Q table by this code below which I think is just creating an array of zeros based on the sizes.

action_size = env.action_space.n
state_size = env.observation_space.n

qtable = np.zeros((state_size, action_size))
print(qtable)

But when I attempt to print the Q table, I get this error: TypeError: only integer scalar arrays can be converted to a scalar index

Any words from the wise on what I am doing wrong would be greatly appreciated!

Same error just running this code:

import numpy as np
action_space = np.array([0,1,2])
observation_space = np.arange(50,150,1)

action_size = action_space
state_size = observation_space

qtable = np.zeros((state_size, action_size))
print(qtable)

Use `env.observation_space.shape[0]` to get the state space. Same for the action space. — Simon, Dec 09 '18 at 17:13
Cool... What does that do? It worked can you help me understand? If you post the answer I can hit the green check box. — bbartling, Dec 10 '18 at 16:08

score 0 · Accepted Answer · answered Dec 11 '18 at 10:12

Use env.observation_space.shape[0] to get the state space size. Same for the action space.

In the blog post you linked, they solve the frozen lake task. The task inherits from the discrete class which defines the action and obs spaces like this

    self.action_space = spaces.Discrete(self.nA)

Their Discrete class has an attribue n specifing the size of the discrete space. Instead, you use np.array which does not. You should actually get an error when you try to do

    action_size = env.action_space.n

or at least, I do if I try to run your code.

Hello @simon, Could possibly give me a tip with this SO post? https://stackoverflow.com/questions/53712923/python-binning-data-openai-gym — bbartling, Dec 12 '18 at 22:32

integer scalar arrays can be converted to a scalar index

1 Answers1