Value Error: too many values to unpack (expected 3)

Question

I was trying to implement the value iteration algorithm. I have a grid

grid = [[0, 0, 0, +1],
    [0, "W", 0, -1],
    [0, 0, 0, 0]]

An actionlist

actlist = {UP:1, DOWN:2, LEFT:3, RIGHT:4}

And a reward function

reward = [[0, 0, 0, 0],
      [0, 0, 0, 0],
      [0, 0, 0,  0]]

I wrote a function T, which returns tuple of 3 tuples.

def T(i,j,actions):
if(i == 0 and j == 0):
    if(actions == UP):
        return (i,i,0.8),(i,i,0.1),(i,j+1,0.1)
    elif(actions == DOWN):
        return (i+1,j,0.8),(i,j,0.1),(i,j+1,0.1)
    elif(actions == LEFT):
        return (i,j,0.8),(i,j,0.1),(i+1,j,0.1)
    elif(actions == RIGHT):
        return (i,j+1,0.8),(i,i,0.1),(i+1,j,0.1)
elif (i == 0 and j == 1):
    if(actions == UP):
        return (i,i,0.8),(i,j-1,0.1),(i,j+1,0.1)
    elif(actions == DOWN):
        return (i,j,0.8),(i,j-1,0.1),(i,j+1,0.1)
    elif(actions == LEFT):
        return (i,j-1,0.8),(i,j,0.1),(i,j,0.1)
    elif(actions == RIGHT):
        return (i,j+1,0.8),(i,j,0.1),(i,j,0.1)
elif(i == 0 and j == 2):
    if(actions == UP):
        return (i,j,0.8),(i,j-1,0.1),(i,j+1,0.1)
    elif(actions == DOWN):
        return(i+1,j,0.8),(i,j-1,0.1),(i,j+1,0.1)
    elif(actions == LEFT):
        return (i,j-1,0.8),(i,j,0.1),(i+1,j,0.1)
    elif(actions == RIGHT):
        return (i,j+1,0.8),(i,j,0.1),(i+1,j,0.1)
elif(i == 0 and j == 3):
    if(actions == UP):
        return (-1,-1,0.8),(-1,-1,0.1),(-1,-1,0.1)
    elif(actions == DOWN):
        return (-1,-1,0.8),(-1,-1,0.1),(-1,-1,0.1)
    elif(actions == LEFT):
        return (-1,-1,0.8),(-1,-1,0.1),(-1,-1,0.1)
    elif(actions == RIGHT):
        return (-1,-1,0.8),(-1,-1,0.1),(-1,-1,0.1)
# 2nd row
elif (i == 1 and j == 0):
    if(actions == UP):
        return (i-1,j,0.8),(i,j,0.1),(i,j,0.1)
    elif(actions == DOWN):
        return (i+1,j,0.8),(i,j,0.1),(i,j,0.1)
    elif(actions == LEFT):
        return (i,j,0.8),(i-1,j,0.1),(i+1,j,0.1)
    elif(actions == RIGHT):
        return (i,j,0.8),(i-1,j,0.1),(i+1,j,0.1)
elif(i == 1 and j ==1):
    if(actions == UP):
        return (i,j,0.8),(i,j,0.1),(i,j,0.1)
    elif(actions == DOWN):
        return (i,j,0.8),(i,j,0.1),(i,j,0.1)
    elif(actions == LEFT):
        return (i,j,0.8),(i,j,0.1),(i,j,0.1)
    elif(actions == RIGHT):
        return (i,j,0.8),(i,j,0.1),(i,j,0.1)
elif (i == 1 and j == 2):
    if(actions == UP):
        return (i-1,j,0.8),(i,j,0.1),(i,j+1,0.1)
    elif(actions == DOWN):
        return (i+1,j,0.8),(i,j,0.1),(i,j+1,0.1)
    elif(actions == LEFT):
        return (i,j,0.8),(i-1,j,0.1),(i+1,j,0.1)
    elif(actions == RIGHT):
        return (i,j+1,0.8),(i-1,j,0.1),(i+1,j,0.1)
elif(i == 1 and j == 3):
    if(actions == UP):
        return (-2,-2,0.8),(-2,-2,0.1),(-2,-2,0.1)
    elif(actions == DOWN):
        return (-2,-2,0.8),(-2,-2,0.1),(-2,-2,0.1)
    elif(actions == LEFT):
        return (-2,-2,0.8),(-2,-2,0.1),(-2,-2,0.1)
    elif(actions == RIGHT):
        return (-2,-2,0.8),(-2,-2,0.1),(-2,-2,0.1)      
# 3rd row
elif(i == 2 and j == 0):
    if(actions == UP):
        return (i-1,j,0.8),(i,j,0.1),(i,j+1,0.1)
    elif(actions == DOWN):
        return (i,j,0.8),(i,j,0.1),(i,j+1,1,0.1)
    elif(actions == LEFT):
        return (i,j,0.8),(i-1,j,0.1),(i,j,0.1)
    elif(actions == RIGHT):
        return (i,j+1,0.8),(i-1,j,0.1),(i,j,0.1)
elif (i == 2 and j == 1):
    if(actions == UP):
        return (i,j,0.8),(i,j-1,0.1),(i,j+1,0.1)
    elif(actions == DOWN):
        return (i,j,0.8),(i,j-1,0.1),(i,j+1,0.1)
    elif(actions == LEFT):
        return (i,j-1,0.8),(i,j,0.1),(i,j,0.1)
    elif(actions == RIGHT):
        return (i,j+1,0.8),(i,j,0.1),(i,j,0.1)
elif(i == 2 and j == 2):
    if(actions == UP):
        return (i-1,j,0.8),(i,j-1,0.1),(i,j+1,0.1)
    elif(actions == DOWN):
        return (i,j,0.8),(i,j-1,0.1),(i,j+1,0.1)
    elif(actions == LEFT):
        return (i,j-1,0.8),(i-1,j,0.1),(i,j,1)
    elif(actions == RIGHT):
        return (i,j+1,0.8),(i-1,j,0.1),(i,j,0.1)
elif(i == 2 and j == 3):
    if(actions == UP):
        return (i-1,j,0.8),(i,j-1,0.1),(i,j,0.1)
    elif(actions == DOWN):
        return (i,j,0.8),(i,j-1,0.1),(i,j,0.1)
    elif(actions == LEFT):
        return (i,j-1,0.8),(i-1,j,0.1),(i,j,0.1)
    elif(actions == RIGHT):
        return (i,j,0.8),(i-1,j,0.1),(i,j,0.1)

This function is called in the value iteration function:

def value_iteration():
U1 = [[0, 0, 0, 0],
      [0, 0, 0, 0],
      [0, 0, 0, 0]] 
while True:
    U=U1.copy()
    delta = 0
    for i in range(len(grid)):
        for j in range(len(grid[i])):
            U1[i][j] = max(sum(p*(R(k,l)+gamma*U[k][l]) for (k,l,p) in T(i,j,a)) for a in actlist)
            print(i,j,U1[i][j])
            delta = max(delta, abs(U1[i][j] - U[i][j]))
    if delta <= epsilon*(1 - gamma)/gamma:
        return U

The problem is, the first two iteration of the for loop went well with output

But then the code stopped with the error

ValueError: too many values to unpack (expected 3)

Can you provide the complete error message so we know where exactly the error is being triggered? — vielkind, Aug 06 '18 at 13:39
`U1[i][j] = max(sum(p*(R(k,l)+gamma*U[k][l]) for (k,l,p) in T(i,j,a)) for a in actlist)` The error is triggered on this line. — Shifat E Arman, Aug 06 '18 at 13:42
Side question: if all your function does is act like a table, why don't you use a dictionary? — Edward Minnix, Aug 06 '18 at 13:48
@ShifatEArman You would simply make a dictionary where the keys were 3-element tuples whose values are `(i, j, action)`, and whose values were the results that you would return from the function (though they will need to be hard-coded. If you are unfamiliar with dictionaries, I'd suggest you reference [the official documentation](https://docs.python.org/3/tutorial/datastructures.html#dictionaries) — Edward Minnix, Aug 06 '18 at 14:17

score 5 · Accepted Answer · answered Aug 06 '18 at 13:41

5

Have a look at the tuple in ** **, maybe that's the reason

# 3rd row
elif(i == 2 and j == 0):
    if(actions == UP):
        return (i-1,j,0.8),(i,j,0.1),(i,j+1,0.1)
    elif(actions == DOWN):
        return (i,j,0.8),(i,j,0.1),**(i,j+1,1,0.1)**
    elif(actions == LEFT):
        return (i,j,0.8),(i-1,j,0.1),(i,j,0.1)
    elif(actions == RIGHT):
        return (i,j+1,0.8),(i-1,j,0.1),(i,j,0.1)
elif (i == 2 and j == 1):
    if(actions == UP):
        return (i,j,0.8),(i,j-1,0.1),(i,j+1,0.1)
    elif(actions == DOWN):
        return (i,j,0.8),(i,j-1,0.1),(i,j+1,0.1)
    elif(actions == LEFT):
        return (i,j-1,0.8),(i,j,0.1),(i,j,0.1)
    elif(actions == RIGHT):
       return (i,j+1,0.8),(i,j,0.1),(i,j,0.1)

answered Aug 06 '18 at 13:41

Piotrek

1,400
9
16

Stupid mistake. Thanks a lot. :) – Shifat E Arman Aug 06 '18 at 13:46
Sure. Another thing, I'm updating the `U=U1.copy()` after both for loop is done. But it is being updated automatically inside my for loop. I can't get the reason. – Shifat E Arman Aug 06 '18 at 13:54
not sure that solves your problem, but everytime you use the function you 'reset' `U` and `U1` and return `U`, which is zeros only – Piotrek Aug 06 '18 at 13:59

score 0 · Answer 2 · answered Oct 28 '18 at 20:21

As @EdwardMinnix mentioned, you should use map and never go for such if/else construction. Or if there is any pattern with those values please take a look at Stragety Pattern.

This will help you a bit:

...

VALUE_A = 0.8
VALUE_B = 0.1


def new_T(i, j, actions):
    result_map = {(0, 0, 1): ((i, i, VALUE_A), (i, i, VALUE_B), (i, j + 1, VALUE_B)),
              (0, 0, 2): ((i + 1, j, VALUE_A), (i, j, VALUE_B), (i, j + 1, VALUE_B)),
              (0, 0, 3): ((i, j, VALUE_A), (i, j, VALUE_B), (i + 1, j, VALUE_B)),
              (0, 0, 4): ((i, j + 1, VALUE_A), (i, i, VALUE_B), (i + 1, j, VALUE_B)),
              (0, 1, 1): ((i, i, VALUE_A), (i, j - 1, VALUE_B), (i, j + 1, VALUE_B)),
              (0, 1, 2): ((i, j, VALUE_A), (i, j - 1, VALUE_B), (i, j + 1, VALUE_B)),
              (0, 1, 3): ((i, j - 1, VALUE_A), (i, j, VALUE_B), (i, j, VALUE_B)),
              (0, 1, 4): ((i, j + 1, VALUE_A), (i, j, VALUE_B), (i, j, VALUE_B)),
              (0, 2, 1): ((i, j, VALUE_A), (i, j - 1, VALUE_B), (i, j + 1, VALUE_B)),
              (0, 2, 2): ((i + 1, j, VALUE_A), (i, j - 1, VALUE_B), (i, j + 1, VALUE_B)),
              (0, 2, 3): ((i, j - 1, VALUE_A), (i, j, VALUE_B), (i + 1, j, VALUE_B)),
              (0, 2, 4): ((i, j + 1, VALUE_A), (i, j, VALUE_B), (i + 1, j, VALUE_B))}

    return result_map.get((i, j, actions))


for i, j, action in itertools.product(range(4), range(4), range(1, 5)):
    print('%s %s %s' % (i, j, action))
    T_value = T(i, j, action)
    new_T_value = new_T(i, j, action)
    if T_value != new_T_value:
        raise AssertionError('Error! \nT:     %s \nNew T: %s' % (T_value, new_T_value))

Value Error: too many values to unpack (expected 3)

2 Answers2