IndexError: list index out of range but I "know" it is

Question

I am wondering why the following situation may occur:

I have a list:

all_hits = list1 + list2
length_hits = len(all_hits)
i = 0 
while i < int(length_hits) - 1:
    if len((all_hits[i])[1]) >=3:
        Do x
    else:
        Do y

This gives me an error:

  File "file.py", line z, in Function:
    if len((all_hits[i])[1]) >= 3: 
IndexError: list index out of range

So far:

If I write each all_hits[i][1] which is >=3 characters long to a file I reach about 18,000 entries but if I print len(all_hits) the answer is 39,000 elements.
I can print the all_hits[-1] entry and it does not correspond with the final entry in the written list.

As far as I can see I am not altering the original list.

Why?! I have been 2 days on this problem and it is driving me crazy.

Edit: 3. I have printed the final entry and a few after/before it to ensure they have the correct number of elements and everything looks ok there.

Edit and Thank you:

Thanks everyone. You were, of course, correct. One of my entries had, out of nowhere, only one list object. I don't know how or why but that's a different kettle of fish altogether!

Thank you for your help!

You are assuming that the inner list has at least two items `[1]`. Check that it does. — André Laszlo, Jan 14 '15 at 10:28
You didn't test if `all_hits[i]` contains at least 2 entries. — Martijn Pieters, Jan 14 '15 at 10:30
Thank you for your comments. I have printed the final entry that will write to a file and it has two entries. It looks exactly like the 'other' entries. — poppyseeds, Jan 14 '15 at 10:33
Can you wrap your list access with a try / except (https://docs.python.org/2/tutorial/errors.html) to determine the wrong "i" value — Jaay, Jan 14 '15 at 10:38

Loupi · Accepted Answer · 2015-01-14T11:43:23.483

In this line if len((all_hits[i])[1]) >= 3: you should know what you are assuming by the double dereference [i][1]:

all_hits is a list of lists. For example [[1,2],[4,5],[7,8],[9]]
and each list within all_hits is at least 2 entries long.

all_hits[0] refers to the first list/entry in all_hits -- [1,2] in my example.

Thus all_hits[0][1] refers to the second element in the [1,2] example list namely 2.

Now you can imagine that when you get to the 4th entry in all_hits namely [9] in my example.

all_hits[3] = [9] but then all_hits[3][1] is out of bounds because you only have one entry namely all_hits[3][0]

So make sure each nested list is at least 2 entries long. If you know they are then you should not have the problem. Rather try this:

    all_hits = list1 + list2
    i = 0 
    while i <= len(all_hits) - 1:
        if len(all_hits[i][1]) >=3:
            Do x
        else:
            Do y
        i += 1

score 1 · Answer 2 · answered Jan 14 '15 at 11:28

In Python, you don't need to know the length of the list to traverse it. Secondly, don't assume the length of the inner item, try and see if it is what you expect:

for something in all_hits:
   try:
      if len(something[1]) >= 3:
          do(x)
      else:
          do(y)
   except IndexError:
      print('{} does not have two items'.format(something))

score 0 · Answer 3 · 2015-01-14T13:50:12.543

I agree with @Loupy about your problem, but if you are sure that it exists and python says that its not, so how can we help to you? You should resolve your "grudges" with python and trust to python.

Also I see in code another problem:

while i < int(length_hits) - 1:

function len always returns integers only, so explicit casting unnecessary
expression int(length_hits) - 1 can be negative(in case of list is emty), so theoretically you can fall in infinite loop(i = -2 will cause to infinite loop).

IMO it should be:

while len(all_hits):
    do_staff()

EDIT

I have no reputation to add comment on Burhan Khalid response, but I wold not recommend to use for on sequence that are changed during execution. Its a bad practice.

IndexError: list index out of range but I "know" it is

3 Answers3