How to solve problem with the items in a list

Question

Question originally posted in Spanish, on es.stackoverflow.com, by Ezequiel:

I am trying to make a program that removes certain elements from a group of lists if a certain condition passes, and that duplicates them if a certain other happens. But I have the following problem: when the program is in iteration 50 of 100, it marks IndexError and closes.

Here is the code snippet in which I have the problem:

nnl_len = len(neuralNetworkList) # longitud de lista de objetos
fitl_copy = fitness # lista de floats
best_fitness_reproduced = 0 

if worst_fitness < 0 and first_iteration == 0: # I guess this can
    worst_fitness = 0                          # be ignored

    for i in range(nnl_len):
        print(i)
        if fitl_copy[i] < best_fitness: # EXACT LINE OF THE ERROR <------------------
            print("I DIED WITH A FITNESS OF ",fitness[i], ", BEING THE LIMIT ",best_fitness)
            neuralNetworkList.pop(i)    
            x.pop(i)
            y.pop(i)
            fitness.pop(i)
            colors.pop(i)
        elif fitl_copy[i] == best_fitness and best_fitness_reproduced:
            print("I DIED BECAUSE A TOP FITNESS CREATURE ALREADY REPRODUCED ",fitness[i])
            neuralNetworkList.pop(i)    
            x.pop(i)
            y.pop(i)
            fitness.pop(i)
            colors.pop(i)               
        else:           
            best_fitness_reproduced = 1
            for j in range(99): # plus the mother is 100
                print("I SURVIVED WITH A FITNESS OF ",fitness[i], ", BEING THE LIMIT ",best_fitness)                    
                neuralNetworkList.append(neuralNetworkList[i])
                if random.randint(1,3) == 1:
                    neuralNetworkList[i].mutate(i)
                x.append(width)
                y.append(height)
                fitness.append(0)

                newcolor = []

                for h in range(3):
                    newcolor.append(round( colors[i][h]*random.choice((0.9, 1.1)) ))

                colors.append(newcolor)     
        #except IndexError:
        #   pass
        #   print("I NEITHER DIED NOR REPRODUCED BECAUSE OF AN INDEX ERROR")

    nnl_len = len(neuralNetworkList)

    for i in range(nnl_len):
        x[i] = width
        y[i] = height
        fitness[i] = 0

    print("population after reproduction:", len(neuralNetworkList))

this is the traceback:

Traceback (most recent call last): 
  File "C:\Users\Ezequiel\Desktop\Archivos pavos\Sublime Text 3\pruebas_phyton.pyw", line 4921, in <module>
    if fitl_copy[i] < best_fitness: # fitness[i] <= worst_fitness*4
IndexError: list index out of range

`nnl_len` is the length of your list `neuralNetworkList`. You iterate over the range of `nnl_len` but then change the actual list itself during this iteration via `pop` and `append`. You don't want to do that... — Alexander, Oct 11 '19 at 21:40
Note that `fitl_copy` is not actually a copy of `fitness`, it is a reference to the same list. Thus as @Alexander and @G. Anderson note, you are changing the list while iterating over it. — rcriii, Oct 11 '19 at 22:18

score 1 · Accepted Answer · answered Oct 11 '19 at 22:20

when the program is in iteration 50 of 100, it marks IndexError and closes.

That is the clue; the problem happens halfway through the process. This is a sign of trying to remove from a list while iterating over it; each time through the loop you remove an element, so after 50 times you have i == 50 and only 50 elements still in the list, so the index is out of bounds.

Why does this happen? Well...

fitl_copy = fitness # lista de floats

This does not make a copy of fitness; it makes another name for fitness.

Consequently, these two lines...

if fitl_copy[i] < best_fitness:
    # ...
    fitness.pop(i)

operate on the same list; and boom.

I think you should first try to rethink your algorithm. My guess is that you don't really need to want to do all this popping from lists at all. I think you will have a much easier time if you:

Rewrite so that you produce a new list with all of the creatures for the new generation, starting from scratch (empty list)
Leave the old creature list alone while you do this, and then just replace it with the new list at the end

You may also benefit from using some kind of structured data for your creatures, instead of having parallel lists with all their attributes; and also by using list comprehensions and generators instead of for-loops.

Actually, reading the code again, it seems that you want exactly one creature to reproduce - the first one in the list that is tied for best fitness. So then you don't really want to process all the list items iteratively at all; you want to *find* the creature that qualifies, and reproduce it. — Karl Knechtel, Oct 11 '19 at 22:23
thank you! I didn't know that list1 = list2 gave a new name to a list. list1 = list (list2) would it work? The reason why I have parallel lists instead of everything together is that I am programming a function to create neural networks, and so as not to have to edit your code every time I use it in different projects, I decided to store information not related to the function in separate lists — , Oct 11 '19 at 22:48
Yes, calling `list()` around an existing list does make a (shallow) copy. [There are a few other ways](https://stackoverflow.com/questions/2612802/how-to-clone-or-copy-a-list) you will see for this in old code. But if the point is to *make a copy*, then in new code (Python 3.3 onward) you should just use the `.copy()` method of the list. — Karl Knechtel, Oct 11 '19 at 22:58

How to solve problem with the items in a list

1 Answers1