Iterate data structure while it is updated

Question

What is the correct way to iterate over a data structure in python while elements are removed from the structure at the same time?

I want to iterate over the someList structure and make sure to reach all items in the list as long as they are in the list and remove some of the items while I iterate it. But I don't understand why some numbers are skipped and how I can avoid this, resp. make sure to actually see each element in the list exactly once, under the condition, that it was not removed in advance. In the example I never saw 1, 5, and 8

class someList():
    def __init__(self):
        self.list = list(range(0,10))

    def getData(self):
        for i in self.list:
            print(i)
            yield i

thing = someList()
for a in thing.getData():
    print("we reached:", a)
    if a % 2 == 1:
        print("remove", a)
        thing.list.remove(a)
    elif (a * 2) in thing.list:
        print("remove double a:", a, " a*2:", a * 2)
        thing.list.remove(a*2)
print(thing.list)

Output:

0
we reached: 0
remove double a: 0  a*2: 0
2
we reached: 2
remove double a: 2  a*2: 4
3
we reached: 3
remove 3
6
we reached: 6
7
we reached: 7
remove 7
9
we reached: 9
remove 9
[1, 2, 5, 6, 8]

Intended Output:

0
we reached: 0
remove double a: 0  a*2: 0
1
we reached: 1
remove 1
2
we reached: 2
remove double a: 2  a*2: 4
3
we reached: 3
remove 3
5
we reached: 5
remove 5
6
we reached: 6
7
we reached: 7
remove 7
8
we reached: 8
9
we reached: 9
remove 9
[2, 6, 8]

Note: this is not the same question as How to remove items from a list while iterating? because I do not want to filter out the elements before I iterate.

The two modifying conditions are just examples as I indeed do iterate over a graph data structure, consume the current element and remove some elements that have a specific relation to the current element.

Why don't you want to create a new list without the elements? You can even re-assign the new list to the original list via a slice `[:]` — Chris_Rands, Dec 16 '19 at 15:30
If you are iterating a updating a `list`, you can do it using `list.copy()`. — accdias, Dec 16 '19 at 15:35
Don't; build a new list by adding the elements you want to keep, rather than removing the elements you don't want from the original list. — chepner, Dec 16 '19 at 15:37
I don't want to copy the data structure because it can be huge. I also can't just add the selected items to a new structure, because I need to remove connected elements, which are not yet visited, to not visit them later on. — white_gecko, Dec 16 '19 at 22:15

score 2 · Answer 1 · answered Dec 16 '19 at 16:20

As others have said, if you try to modify the list at runtime you'll not get correct itiration, so instead use another list to store what you want to remove,

class someList():
    def __init__(self):
        self.list = list(range(0,10))

    def getData(self):
        for i in self.list:
            print(i)
            yield i

thing = someList()
rem_list=[]
for a in thing.getData():
    print("we reached:", a)
    if a in rem_list:
        pass
    elif a % 2 == 1:
        print("remove", a)
        rem_list.append(a)
    elif (a * 2) in thing.list:
        print("remove double a:", a, " a*2:", a * 2)
        rem_list.append(2*a)
thing.list=[x for x in thing.list if x not in rem_list]
print(thing.list) #outputs [2, 6, 8]

using the rem_list to store the to-be-removed members and not checking them in loop will give you expected result..

Iterate data structure while it is updated

1 Answers1