Elements skipped when trying to remove from a list

Question

I have a list of floats and I want to remove elements which are either greater than a certain number or less than a certain (different) number. This is the code that I have so far:

a = ['239.12835442230858', '22.756759356300982', '32.98715001831536', '460.11899977144816', '747.5029387021966', '25.97292253382583', '150.94457242947556', '197.14098031753895', '394.44390781374057', '320.11301380981826', '138.66237294322895', '87.7095435481745', '427.8799219223041', '445.9006435386053', '62.716459379313704', '259.19872485615906', '212.75590521862267', '455.3882240904654', '259.07638854400676', '425.90576134140446']
print type(a[0])
for i, val in enumerate(a):
    a[i] = float(val)

print type(a[0])

for i in a:
    if i < 30 or i > 200:
        a.remove(i)

But this doesn't work as I expected it to. When I visualize this code here: http://www.pythontutor.com/visualize.html, it seems to be skipping the elements that come after an element that is removed. I see no reason why this would be the case with my code.

P.S. I am aware that I can do what I'm trying to do using np.where, but I want to know why my current code does not work.

I would be interested to see an answer to the exact question asked, not code that gives the desired output. Why do some elements get skipped when performing this method? — d_kennetz, Oct 04 '18 at 03:40
@d_kennetz: List iterators are index based. If you remove an element while iterating, the next value is shifted down to the current index, then the iterator advances the index to the next item, skipping the item that was just shifted down. I swear this question has been asked a dozen times already (just added four duplicates, there are *many* more). — ShadowRanger, Oct 04 '18 at 03:44

Matt Messersmith · Accepted Answer · 2018-10-04T03:49:59.637

2

You're modifying the list as you're iterating over it. Do a list comprehension instead:

[i for i in a if not (i < 30 or i > 200)]

edited Oct 04 '18 at 03:49

answered Oct 04 '18 at 03:26

Matt Messersmith

12,939
6
51
52

Ah! That makes sense, thank you – Always Learning Forever Oct 04 '18 at 03:27
Cant compare `str` and `float` and operators are backwards, we are removing `i < 30` and `i > 200` using this comprehension would keep those values instead – vash_the_stampede Oct 04 '18 at 03:33
@vash_the_stampede In my code, I do change all the `str`s to `float`s – Always Learning Forever Oct 04 '18 at 03:34
@AlwaysLearningForever im speaking of this line – vash_the_stampede Oct 04 '18 at 03:35
I agree with that, but I already know what I am doing wrong and changing from `>` to `<` and from `<` to `>` should be pretty straightforward. :) – Always Learning Forever Oct 04 '18 at 03:35
2

But why do the elements get skipped? I am interested in his original question. I know this code gives the desired behavior, but I am curious as to why some items do not get removed that meet the two conditions. – d_kennetz Oct 04 '18 at 03:35
@d_kennetz That's a good point and something to play around with. It's interesting – Always Learning Forever Oct 04 '18 at 03:36
@AlwaysLearningForever not that simple because before you were dropping values under 30 and values over 200 when you flip operators now you have `i > 30` and `i < 200` which is not exactly the same values you are looking for – vash_the_stampede Oct 04 '18 at 03:37
1

if you do the operation simply as: `for i in a: if float(i) > 200: a.remove(i)` you actually still get values > 200. When I try it, the floats 747. and 445. are still hanging around. – d_kennetz Oct 04 '18 at 03:39
@d_kennetz it has to do with the fact that items get moved forward in the "slots" of the list to replace the item that was removed, so if you remove items while iterating you'll *always* skip the item immediately to the right of the item removed. – Matt Messersmith Oct 04 '18 at 03:42
Oh this makes sense, you would need to do while to remove all elements that fit the criteria? – d_kennetz Oct 04 '18 at 03:43
1

This list comprehension returns values that are ones that you are trying to remove – vash_the_stampede Oct 04 '18 at 03:48
1

@vash_the_stampede you're right, i'll edit accordingly – Matt Messersmith Oct 04 '18 at 03:49

Elements skipped when trying to remove from a list

1 Answers1