Deleting elements from list or recreating new list (python)

Question

What it the best/fastest way to delete objects from list? Deleting some objects:

[objects.remove(o) for o in objects if o.x <= 0]

or recreating new object:

new_objects = [o for o in objects if o.x > 0]

for starters, you should not use list comprehensions for side effects. `[objects.remove(n) for o in objects if o.x <= 0]` creates a needless list of `None`'s (one for every item removed). Just use a loop. However, your algorithm has a classic bug of changing the size of the list while iterating over it. You can fix that in several ways, by manually handling the indexing, or by iterating backwards and deleting by index. But more to the point, removing from a list will be inefficient. Creating a new list has better time complexity. — juanpa.arrivillaga, Feb 16 '23 at 05:43
@juanpa.arrivillaga Just wanted to clarify that it will have better time complexity but worse space complexity. If you are working with limited memory (e.g. microcontrollers etc) you should consider that too. — Selcuk, Feb 16 '23 at 05:45
@Scott It depends on your use case. If you are not concerned about memory usage go for the fastest option as juanpa.arrivilaga explained. For general purpose computers and small lists memory is not the issue. — Selcuk, Feb 16 '23 at 05:48
@juanpa.arrivillaga, I actually used list comprehensions, becase they are faster. But I should not use them for this kinda purposes? — Scott, Feb 23 '23 at 01:59
@Scott they are not really faster than the equivalent loop, certainly not if you don't require creating an actual list, which you don't — juanpa.arrivillaga, Feb 23 '23 at 03:01
@juanpa.arrivillaga, according to many posts on the internet (including [this one](https://stackoverflow.com/questions/30245397/why-is-a-list-comprehension-so-much-faster-than-appending-to-a-list)), list comps are faster — Scott, Feb 23 '23 at 03:40
that is marginal, mostly due to avoiding the resoution of `.append` again and again (and I believe another bytecode hack). But as the work inside the body increases, those differences become irrelevant. List comprehensions exist for expressiveness, not as an optimization. — juanpa.arrivillaga, Feb 23 '23 at 08:16

score 3 · Accepted Answer · answered Feb 16 '23 at 05:49

For starters, don't use list comprehensions for side effects. It needlessly creates a list of None's here, which is simply ignored and garbage collected, and is just bad style. List comprehensions are for functional mapping/filtering operations to create new lists.

However, even converted to an equivalent loop there is a classic bug:

>>> objects = [1,1,2,2,1,1]
>>> for obj in objects:
...     if obj == 2:
...         objects.remove(obj)
...
>>> objects
[1, 1, 2, 1, 1]

This is because the internal iterator essentially keeps and index which it simply increments. Since the list changes size by removing an item, every index is shifted down, and an item is skipped. So when there are two matching items to be removed in a row, one is skipped.

But more to the point, removing from a list in a loop is inefficient even if you do it correctly.

It is quadratic time, whereas creating the new list is linear. So really, I think those are the clear advantages of creating a new list.

As pointed out by @Selcuk in the comments, the advantage of modifying the list is that you don't use auxiliary space.

score 2 · Answer 2 · edited Feb 18 '23 at 00:47

There is a potential issue with modifying a list this way while iterating over it. When you remove an item from the list, the indices of the remaining items shift down by one, which might be an issue if you are trying to access it by index.

The best method is using a del statement as it is faster than remove() method, because it avoids the need to search the list for the item to remove.

    i = 0
while i < len(objects):
    if objects[i].x <= 0:
        del objects[i]
    else:
        i += 1

Just the the readability of code has decreased now .

For question of recreating vs updating , no doubt recreating is faster as while updating the index gets changed again and again , creating a new list avoids the need to shift the indices of the remaining items in the list. But it can increase the space complexity by huge amount if the list is large.

For a faster way of your problem you can consider a generator expression instead of a list comprehension. A generator expression is similar to a list comprehension, but it produces a generator object that can be iterated over lazily, rather than creating a new list in memory.

new_objects = (o for o in objects if o.x > 0)

For more information about generator expression , you can check this out : Generator expressions vs. list comprehensions

Helpful answer. But, I didn't use indices while removing objects. For removing objects from list, you don't need indices, cuz no object is same even tho they are derived from same class. So, you can just use `.remove()` for deleting any object from list. And, probably you should consider using `for` instead of `while` (since you work with indices) — Scott, Feb 23 '23 at 02:09

Deleting elements from list or recreating new list (python)

2 Answers2