172

I have a list where I want to replace values with None where condition() returns True.

[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10]

For example, if condition checks bool(item%2) should return:

[None, 1, None, 3, None, 5, None, 7, None, 9, None]

What is the most efficient way to do this?

PolyGeo
  • 1,340
  • 3
  • 26
  • 59
ak.
  • 2,422
  • 4
  • 21
  • 18

7 Answers7

246

Build a new list with a list comprehension:

new_items = [x if x % 2 else None for x in items]

You can modify the original list in-place if you want, but it doesn't actually save time:

items = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
for index, item in enumerate(items):
    if not (item % 2):
        items[index] = None

Here are (Python 3.6.3) timings demonstrating the non-timesave:

In [1]: %%timeit
   ...: items = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
   ...: for index, item in enumerate(items):
   ...:     if not (item % 2):
   ...:         items[index] = None
   ...:
1.06 µs ± 33.7 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each)

In [2]: %%timeit
   ...: items = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
   ...: new_items = [x if x % 2 else None for x in items]
   ...:
891 ns ± 13.6 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each)

And Python 2.7.6 timings:

In [1]: %%timeit
   ...: items = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
   ...: for index, item in enumerate(items):
   ...:     if not (item % 2):
   ...:         items[index] = None
   ...: 
1000000 loops, best of 3: 1.27 µs per loop
In [2]: %%timeit
   ...: items = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
   ...: new_items = [x if x % 2 else None for x in items]
   ...: 
1000000 loops, best of 3: 1.14 µs per loop
user2357112
  • 260,549
  • 28
  • 431
  • 505
John Millikin
  • 197,344
  • 39
  • 212
  • 226
  • 2
    is that the most efficient? doesn't enumerate have to create an iterator and form a tuple, adding overhead? are lists in python arraylists, giving you constant time access? – geowa4 Oct 08 '09 at 20:05
  • I think, and I might be wrong, that he meant for a copy of the list to be returned instead of modifying the original in place. But still, +1 for offering the efficient solution when in-place modification is allowed. – A. Levy Oct 08 '09 at 20:07
  • If I wanted to modify the original in place, wouldn't it also possible to use imap from itertools? – ak. Oct 08 '09 at 20:19
  • 2
    @geowa4: Python "lists" are actually arrays. `enumerate()` will ad a small overhead, but if that's unacceptable the index can be tracked manually. @ak: I don't understand the question. `imap()` is not an in-place operation. – John Millikin Oct 08 '09 at 20:30
  • @John Millikin: I was just thinking, what if I had a generator that yields the values from range(11) instead of a list. Would it be possible to replace values in the generator? – ak. Oct 08 '09 at 20:56
  • Generators don't have values to replace; when called, their `next()` method returns values until eventually it doesn't. This is a useful API for generic iteration, but it prevents some performance optimizations. – John Millikin Oct 08 '09 at 22:00
  • @JohnMillikin Please comment on my [answer](http://stackoverflow.com/a/24203748/307454) for a related [question](http://stackoverflow.com/questions/24201926/how-to-replace-all-occurrences-of-an-element-in-a-list-in-python-in-place/24203748#24203748) – lifebalance Jun 13 '14 at 13:07
  • 2
    Thanks for the answers. But isn't there enough of a use case here for python lists to have a replace method? (similar to str.replace). Something that avoids the need for this: `def replace(items, a, b): return [b if x == a else x for x in items]` – Bill Sep 13 '16 at 17:07
  • I like this answer, but if you want predefined indecies, say: `RemoveItems = [1,3,4]` then you can retain the `enumerate` command like so `[None if n in RemoveItems else item for n,item in enumerate(items)]` – Timo Kvamme Nov 15 '16 at 22:41
  • 3
    I think you got "easiest to read" and "most efficient" backwards – endolith Mar 04 '17 at 23:33
  • Python 3 is really 100x faster than Python 2 for this task? – BallpointBen Apr 19 '18 at 15:36
  • You can modify the list in place and use a comprehension at the same time: `items[:] = [b if x == a else x for x in items]`. – Marius Jul 22 '21 at 05:21
79
ls = [x if (condition) else None for x in ls]
Laurence Gonsalves
  • 137,896
  • 35
  • 246
  • 299
13

Here's another way:

>>> L = range (11)
>>> map(lambda x: x if x%2 else None, L)
[None, 1, None, 3, None, 5, None, 7, None, 9, None]
balpha
  • 50,022
  • 18
  • 110
  • 131
  • 9
    @gath: Don't aspire to write one-liners for every purpose. Sometimes, they increase readability or performance, but often they don't. As to hints: Get to know the tools that Python ofters, especially list (and for Python 3 also dict) comprehensions, the ternary operator, anonymous (lambda) functions, and functions like map, zip, filter, reduce, etc. – balpha Oct 09 '09 at 12:56
12

Riffing on a side question asked by the OP in a comment, i.e.:

what if I had a generator that yields the values from range(11) instead of a list. Would it be possible to replace values in the generator?

Sure, it's trivially easy...:

def replaceiniter(it, predicate, replacement=None):
  for item in it:
    if predicate(item): yield replacement
    else: yield item

Just pass any iterable (including the result of calling a generator) as the first arg, the predicate to decide if a value must be replaced as the second arg, and let 'er rip.

For example:

>>> list(replaceiniter(xrange(11), lambda x: x%2))
[0, None, 2, None, 4, None, 6, None, 8, None, 10]
Alex Martelli
  • 854,459
  • 170
  • 1,222
  • 1,395
  • +1 hehe... i want to learn how to write this "one" line nifty python solution... hint pls – gath Oct 09 '09 at 11:34
  • @gath, I don't understand your question -- comments are pretty limiting so you should open a new question so you can expand and clarify what is it you're looking for... – Alex Martelli Oct 09 '09 at 15:23
2
>>> L = range (11)
>>> [ x if x%2 == 1 else None for x in L ]
[None, 1, None, 3, None, 5, None, 7, None, 9, None]
eduffy
  • 39,140
  • 13
  • 95
  • 92
1

In case you want to replace values in place, you can update your original list with values from a list comprehension by assigning to the whole slice of the original.

data = [*range(11)] # [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
id_before = id(data)
data[:] = [x if x % 2 else None for x in data]
data
# Out: [None, 1, None, 3, None, 5, None, 7, None, 9, None]
id_before == id(data)  # check if list is still the same
# Out: True

If you have multiple names pointing to the original list, for example you wrote data2=data before changing the list and you skip the slice notation for assigning to data, data will rebind to point to the newly created list while data2 still points to the original unchanged list.

data = [*range(11)] # [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
data2 = data
id_before = id(data)
data = [x if x % 2 else None for x in data]  # no [:] here
data
# Out: [None, 1, None, 3, None, 5, None, 7, None, 9, None]
id_before == id(data)  # check if list is still the same
# Out: False
data2
# Out: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10]

Note: This is no recommendation for generally preferring one over the other (changing list in place or not), but behavior you should be aware of.

Darkonaut
  • 20,186
  • 7
  • 54
  • 65
-7

This might help...

test_list = [5, 8]
test_list[0] = None
print test_list
#prints [None, 8]
Emil
  • 41
  • 2
  • 10