29

I'm trying to use map in Python3. Here's some code I'm using:

import csv

data = [
    [1],
    [2],
    [3]
]

with open("output.csv", "w") as f:
    writer = csv.writer(f)
    map(writer.writerow, data)

However, since map in Python3 returns an iterator, this code doesn't work in Python3 (but works fine in Python2 since that version of map always return a list)

My current solution is to add a list function call over the iterator to force the evaluation. But it seems odd (I don't care about the return value, why should I convert the iterator into a list?)

Any better solutions?

yegle
  • 5,795
  • 6
  • 39
  • 61
  • 3
    Using `map` for side effects is what's odd. Python 2 `map` also collects the return values. The new behavior merely highlights it further. Just don't do that, [use a for loop](http://stackoverflow.com/q/5753597/395760). –  Aug 25 '13 at 20:45
  • @delnan Thank you for the link, indeed I shouldn't use `map` for side effect. – yegle Aug 25 '13 at 20:48
  • 4
    for Python 3, `list(map(lambda x:2*x, [1,2,3]))` – Zhe Hu Aug 05 '16 at 21:01
  • I feel like "map for the side effects" *is* a common-enough idiom that one could justify including it as a language feature. However, python3 has decided that `map` shall be a lazily-evaluated function more in line with other languages. In any case, I find myself typing `[*map(...)]` **almost all the time** when I'm using Python for scientific statistics, where usually an immediate result is needed (e.g. `numpy.array(...)` understands lists but not generators). So the non-lazy map is the superior default for some applications. – MRule Oct 28 '22 at 07:54
  • FWIW I've defined the helpers `lmap` and `amap` that add the necessary boiler plate to immediately collect the results of `map()` as a list or np.array, respectively. It makes the code significantly cleaner, easier to read, and easier to maintain. Scattering casts to `list` or `[*map(...)]` everywhere, or using a list comprehension when `map` is much more succinct, just looks sloppy. – MRule Oct 28 '22 at 07:55

6 Answers6

29

Using map for its side-effects (eg function call) when you're not interested in returned values is undesirable even in Python2.x. If the function returns None, but repeats a million times - you'd be building a list of a million Nones just to discard it. The correct way is to either use a for-loop and call:

for row in data:
    writer.writerow(row)

or as the csv module allows, use:

writer.writerows(data)

If for some reason you really, really wanted to use map, then you can use the consume recipe from itertools and generate a zero length deque, eg:

from collections import deque
deque(map(writer.writerow, data), maxlen=0)
Jon Clements
  • 138,671
  • 33
  • 247
  • 280
6

You can set up a zero length deque to do this:

with open("output.csv", "w") as f:
    writer = csv.writer(f)
    collections.deque(map(writer.writerow, data),0)

This is the same way that itertools.consume(iterator, None) recipe works. It functionally will exhaust the iterator without building the list.

You can also just use the consume recipe from itertools.

But a loop is more readable and Pythonic to me, but YMMV.

sds
  • 58,617
  • 29
  • 161
  • 278
dawg
  • 98,345
  • 23
  • 131
  • 206
5
list(map(lambda x: do(x),y))

will trigger an evaluation and stay in english pen-and-paper math semantics. Some like this more than others, but I've had a coder who spoke English as a second language tell me this is harder to understand than:

[ do(x) for x in y ]

¯\(ツ)/¯

Anyhow, if you want to flush the map stack, trigger it with a list type conversion or a list comprehension.

With context:

import csv

data = [
    [1],
    [2],
    [3]
]

with open("output.csv", "w") as f:
    writer = csv.writer(f)
    list(map(writer.writerow, data))
    # or: [ writer.writerow(x) for x in data ]
Chris
  • 28,822
  • 27
  • 83
  • 158
3

If you don't care about the return value, then map is not the best tool for the job. A simple for would be better:

for d in data:
    writer.writerow(d)

That'll work fine in Python 2.x and 3.x. Notice that map is useful when you want to create a new list, if you're traversing over an iterable just for the effect, then use a for.

Óscar López
  • 232,561
  • 37
  • 312
  • 386
2

You can also use list comprehension, as suggested in the official FAQ:

with open("output.csv", "w") as f:
    writer = csv.writer(f)
    [writer.writerow(elem) for elem in data]

List comprehension will force the evaluation of each element even if you don't assign the newly created list to any variable.

Note, however, that the list may still be created behind the scenes, creating a potential performance pitfall, so while relatively succinct, it should not be used if the input sequence can get very long.

undercat
  • 529
  • 5
  • 17
0

I would use a function to extract data from the iterable using something like this:

def rake(what, where=None):
    for i in what: 
        if where: where.append(i)

rake(map(writer.writerow, data))

If you know up front that you won't ever be collecting the output of the mapped function then you could simplify that to just:

for i in what: pass

But neither approach keeps excess data around unless you supply a list to put it in. And this approach should work equally well with map, filter, reduce, generators, range, and anything else that you could pass in to the rake function that for can iterate over.

user1637146
  • 1