20

In Python 2, a common (old, legacy) idiom is to use map to join iterators of uneven length using the form map(None,iter,iter,...) like so:

>>> map(None,xrange(5),xrange(10,12))
[(0, 10), (1, 11), (2, None), (3, None), (4, None)]

In Python 2, it is extended so that the longest iterator is the length of the returned list and if one is shorter than the other it is padded with None.

In Python 3, this is different. First, you cannot use None as an argument for the callable in position 1:

>>> list(map(None, range(5),range(10,12)))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: 'NoneType' object is not callable

OK -- I can fix that like so:

>>> def f(*x): return x    
... 
>>> list(map(f, *(range(5),range(10,12))))
[(0, 10), (1, 11)]

But now, I have a different problem: map returns the shortest iterator's length -- no longer padded with None.

As I port Python 2 code to Python 3, this is not a terrible rare idiom and I have not figured out an easy in place solution.

Unfortunately, the 2to3 tools does not pick this up -- unhelpfully suggesting:

-map(None,xrange(5),xrange(10,18))
+list(map(None,list(range(5)),list(range(10,18))))

Suggestions?

Edit

There is some discussion of how common this idiom is. See this SO post.

I am updating legacy code written when I was still in high school. Look at the 2003 Python tutorials being written and discussed by Raymond Hettinger with this specific behavior of map being pointed out...

Community
  • 1
  • 1
the wolf
  • 34,510
  • 13
  • 53
  • 71
  • 3
    `map(None,*(iter,iter))` is not a common idiom. Have you heard of [`itertools.izip_longest()`](http://docs.python.org/library/itertools.html#itertools.izip_longest)? Does exactly what you described, but without gimmicks. – Tadeck Aug 18 '12 at 03:17
  • 4
    @Tadeck: Since the behavior `map` described is [straight from the Python 2 documents](http://docs.python.org/library/functions.html#map), yeah -- it is not uncommon... – dawg Aug 18 '12 at 03:21
  • 3
    @drewk: I am familiar with how `map()` works, but using `map(None, *(iter, iter))` in this case seems to be very unpythonic for me. Also, I also wonder, why `map(None,*(xrange(5),xrange(10,12)))` and not more direct `map(None, xrange(5), xrange(10,12))`? Does exactly the same. Any reference that could prove it is pythonic? Guido's confirmation would be enough, or Raymond Hettinger's, or part of Python's docs would do. – Tadeck Aug 18 '12 at 03:27
  • @drewk The fact that this behavior was removed from Python 3 is pretty strong evidence that it's not very pythonic. Use `zip` or `itertools.zip_longest` instead.... – Danica Aug 18 '12 at 03:31
  • 2
    I did not state it was *Pythonic* or even good practice; I said it was common because that is the behavior that is documented in `map` in Python 2. Google 'python join list of different lengths' Some of the older answers use map. – dawg Aug 18 '12 at 03:41
  • 2
    Well, "common" and "supported" are different things; I've certainly never seen it before. In any case, this functionality has been supplanted by `zip` and `zip_longest`, which are far more understandable IMO. – Danica Aug 18 '12 at 04:01
  • 2
    @Tadeck: Raymond Hettinger writes about map(None,..) [HERE](http://mail.python.org/pipermail/python-dev/2003-August/037636.html) – dawg Aug 18 '12 at 04:17
  • Guys: I was in high school when this code was written! Does not free me from the job of translating it! – the wolf Aug 18 '12 at 04:25
  • 1
    @drewk: The message you are referring to was written in August 2003 (Python <=2.3), so 9 years ago. At that time that _could_ be the only reasonable choice. Since Python 2.6 you have `itertools.izip_longest()`, and since Python 3.0 `map()` no longer supports the behaviour OP described. These facts pretty much speak for themselves. If that code is old, that is okay. But if the code is new (written for Python 2.6+), then this is not "okay". Anyway, hope such pieces of code will die with wider adoption of Python 3 from natural cases. – Tadeck Aug 18 '12 at 05:44
  • @drewk And then Neil Schemenauer replied: "I don't think I've ever written a `map(None, ...)` expression that wasn't better expressed with the newer `zip()` builtin." `map(None, ...)` is silly and archaic and it was removed in 3 for a good reason. :) – Danica Aug 18 '12 at 06:24
  • 2
    @Dougal: Folks -- seriously -- I don't think anyone is proposing using this construct in new code. My question was how to efficiently update OLD code. Why the bickering? – the wolf Aug 18 '12 at 06:37

4 Answers4

19

itertools.zip_longest does what you want, with a more comprehensible name. :)

Danica
  • 28,423
  • 6
  • 90
  • 122
  • 3
    Exactly, and in Python 2.x it is called [`itertools.izip_longest()`](http://docs.python.org/library/itertools.html#itertools.izip_longest), so there is no need to do `map(None, *(iter, iter))`. +1 – Tadeck Aug 18 '12 at 03:20
  • you'll still need to stick a `list()` around it, though. – andrew cooke Aug 18 '12 at 04:15
  • 4
    @andrewcooke Well, depending on if you actually need a list or just want to iterate over it.... – Danica Aug 18 '12 at 06:22
2

I'll answer my own question this time.

With Python 3x, you can use itertools.zip_longest like so:

>>> list(map(lambda *a: a,*zip(*itertools.zip_longest(range(5),range(10,17)))))
[(0, 10), (1, 11), (2, 12), (3, 13), (4, 14), (None, 15), (None, 16)]

You can also roll ur own I suppose:

>>> def oldMapNone(*ells):
...     '''replace for map(None, ....), invalid in 3.0 :-( '''
...     lgst = max([len(e) for e in ells])
...     return list(zip(* [list(e) + [None] * (lgst - len(e)) for e in ells]))
... 
>>> oldMapNone(range(5),range(10,12),range(30,38))
[(0, 10, 30), (1, 11, 31), (2, None, 32), (3, None, 33), (4, None, 34), (None, None, 35), (None, None, 36), (None, None, 37)]
the wolf
  • 34,510
  • 13
  • 53
  • 71
  • 6
    I do not understand the complexity of your answer. Will the following just work: list(itertools.zip_longest(range(5),range(10,17))) – Yongwei Wu Sep 29 '16 at 10:03
1

One way if you need some obsolete functionality in from the Python 2 then one way - write it yourself. Of course, it's not built-in functionality, but at least it is something.

The code snippet below takes 27 lines

#!/usr/bin/env python3

def fetch(sequence, index):
  return None if len(sequence) <= index else sequence[index]

def mymap(f, *args):
  max_len = 0
  for i in range(len(args)): 
      max_len = max(max_len, len(args[i]))
  out = []
  for i in range(max_len):
      t = []
      for j in range(len(args)): 
          t.append(fetch(args[j],i))      

      if f != None:
          # Use * for unpack arguments from Arbitarily argument list
          # Use ** for unpack arguments from Keyword argument list
          out.append(f(*t))
      else:
          out.append(tuple(t))
  return out 

if __name__ == '__main__':
    print(mymap(None, [1,2,3,4,5],[2,1,3,4],[3,4]))
    print(mymap(None,range(5),range(10,12)))
Konstantin Burlachenko
  • 5,233
  • 2
  • 41
  • 40
-1

you can solve the problem like this: list(map(lambda x, y: (x, y),[1, 2, 3 ,4, 5], [6, 7, 8, 9, 10]))

user9018699
  • 11