8

Why does map when called with an object which can be iterated over multiple times not return an object which can also be iterated multiple times? I think the latter is much more reasonable.

My use case is that I have a lot of data, such that it can only be iterated over. map is (in theory) perfect for operations on data, since it is lazy. However in the following example I would expect that the length is both times the same.

iterable = [1,2,3,4]  # this can be iterated repeatedly
m = map(lambda x:x**2, iterable) # this again should be iterable repeatedly
print(len(list(m))) # 4
print(len(list(m))) # 0

How can I map over an iterable structure and get an iterable structure back?

Edit: This is an example of how it imho should work, demonstrating the lazy evaluation:

def g(): 
    print('g() called')

data = [g, g]

# map is lazy, so nothing is called
m = map(lambda g: g(), data)
print('m: %s' % len(list(m))) # g() is called here
print('m: %s' % len(list(m))) # this should work, but doesnt

# this imap returns an iterable
class imap(object):
    def __init__(self, fnc, iterable):
        self.fnc = fnc
        self.iterable = iterable
    def __iter__(self):
        return map(self.fnc, self.iterable)

# imap is lazy, so nothing is called
im = imap(lambda g: g(), data)    
print('im: %s' % len(list(im))) # g() is called here
print('im: %s' % len(list(im))) # works as expected
wim
  • 338,267
  • 99
  • 616
  • 750
Manuel Schmidt
  • 2,429
  • 1
  • 19
  • 32
  • If you need to use `len`, then why not just use a list comprehension to begin with? It's just as lazy and creates a permanent result. If you need `len` but really want to use `map`, just call `len` on the input instead of the output of `map`. – Mad Physicist Mar 31 '17 at 17:52
  • If you want to be able to iterate over it repeatedly, why no *use the list then*? Or just use a list comprehension? – juanpa.arrivillaga Mar 31 '17 at 17:53
  • 1
    For example, maybe it consumes too much memory. Pretty sure `len` is just used as an example to show that it can't be consumed twice. – wim Mar 31 '17 at 17:57

1 Answers1

7

Why does map when called with an object which can be iterated over multiple times not return an object which can also be iterated multiple times?

Because there is no interface to tell whether an object can be iterated over repeatedly. map has no way to tell whether the thing it's iterating over supports repeat iteration, and unless map manages to determine this information somehow and invents an API to expose it to its users, map users would have no way to tell whether their map object supports repeat iteration.

Also, with repeat iteration comes the need to either repeat the function evaluations or cache the results (but if you're going to cache the results, why redesign map to return an iterator at all?). Repeated function evaluations are inefficient, potentially dangerous, and usually not what users want. It's better to have users explicitly repeat the map call or explicitly call list if they want to iterate again.

It's simpler if map objects are always just iterators.

user2357112
  • 260,549
  • 28
  • 431
  • 505
  • To clarify: `x is iter(x)` could be used to confirm that an iterable is *not* repeatable, but you [can't prove the converse](http://stackoverflow.com/a/40446517). – Zero Piraeus Mar 31 '17 at 17:54
  • 1
    @ZeroPiraeus: There's even a counterexample right in the question: the question's own `imap` class has `x is not iter(x)`, but only supports repeat iteration if the underlying iterable supports it. – user2357112 Mar 31 '17 at 18:15
  • The first paragraph is the answer to my question - that it is not possible in python collections. Your second point is a tradeoff every other language implementing map over iterables (Haskell, Java, Scala, Closoure, ...) has to face and solve in some way. I found the PyFunctional lib which seems to does what I need. – Manuel Schmidt Apr 01 '17 at 06:44