3

I am wondering if there is there is a simple Pythonic way (maybe using generators) to run a function over each item in a list and result in a list of returns?

Example:

def square_it(x):
    return x*x

x_set = [0,1,2,3,4]
squared_set = square_it(x for x in x_set)

I notice that when I do a line by line debug on this, the object that gets passed into the function is a generator.

Because of this, I get an error: TypeError: unsupported operand type(s) for *: 'generator' and 'generator'

I understand that this generator expression created a generator to be passed into the function, but I am wondering if there is a cool way to accomplish running the function multiple times only by specifying an iterable as the argument? (without modifying the function to expect an iterable).

It seems to me that this ability would be really useful to cut down on lines of code because you would not need to create a loop to fun the function and a variable to save the output in a list.

Thanks!

Ramchandra Apte
  • 4,033
  • 2
  • 24
  • 44
Pswiss87
  • 725
  • 1
  • 6
  • 16

5 Answers5

5

You want a list comprehension:

squared_set = [square_it(x) for x in x_set]
BrenBarn
  • 242,874
  • 37
  • 412
  • 384
4

There's a builtin function, map(), for this common problem.

>>> map(square_it, x_set)
[0,1,4,9,16] # On Python 3, a generator is returned.

Alternatively, one can use a generator expression, which is memory-efficient but lazy (meaning the values will not be computed now, only when needed):

>>> (square_it(x) for x in x_set)
    <generator object <genexpr> at ...>

Similarly, one can also use a list comprehension, which computes all the values upon creation, returning a list.

Additionally, here's a comparison of generator expressions and list comprehensions.

Community
  • 1
  • 1
Ramchandra Apte
  • 4,033
  • 2
  • 24
  • 44
  • Sadly, `map` is considered bad practice in modern Python code. – Matt Bryant Sep 10 '13 at 16:50
  • 1
    @MattBryant Sure, it's bad to use lambdas with map, but I think it's okay to use predefined functions with map. `map` can especially be useful when chaining many functions together, i.e. `map(squirtize, map(spoggle, list1))` – Ramchandra Apte Sep 10 '13 at 16:53
  • I agree and am a big fan of `map`. I'm just pointing out that many people don't approve of it. – Matt Bryant Sep 10 '13 at 16:55
  • 3
    The deal with "map is bad practice" is that map has worse performance than a list comprehension, *unless* the function passed is built-in (i.e. compiled). Plus Guido is uncomfortable with functional programming. – Marcin Sep 10 '13 at 16:56
  • 2
    @Marcin Well, the performance difference is typically very small, that's not an argument against `map`. If one is really squaring millions of numbers, one might as well use modules like `numpy`, which offers accelerated C methods of doing such problems. – Ramchandra Apte Sep 10 '13 at 16:58
  • 1
    @RamchandraApte Regardless of whether or not you think it's a good argument, that is the main argument against it. – Marcin Sep 10 '13 at 17:12
2

You want to call the square_it function inside the generator, not on the generator.

squared_set = (square_it(x) for x in x_set)
Matt Bryant
  • 4,841
  • 4
  • 31
  • 46
1

As the other answers have suggested, I think it is best (most "pythonic") to call your function explicitly on each element, using a list or generator comprehension.

To actually answer the question though, you can wrap your function that operates over scalers with a function that sniffs the input, and has different behavior depending on what it sees. For example:

>>> import types
>>> def scaler_over_generator(f):
...     def wrapper(x):
...         if isinstance(x, types.GeneratorType):
...             return [f(i) for i in x]
...         return f(x)
...     return wrapper
>>> def square_it(x):
...     return x * x
>>> square_it_maybe_over = scaler_over_generator(square_it)
>>> square_it_maybe_over(10)
    100
>>> square_it_maybe_over(x for x in range(5))
    [0, 1, 4, 9, 16]

I wouldn't use this idiom in my code, but it is possible to do.

You could also code it up with a decorator, like so:

>>> @scaler_over_generator
... def square_it(x):
...     return x * x
>>> square_it(x for x in range(5))
    [0, 1, 4, 9, 16]

If you didn't want/need a handle to the original function.

Matt Anderson
  • 19,311
  • 11
  • 41
  • 57
1

Note that there is a difference between list comprehension returning a list

squared_set = [square_it(x) for x in x_set]

and returning a generator that you can iterate over it:

squared_set = (square_it(x) for x in x_set)
Mauricio Abreu
  • 155
  • 1
  • 9