Pass a variable number of arguments to the map function

Question

Suppose I have a function whose the prototype is:

def my_func(fixed_param, *args)

I would like to run this function with multiple arguments (not necessary the same number of arguments for each run) such as:

res = map(partial(my_func, fixed_param=3), [[1, 2, 3], [1, 2, 3, 4]])

where [1, 2, 3] and [1, 2, 3, 4] respectively correspond to the first and second set of parameters for args.

But this line of code fails with the following error:

TypeError: my_func() got multiple values for keyword argument 'fixed_param'

Does [this](http://stackoverflow.com/q/24755463/5018771) help you ? — AdrienW, Jul 27 '16 at 11:06

Eli Korvigo · Answer 1 · 2018-06-07T20:30:12.533

I would not be so sure about map vs list comprehension performance, as oftentimes it's a question of the function you use. Anyway, your options are:

map(lambda x: my_func(3, *x), ...)

Or

from itertools import starmap

starmap(partial(my_func, 3), ...)

Like all function in itertools, starmap returns an iterator, hence you must pass it to the list constructor, if you want a list. That will be slower than a listcomp for sure.

EDIT Benchmarks:

In [1]: def my_func(x, *args):
   ...:     return (x, ) + args
   ...: 

In [2]: from functools import partial

In [3]: from itertools import starmap

In [4]: import random

In [5]: samples = [range(random.choice(range(10))) for _ in range(100)]

In [6]: %timeit map(lambda x: my_func(3, *x), samples)
10000 loops, best of 3: 39.2 µs per loop

In [7]: %timeit list(starmap(partial(my_func, 3), samples))
10000 loops, best of 3: 33.2 µs per loop

In [8]: %timeit [my_func(3, *s) for s in samples]
10000 loops, best of 3: 32.8 µs per loop

For the sake of comparison, lets change the function a bit

In [9]: def my_func(x, args):
       ...:     return (x, )  + tuple(args)
       ...: 

In [10]: %timeit [my_func(3, s) for s in samples]
10000 loops, best of 3: 37.6 µs per loop

In [11]: %timeit map(partial(my_func, 3), samples)
10000 loops, best of 3: 42.1 µs per loop

Once again, list comprehension is faster.

score 4 · Answer 2 · answered Jul 27 '16 at 11:08

You can unpack the argument list if you use a list comprehension:

res= [my_func(3, *args) for args in [[1, 2, 3], [1, 2, 3, 4]]]

But you can't do this with map. If you insist on using map, you'll need a helper function:

def unpack(args):
    return my_func(3, *args)
res= map(unpack, [[1, 2, 3], [1, 2, 3, 4]])

score 3 · Answer 3 · answered Jul 27 '16 at 11:22

3

I think your code will work with a few small changes:

def f(l, all_args):
    print "Variable", all_args
    print "Fixed", l

map(partial(f, 1), [[1, 2, 3], [1, 2, 3, 4]])

>>> Variable [1, 2, 3]
>>> Fixed 1
>>> Variable [1, 2, 3, 4]
>>> Fixed 1

answered Jul 27 '16 at 11:22

kezzos

3,023
3
20
37

Why does the fact to pass '1' instead of 'l=1' in the partial function work? – floflo29 Jul 27 '16 at 12:13
@floflo29 because `partial` matches position arguments, you should only pass keyword arguments if you want to override this behaviour. – Eli Korvigo Jul 27 '16 at 15:50
@EliKorvigo ok but writing 'l=1' does not thus override it? – floflo29 Jul 27 '16 at 17:02
@floflo29 in this case it makes no difference, because `l` is the first argument. – Eli Korvigo Jul 27 '16 at 20:05

Pass a variable number of arguments to the map function

3 Answers3