29

In Python, suppose I have a function f that I want to pass around with some secondary arguments (assume for simplicity that it's just the first argument that remains variable).

What are the differences between doing it these two ways (if any)?

# Assume secondary_args and secondary_kwargs have been defined

import functools

g1 = functools.partial(f, *secondary_args, **secondary_kwargs)
g2 = lambda x: f(x, *secondary_args, **secondary_kwargs)

In the doc page for partial, for example, there is this quote:

partial objects defined in classes behave like static methods and do not transform into bound methods during instance attribute look-up.

Will the lambda-method suffer from this if used to make a class method from arguments supplied to the class (either in the constructor or through a function later on)?

Ashwini Chaudhary
  • 244,495
  • 58
  • 464
  • 504
ely
  • 74,674
  • 34
  • 147
  • 228
  • 1
    I think my question is quite different from the linked one, however the top answer to that question is so thorough that it also answers (by coincidence) much of what I was looking for. I won't complain if it's closed as a duplicate. – ely Aug 06 '12 at 12:43
  • I agree it is not a duplicate, I have re-opened your question. For others reference, the related question is [here](http://stackoverflow.com/questions/220658/what-is-the-difference-between-a-closure-and-a-lambda). – wim Jul 30 '14 at 11:46
  • Perhaps this question should be merged with [functional programming - Python: Why is functools.partial necessary? - Stack Overflow](https://stackoverflow.com/questions/3252228/python-why-is-functools-partial-necessary) – user202729 Aug 15 '21 at 06:40

6 Answers6

17
  1. A lambda function has the same type as a standard function, so it will behave like an instance method.

  2. The partial object in your example can be called like this:

    g1(x, y, z)
    

    leading to this call (not valid Python syntax, but you get the idea):

    f(*secondary_args, x, y, z, **secondary_kwargs)
    

    The lambda only accepts a single argument and uses a different argument order. (Of course both of these differences can be overcome – I'm just answering what the differences between the two versions you gave are.)

  3. Execution of the partial object is slightly faster than execution of the equivalent lambda.

Lauritz V. Thaulow
  • 49,139
  • 12
  • 73
  • 92
Sven Marnach
  • 574,206
  • 118
  • 941
  • 841
  • 2
    Have you profiled the `partial` object in comparison to `lambda`? – orlp Aug 06 '12 at 12:42
  • 5
    @nightcracker: Yes, not now, but several times in the past. The `partial` object does not create a Python code object, so it saves a full Python stack frame. – Sven Marnach Aug 06 '12 at 12:44
  • @Sven Marnach: and what exactly is faster, the creation code, the calling code or both? – orlp Aug 06 '12 at 12:46
  • 5
    @nightcracker: Most of the time, only the execution times matter, so this is what I measured. – Sven Marnach Aug 06 '12 at 12:48
14

Summary

The practical differences between lambda and functools.partial in the common use cases seems to be

  • functools.partial needs an import, lambda does not.
  • The function definition for functions created with functools.partial is visible just by printing the created function. The functions created with lambda should be inspected with inspect.getsource().

These were found to be practically identical for lambda and functools.partial

  • Speed
  • Tracebacks

Speed (lambda vs functools.partial)

I think that tests and real data speaks louder than just guesses about which one is faster than the another.

Looks like that there is no statistical proof for speed difference between lambda and functools.partial. I ran different tests with different amount of repetitions, getting slightly different results each time; any of the three approaches could be the fastest. The speeds were identical with 95% (2 sigma) confidence. Here are some numerical results*

# When functions are defined beforehand
In [1]: timeit -n 1000 -r 1000 f_partial(data)
23.6 µs ± 2.92 µs per loop (mean ± std. dev. of 1000 runs, 1000 loops each)

In [2]: timeit -n 1000 -r 1000 f_lambda(data)
22.6 µs ± 2.6 µs per loop (mean ± std. dev. of 1000 runs, 1000 loops each)

# When function is defined each time again
In [3]: timeit -n 1000 -r 1000 (lambda x: trim_mean(x, 0.1))(data)
22.6 µs ± 1.98 µs per loop (mean ± std. dev. of 1000 runs, 1000 loops each)

In [4]: timeit -n 1000 -r 1000 f_lambda = lambda x: trim_mean(x, 0.1); f_lambda(data)
23.7 µs ± 3.89 µs per loop (mean ± std. dev. of 1000 runs, 1000 loops each)

In [5]: timeit -n 1000 -r 1000 f_partial = partial(trim_mean, proportiontocut=0.1); f_partial(data)
24 µs ± 3.38 µs per loop (mean ± std. dev. of 1000 runs, 1000 loops each)

Tracebacks

I also tried running the f_lambda and f_partial using list with string element inserted, and the tracebacks were equal (except for the very first entry, of course). So there is no difference there.

Inspecting the source code

  • The function definition for functions created with functools.partial is visible just by printing the created function. The functions created with lambda should be inspected with inspect.getsource().
# Can be inspected with just printing the function
In [1]: f_partial
Out[1]: functools.partial(<function trim_mean at 0x000001463262D0D0>, proportiontocut=0.1)

In [2]: print(f_partial)
functools.partial(<function trim_mean at 0x000001463262D0D0>, proportiontocut=0.1)

# Lambda functions do not show the source directly
In [3]: f_lambda
Out[3]: <function __main__.<lambda>(x)>

# But you can use inspect.getsource()
In [4]: inspect.getsource(f_lambda)
Out[4]: 'f_lambda = lambda x: trim_mean(x, 0.1)\n'

# This throws a ValueError, though.
In [5]: inspect.getsource(f_partial)

Appendix

* Setup used in the tests

from functools import partial
from scipy.stats import trim_mean
import numpy as np
data = np.hstack((np.random.random(1000), np.random.random(50)*25000))

f_lambda = lambda x: trim_mean(x, 0.1)
f_partial = partial(trim_mean, proportiontocut=0.1)

The tests were performed on Python 3.7.3 64-bit (Windows 10).

Niko Föhr
  • 28,336
  • 10
  • 93
  • 96
  • 1
    "`lambda` functions only accept one positional argument." Only if defined that way. They can be given any signature (but no annotations). – Kyuuhachi Aug 25 '20 at 14:13
9

The most important point here is missed - lambda have links to input vars, but partition make a copy of args during creation:

>>> for k,v in {"1": "2", "3": "4"}.items():
...     funcs.append(lambda: print(f'{k}: {v}'))
...
>>> print(funcs)
[<function <lambda> at 0x106db71c0>, <function <lambda> at 0x10747a3b0>]
>>> for f in funcs:
...     f()
...
3: 4  # result are indentical
3: 4
>>> import functools
>>> funcs = []
>>> for k,v in {"1": "2", "3": "4"}.items():
...     funcs.append(functools.partial(print, f'{k}: {v}'))
...
>>> print(funcs)
[functools.partial(<built-in function print>, '1: 2'), functools.partial(<built-in function print>, '3: 4')]
>>>
>>> for f in funcs:
...     f()
...
1: 2  # result differs
3: 4
mrvol
  • 2,575
  • 18
  • 21
  • 1
    This is not really about `lambda` vs `partial` though. This is just that an argument must be evaluated at call time in the case of `partial` (which is a function that returns a function), so the resolution through scope closure occurs at that time (as it would with any function call). `lambda` is a literal value (that happens to be a function) and does not induce any evaluation (to trigger finding values by scope closure) at the time it is defined. – ely Oct 07 '22 at 17:41
  • 1
    In other words, the point you demonstrate is very valid, but it about argument binding induced by a function call vs. a value assignment. Demonstrating it with `lambda` and `partial` is just one example, but any other example of alternatives differing by value assignment vs. function evaluation would show the same effect. – ely Oct 07 '22 at 17:42
  • 3
    For example, consider if you made a helper function for the lambda case, `def mk_lambda(k, v): return lambda: print(f'{k}: {v}')` and then did `funcs.append(mk_lambda(k, v))`. The induction of the function call `mk_lambda` will resolve the arg values at call time, so the lambda will retain the correct args in `funcs` later. It's not a property of lambdas, just of Python's choices for when scope closures are resolved. – ely Oct 07 '22 at 17:54
  • 2
    The explanation here might be less than perfect from a language-lawyer perspective, as other comments indicate --- but it did immediately "click" for me at least. Thank you @mrvol! – ev-br Jan 09 '23 at 16:34
1

partials are not only about 20% faster than equivalent lambdas as already said but they keep a direct ref to they function the relate to. While in lambdas that function is 'buried' within the function body.

=> If you need to only solve the problem of deferring evaluation of one function until all args are known then use partials. You'll have way better introspection methods compared to bury the calls into anonymous functions, i.e. lambdas.

martineau
  • 119,623
  • 25
  • 170
  • 301
Red Pill
  • 511
  • 6
  • 15
0

I believe that the class method thing only applies to functions assigned during class definition. Functions assigned later are not treated specially.

Other than that, I'd personally favor lambdas since they're more common and hence make the code easier to understand.

class Foo(object):
    def __init__(self, base):
        self.int = lambda x:int(x, base)

print Foo(4).int('11')
Antimony
  • 37,781
  • 10
  • 100
  • 107
  • 6
    `I'd personally favor lambdas since they're more common and hence make the code easier to understand.` I've never heard either of these statements before. – phant0m Aug 06 '12 at 12:45
  • @phant0m Most statements I've heard about Python lambdas had the word 'broken' in them. ;-) – Chris Wesseling Nov 21 '13 at 15:44
  • @Chris functools.partial is cleaner, but I think lambdas are easier to understand as long as you don't have scoping issues. For one thing, lambdas mean you don't have to remember the precise argument order that functools.partial takes. – Antimony Nov 21 '13 at 18:11
0

Yes, lambda will "suffer" from this. partial doesn't have this problem because it is an object with the call operator overloaded, rather than a real function.

But using a lambda like this in a class definition is just misuse.

orlp
  • 112,504
  • 36
  • 218
  • 315