4

I am looking for a way to partially apply functions in python that is simple to understand, readable, resusable and as little error prone to coder mistakes as possible. Most of all I want the style to be as performant as possible - less frames on the stack is nice, and less memory footprint for the partially applied functions is also desirable. I have considered 4 styles and written examples below:

import functools

def multiplier(m):
    def inner(x):
        return m * x

    return inner

def divide(n,d):
    return n/d

def divider(d):
    return functools.partial(divide,d=d)

times2 = multiplier(2)
print(times2(3))  # 6

by2 = divider(2)
print(by2(6)) # 3.0

by3 = functools.partial(divide,d=3)
print(by3(9)) # 3.0

by4 = lambda n: divide(n,4)
print(by4(12)) # 3.0

My analysis of them are:

times2 is a nested thing. I guess python makes a closure with the m bound, and everything is nice. The code is readable (I think) and simple to understand. No external libraries. This is the style I use today.

by2 has an explicit named function which makes it simple for the user. It uses functools, so it gives you extra imports. I like this style to some extent since it is transparent, and give me the option to use divide in other ways if I want to. Contrast this with inner which is not reachable.

by3 is like by2, but forces the reader of the code to be comfortable with functools.partial since they have it right in the face. what I like less is that PyCharm cant give my tooltips with what the arguments to functools.partial should be, since they are effectively arguments to by3. I have to know the signature of divide myself everytime I define some new partial application.

by4 is simple to type, since I can get autocompletion. It needs no import of functools. I think it looks non-pythonic though. Also, I always feel uncomfortable about scoping of variables / closures with lambdas work in python. Never sure how that behaves....

What is the logical difference between the styles and how does that affect memory and CPU?

LudvigH
  • 3,662
  • 5
  • 31
  • 49
  • 1
    This is fairly broad. I don't think there is any consensus about this, so use what feels right. In terms of functionality, they are basically equivalent. In terms of speed, you can check it yourself with `timeit` (don't think there would be a significant difference) – Yevhen Kuzmovych Sep 06 '19 at 13:29
  • @JohnColeman I understand that. I'll rewrite to narrow it down ! – LudvigH Sep 06 '19 at 13:43
  • 1
    The comparisons might be more directly meaningful if you have exactly the same function (say the one that sends x to 2x) defined in four different ways rather than 4 different functions defined in 4 different ways. – John Coleman Sep 06 '19 at 14:20
  • Might also be interesting: [Currying decorator in Python](https://stackoverflow.com/q/9458271/1639625) – tobias_k Sep 06 '19 at 15:35
  • @JohnColeman thanks for pointing out the imporance of doing that same maths in every case to making it comparable. When I started a rewrite of the question I realized your answer below actually answers my question well. Also - It gave me the tools I needed to settle the question myself on my machine and with the in-context functions. Thank you! – LudvigH Sep 06 '19 at 16:09

2 Answers2

3

The first way seems to be the most efficient. I tweaked your code so that all 4 functions compute exactly the same mathematical function:

import functools,timeit

def multiplier(m):
    def inner(x):
        return m * x

    return inner

def mult(x,m):
    return m*x

def multer(m):
    return functools.partial(mult,m=m)

f1 = multiplier(2)
f2 = multer(2)
f3 = functools.partial(mult,m=2)
f4 = lambda x: mult(x,2)

print(timeit.timeit('f1(10)',setup = 'from __main__ import f1'))
print(timeit.timeit('f2(10)',setup = 'from __main__ import f2'))
print(timeit.timeit('f3(10)',setup = 'from __main__ import f3'))
print(timeit.timeit('f4(10)',setup = 'from __main__ import f4'))

Typical output (on my machine):

0.08207898699999999
0.19439769299999998
0.20093803199999993
0.1442435820000001

The two functools.partial approaches are identical (since one of them is just a wrapper for the other), the first is twice as fast, and the last is somewhere in between (but closer to the first). There is a clear overhead in using functools over a straightforward closure. Since the closure approach is arguably more readable as well (and more flexible than the lambda which doesn't extend well to more complicated functions) I would just go with it.

John Coleman
  • 51,337
  • 7
  • 54
  • 119
3

Technically you're missing one other option as operator.mul does the same thing you're looking to do and you can just use functools.partial on it to get a default first argument without having to reinvent the wheel.

Not only is it the fastest option it also uses the least space compared to a custom function or a lambda statement. The fact that it's a partial is why it uses the same space as the others and I think that's the best route here.

from timeit import timeit
from functools import partial
from sys import getsizeof
from operator import mul

def multiplier(m):
    def inner(x):
        return m * x
    return inner

def mult(x,m):
    return m*x

def multer(m):
    return partial(mult,m=m)

f1 = multiplier(2)
f2 = multer(2)
f3 = partial(mult,m=2)
f4 = lambda n: mult(n,2)
f5 = partial(mul, 2)
from_main = 'from __main__ import {}'.format

print(timeit('f1(10)', from_main('f1')), getsizeof(f1))
print(timeit('f2(10)', from_main('f2')), getsizeof(f2))
print(timeit('f3(10)', from_main('f3')), getsizeof(f3))
print(timeit('f4(10)', from_main('f4')), getsizeof(f4))
print(timeit('f5(10)', from_main('f5')), getsizeof(f5))

Output

0.5278953390006791 144
1.0804575479996856 96
1.0762036349988193 96
0.9348237040030654 144
0.3904160970050725 96

This should answer your question as far as memory usage and speed.

Jab
  • 26,853
  • 21
  • 75
  • 114
  • 1
    Not sure I understand your answer. You are saying that you can use `operator.mul`, which is somewhat besides the point, as the multiplication-function is probably just a simple example for testing and not what OP really would be using. Then you say that for `operator.mul`, `partial` is best, and thus `partial` _is_ best (in all cases?) even though it is slower than a `def` function? – tobias_k Sep 06 '19 at 15:49
  • Where did I say for all cases? In this case using `partial` and `operator.mul` is the fastest as all OP seems to be trying to do is add a default arg to another but not the most estensible as partial cannot accept positional arguments. It is although an option that can be considered. I am only stating that `operator.mul` IS another option and is faster than making your own. – Jab Sep 06 '19 at 17:36