Is there any way to "call a function" in python without incurring the usual performance hit?

Question

Captain Hindsight, reporting in:

After reading through the comment and answer and running a few tests, I found out that I had made a subtle error in my calculations. Turns out, I was comparing compiled lookups to interpreted calls. When I precompiled the call using the NON-IPython line magic version ( ie: timeit.timeit(codestr, setup_codestr), I found that the function calls were indeed on the same order of magnitude as the lookups :)

Now there's a whole world of caching function results, precompiling functions, and precompiling types to explore! ..and that's nice :)

For posterity:

I realize that sounds like a strange question, but someone might know a way around this, and that would be great. So here goes:

If I do something like:

%%timeit somelist[42]

Then I get times in the 90 nanosecond range. A slice will get it up to 190ish; and, to my pleasant surprise, even big crazy ones were still fast. This bad boy, for instance, weighs in at 385 nanseconds:

%%timeit some_nested_list[2:5][1][6:13]

Here's the thing. Function calls, it seems, are a lot slower than that. I like decomposing problems functionally, and am starting to give functional programming a bit more thought, but the speed difference is significant and (3.34 microseconds vs 100-150 nanoseconds (realistic actual avgs of conditionals, etc)). The following takes 3.34 micros:

def func():
    some_nested_list[2:5][1][6:13]
%%timeit func()

So, there's presumably a lot of functional programmers out there? You all must have dealt with this little hiccup? Someone care to point me in the right direction?

Answer 1

Not really. Python function calls involve a certain amount of overhead for setting up the stack frame, etc., and you can't eliminate that overhead while still writing a Python function. The reason the operations in your example are fast is that you're doing them on a list, and lists are written in C.

One thing to keep in mind is that, in many practical situations, the function call overhead will be small relative to what the function actually does. See this question for some discussion. However, if you move toward a pure-functional style in which each function just evaluates one expression, you may indeed suffer a performance penalty.

An alternative is to look at PyPy, which makes many pure-Python operations faster. I don't know whether it improves function call speed specifically. Also, by using PyPy you restrict the set of libraries you can use.

Finally, there is Cython, which allows you to write code in a language that looks basically the same as Python, but actually compiles to C. This can be much faster than Python in some cases.

The bottom line is that how to speed up your functions depends on what your functions actually do. There is no magic way to just magically make all function calls faster while still keeping everything else about Python the same. If there were, they probably would have already added it to Python.