When a function is used in python, what objects need to be created?

Question

I read somewhere that it is bad to define functions inside of functions in python, because it makes python create a new function object very time the outer function is called. Someone basically said this:

#bad
def f():
    def h():
        return 4
return h()

#faster
def h():
    return 4

def f(h=h):
    return h()

Is this true? Also, what about a case when I have a ton of constants like this:

x = # long tuple of strings
# and several more similar tuples
# which are used to build up data structures

def f(x):
    #parse x using constants above
    return parse dictionary

Is it faster if I put all the constants inside the definition of f? Or should I leave them outside and bind them to local names in a keyword argument? I don't have any data to do timings with unfortunately, so I guess I'm asking about your experiences with similar things.

I wouldn't worry about speed and focus on what is most readable / maintainable, you can always optimize it later if the code becomes a bottleneck. — GWW, Nov 03 '13 at 16:04
As someone says every time something about timings comes up, "Premature optimization is the root of all evil"! Only worry about speed when the program is slow (although I appreciate you might still want to know the answer out of curiosity). — rlms, Nov 03 '13 at 16:06
Also if you're concerned with timings it takes to execute different versions of the same code use `timeit` module. As for making functions in a function it's sometimes useful to have a function which returns a function even. — Aleksander Lidtke, Nov 03 '13 at 16:10
please check http://stackoverflow.com/questions/4831680/function-inside-function for more detail. — Gaurav Pant, Nov 03 '13 at 16:17

Roman Bodnarchuk · Answer 1 · 2013-11-04T09:39:57.323

Short answers to you questions - it is true. Inner functions are created each time outer function is called, and this takes some time. Also access to objects, which were defined outside of a function, is slower, comparing to access to local variables.

However, you also asked more important question - "should I care?". Answer to this, almost always, NO. Performance difference will be really minor, and readability of your code is much more valuable.

So, if you think that this function belongs to body of other function and makes no sense elsewhere - just put it inside and do not care about performance (at least, until your profiler tells you otherwise).

score 0 · Answer 2 · answered Nov 03 '13 at 16:11

When a function is executed, all the code that is inside needs to be executed. So of course, very simply said, the more you put in the function, the more effort it takes Python to execute this. Especially when you have constant things that do not need to be constructed at run time of the function, then you can save quite a bit by putting it in an upper scope so that Python only needs to look it up instead of generating it again and allocating (temporary) memory to save it for the short run time of a function.

So in your case, if you have a large tuple or anything that does not depend on the input x to the function f, then yes, you should store it outside.

Now the other thing you mentioned is a scope lookup for functions or constants using a keyword argument. In general, looking up variables in an outer scope is more expensive than looking it up in the most local scope. So yes, when you define those constants on module level and you access them inside a function, the lookup will be more expensive than when the constants would be defined inside the function. However, actually defining them inside the function (with memory allocation and actual generation of the data) is likely to be more expensive, so it’s really not a good option.

Now you could pass the constants as a keyword argument to the function, so the lookup inside the function would be a local scope lookup. But very often, you don’t need those constants a lot. You maybe access it once or twice in the function and that’s absolutely not worth adding the overhead of another argument to the function and the possibility to pass something different/incompatible to it (breaking the function).

If you know that you access some global stuff multiple times, then create a local variable at the top of the function which looks that global stuff up once and then use the local variable in all further places. This also applies to member lookup, which can be expensive too.

In general though, these are all rather micro optimizations and you are unlikely to run into any problems if you do it one or the other way. So I’d suggest you to write clear code first and make sure that the rest of it works well, and if you indeed run into performance problems later, then you can check where the issues are.

score 0 · Accepted Answer · answered Dec 02 '13 at 21:50

In my tests, the fastest way to do what I needed to was to define all the constants I needed outside, then make the list of functions who need those constants outside, then pass the list of functions to the main function. I used dis.dis, cProfile.run, and timeit.timeit for my tests, but I can't find the benchmarking script and can't be bothered to rewrite it and put up the results.

When a function is used in python, what objects need to be created?

3 Answers3