262

What are the pros and cons of importing a Python module and/or function inside of a function, with respect to efficiency of speed and of memory?

Does it re-import every time the function is run, or perhaps just once at the beginning whether or not the function is run?

Michael Currie
  • 13,721
  • 9
  • 42
  • 58
Tim McJilton
  • 6,884
  • 6
  • 25
  • 27
  • 5
    There is no speed benefit (calling `import` is very expensive, even if the module is already loaded). If you want a speed benefit, it is faster (if you access the module at least 4-5 times) to just assign the module to a local variable as the first thing you do in your function, and then access it through that local variable (because local variable lookups are VERY fast). – Nick Bastin Jun 22 '10 at 16:52
  • @Nick: When listening to this, it appears that it is slow to repeatedly import since each time you are trying and checking whether or not it was imported. Are you saying that outside of the function import it and set it as a global variable and grab the global variable inside the function? – Tim McJilton Jun 22 '10 at 20:34
  • 1
    @Tim: The optimal way to speed up module access (assuming that's what you're trying to do, and you access the module enough to make local assignment worthwhile) is to `import` the module at the file level as usual, and then inside the function assign the module to a local variable. To make the assignment worthwhile, you'll need to access the module probably at least 4 times inside the function - if you use the module less frequently than that, doing the direct module.symbol lookup at the global level won't be any slower than local assignment/lookup. – Nick Bastin Jun 23 '10 at 04:34
  • @Nick: Is this any different than the optimization of just avoiding using periods? IE if you call function utility.foo() N time where N is large, that it is better to do my_foo=utility.foo() and then do my_foo() N times? Where utility could be a module or a class or whatever you would like. – Tim McJilton Jun 23 '10 at 22:54
  • @Tim: The concept is the same, although what's happening under the covers is slightly different between modules and say, an instancemethod. Still, you should really do `neither` of these things as a matter of general practice - you should import at module level and use methods/functions in the namespace they are currently in, and only in extreme performance situations should you think about micro-optimizations like reassigning to a local. – Nick Bastin Jun 24 '10 at 09:41
  • 1
    @NickBastin Is it still the case that assigning the module to a local variable is an optimization 5 and a half years later? – 2rs2ts Jan 12 '16 at 18:50
  • @2rs2ts: yes - local variables are the fastest access you can have. I still wouldn't recommend doing this unless you have a loop with an extremely large number of iterations that consumes a significant amount of time. – Nick Bastin Jan 12 '16 at 21:26
  • 4
    @NickBastin Your pycon link is broken. – Navin May 05 '18 at 17:43

6 Answers6

241

Does it re-import every time the function is run?

No; or rather, Python modules are essentially cached every time they are imported, so importing a second (or third, or fourth...) time doesn't actually force them to go through the whole import process again. 1

Does it import once at the beginning whether or not the function is run?

No, it is only imported if and when the function is executed. 2, 3

As for the benefits: it depends, I guess. If you may only run a function very rarely and don't need the module imported anywhere else, it may be beneficial to only import it in that function. Or if there is a name clash or other reason you don't want the module or symbols from the module available everywhere, you may only want to import it in a specific function. (Of course, there's always from my_module import my_function as f for those cases.)

In general practice, it's probably not that beneficial. In fact, most Python style guides encourage programmers to place all imports at the beginning of the module file.

Community
  • 1
  • 1
mipadi
  • 398,885
  • 90
  • 523
  • 479
  • 45
    In the same line of thought, this can make a depedency optional if the import is tucked inside an ancillary function. – M. Toya Mar 24 '15 at 08:12
  • 6
    I use it for optional dependencies when I write library modules for myself. I make each function inside a library module depend on a minimal number of imports. – CodeMonkey Jul 01 '16 at 09:10
  • 3
    thanks! it saved my web2py app lots of loading time by put slow module ` import plotly` inside the function where it is called. – laviex Feb 04 '19 at 05:49
  • 2
    I find that sometimes I'll need **package A** for a particular task. Later I rewrite the code and decide to use **package B** instead. At that point, I may not remember if **package A** was used elsewhere in my project and may either not remove it while it no longer needed or remove it while it is still needed (something I'll not notice right away if it's a rarely used part of the code). I wonder if there's something others do to avoid this sort of problem when you pythonically do not import inside functions? – PatrickT May 13 '20 at 21:02
  • 3
    @PatrickT If you're using an IDE, they tend to tell you if an import is not accessed. For example, in VSCode, Pylance will tell me if I do not access an import. – Kraigolas Jan 05 '22 at 05:41
59

The very first time you import goo from anywhere (inside or outside a function), goo.py (or other importable form) is loaded and sys.modules['goo'] is set to the module object thus built. Any future import within the same run of the program (again, whether inside or outside a function) just look up sys.modules['goo'] and bind it to barename goo in the appropriate scope. The dict lookup and name binding are very fast operations.

Assuming the very first import gets totally amortized over the program's run anyway, having the "appropriate scope" be module-level means each use of goo.this, goo.that, etc, is two dict lookups -- one for goo and one for the attribute name. Having it be "function level" pays one extra local-variable setting per run of the function (even faster than the dictionary lookup part!) but saves one dict lookup (exchanging it for a local-variable lookup, blazingly fast) for each goo.this (etc) access, basically halving the time such lookups take.

We're talking about a few nanoseconds one way or another, so it's hardly a worthwhile optimization. The one potentially substantial advantage of having the import within a function is when that function may well not be needed at all in a given run of the program, e.g., that function deals with errors, anomalies, and rare situations in general; if that's the case, any run that does not need the functionality will not even perform the import (and that's a saving of microseconds, not just nanoseconds), only runs that do need the functionality will pay the (modest but measurable) price.

It's still an optimization that's only worthwhile in pretty extreme situations, and there are many others I would consider before trying to squeeze out microseconds in this way.

Alex Martelli
  • 854,459
  • 170
  • 1,222
  • 1,395
  • This optimization is actually never worthwhile - no amount of fast local variable access will make up for the incredible expense of calling `import`, even when the module is already loaded. Checking to see whether a module has been loaded is a very expensive operation (relative to a few global dictionary lookups). – Nick Bastin Jun 22 '10 at 16:50
  • 6
    Importing a module for the first time is expensive. Try running an empty script vs one containing just `import string,itertools,fractions,heapq,re,array,bisect,collections,math,os`. The first takes an average of 180 ms, and the second 230 ms. So it's not microseconds for starters. It's tens of milliseconds (maybe a disc access happens?). This is significant for tiny scripts that run many times (like serving web requests, perhaps). – Evgeni Sergeev Mar 26 '16 at 03:43
  • 1
    @EvgeniSergeev In that case you usually have a server running all the time, thus it won't reimport over and over again – Tobias Kienzler Aug 24 '16 at 06:13
  • 3
    @TobiasKienzler it still applies for the FaaS (Function-as-a-Service) environment and/or for some low-performance (e.g. embedded) appliances. The argument is that the difference in latency can be significant and it can not be simply neglected. – user3576508 Nov 28 '17 at 05:29
22

It imports once when the function executes first time.

Pros:

  • imports related to the function they're used in
  • easy to move functions around the package

Cons:

  • couldn't see what modules this module might depend on
SilentGhost
  • 307,395
  • 66
  • 306
  • 293
  • 3
    if you do something like `grep import /path/to/module` it will show you all of the modules that it imports. –  Dec 23 '16 at 02:11
11

Might I suggest in general that instead of asking, "Will X improve my performance?" you use profiling to determine where your program is actually spending its time and then apply optimizations according to where you'll get the most benefit?

And then you can use profiling to assure that your optimizations have actually benefited you, too.

gomad
  • 1,029
  • 7
  • 16
  • 5
    I agree with that, this was more of a curiosity question. I was wondering how Python's import method works in more detail, more than trying to do premature performance enhancement. Thanks though :) – Tim McJilton Jun 22 '10 at 20:46
  • 3
    Ah. Well, I hope that the excellent answers here have satisfied your curiosity! Effbot has some information that might be of use to you: http://effbot.org/zone/import-confusion.htm Scroll down to "What Does Python Do to Import a Module?" – gomad Jun 22 '10 at 21:47
  • Thanks for the information, the answers have been very excellent and helped. – Tim McJilton Jun 22 '10 at 22:09
7

Importing inside a function will effectively import the module once.. the first time the function is run.

It ought to import just as fast whether you import it at the top, or when the function is run. This isn't generally a good reason to import in a def. Pros? It won't be imported if the function isn't called.. This is actually a reasonable reason if your module only requires the user to have a certain module installed if they use specific functions of yours...

If that's not he reason you're doing this, it's almost certainly a yucky idea.

royal
  • 530
  • 1
  • 6
  • 12
4

It imports once when the function is called for the first time.

I could imagine doing it this way if I had a function in an imported module that is used very seldomly and is the only one requiring the import. Looks rather far-fetched, though...

Tim Pietzcker
  • 328,213
  • 58
  • 503
  • 561