0

The library is libphonenumbers' python port

It's only going to be used rarely

I normally import libraries in a common place for every request to use, however, would importing this library impair the performance of the app, cause instances to start slower, this is my concern.

So would it be logical to import it inline in the function that will use it?

I'm guessing one way or another, it will load into memory and increase the memory usage, however, importing it inline at the time of use might increase the instance cold-start performance

Would this be the case?

Kaan Soral
  • 1,589
  • 1
  • 15
  • 31
  • Instead of trying to guess, why not time it and see how long it takes to import? – abarnert Aug 12 '14 at 05:18
  • Good idea, however there is also the issue of memory usage, I don't know how imports work on appengine, whether the dynamically imported library will be released after usage etc, if it's not going to be, importing it for all requests might be simpler – Kaan Soral Aug 12 '14 at 05:24
  • 1
    Python never releases modules until shutdown time. But if GAE is starting and stopping Python processes regularly, that might make a difference. Also, it's possible to encourage Python to release a module early (`del` the module from your globals, and from `sys.modules`, and make sure to release any other references to any objects from the module, and it _might_ go away), but this usually isn't a good idea. – abarnert Aug 12 '14 at 05:35

1 Answers1

1

The actual time to import a library is roughly the same regardless of doing it at instance startup time or on-demand, inlined inside a particular request needing it. Similarly, the memory footprint of the library is the same once loaded.

So, for the request actually needing the library - it doesn't really matter.

But for the other requests, it may matter, possibly a lot.

Blindly loading all possibly needed libraries at instance startup time means the instance will start slower and will use more memory than if loading only the libraries necessary for the request that triggered the instance starting. But once the instance is started all subsequent requests will benefit from the minimum response time - no additional code needs to be loaded, while in the other case the response time for some subsequent requests will be increased by the time required to import the libraries not yet imported (once per library per instance lifetime).

Some will consider apps importing all libraries at startup more costly, slower to start and bulkier than their on-demand importing counterparts. Others will see them as more deterministic in performance, which may matter for some apps.

On-demand importing libraries, especially rarely-used hefty ones, offers a unique advantage: the app can (at least partially or in degraded mode) run even if the total memory footprint of all libs exceeds the memory limit of the instance type - which would be impossible if attempting to load all the libs at instance startup time.

A variation of on-demand importing is lazy-loading of entire files containing request handler code, see App Engine: Few big scripts or many small ones? An actual example is here: What determines start up time of dynamic instance and can it vary between weeks if code is same)

Community
  • 1
  • 1
Dan Cornilescu
  • 39,470
  • 12
  • 57
  • 97