Module namespace initialisation before execution

Question

I'm trying to dynamically update code during runtime by reloading modules using importlib.reload. However, I need a specific module variable to be set before the module's code is executed. I could easily set it as an attribute after reloading but each module would have already executed its code (e.g., defined its default arguments).

A simple example:

# module.py

def do():
  try:
    print(a)
  except NameError:
    print('failed')

# main.py

import module

module.do() # prints failed

module.a = 'succeeded'

module.do() # prints succeeded

The desired pseudocode:

import_module_without_executing_code module

module.initialise(a = 'succeeded')

module.do()

Is there a way to control module namespace initialisation (like with classes using metaclasses)?

Can you give any code examples for context? It's tough to assess the sort of situation you're describing without seeing it — Ken Bellows, Oct 19 '17 at 16:42
That example doesn't suffer from `a`'s assignment being delayed; you just shouldn't call `do()` before setting it. — Davis Herring, Oct 20 '17 at 15:29
I cleaned up the terminology, integrated your example from your comment, and gave an example of immediate module execution for anyone used to assuming that modules don't do anything at `import` time. — Davis Herring, Oct 20 '17 at 15:35

score 1 · Accepted Answer · answered Oct 20 '17 at 15:44

1

It's not usually a good idea to use reload other than for interactive debugging. For example, it can easily create situations where two objects of type module.A are not the same type.

What you want is execfile. Pass a globals dictionary (you don't need an explicit locals dictionary) to keep each execution isolated; anything you store in it ahead of time acts exactly like the "pre-set" variables you want. If you do want to have a "real" module interface change, you can have a wrapper module that calls (or just holds as an attribute) the most recently loaded function from your changing file.

Of course, since you're using Python 3, you'll have to use one of the replacements for execfile.

answered Oct 20 '17 at 15:44

Davis Herring

36,443
4
48
76

Thing is, `reload` is exactly what I want because I want the aforementioned objects to be different. I want new code to be executed and completely new object created. – Exa Oct 21 '17 at 18:53
@Exa: You can still do that via the wrapper approach or by installing function objects (but note that they won’t look up global names in the host module). This is just a way to control what you replace. – Davis Herring Oct 21 '17 at 19:24
After a bunch of snooping around with importlib, I came to the conclusion that this is the best way I could approach this even if it's not the most convenient since it does not account for package modules. – Exa Oct 23 '17 at 17:25
@Exa: You can provide `__package__` in the `exec` dictionary to enable relative imports. – Davis Herring Oct 24 '17 at 01:41

score 0 · Answer 2 · answered Oct 20 '17 at 16:16

0

Strictly speaking, I don't believe there is a way to do what you're describing in Python natively. However, assuming you own the module you're trying to import, a common approach with Python modules that need some initializing input is to use an init function.

If all you need is some internal variables to be set, like a in you example above, that's easy: just declare some module-global variables and set them in your init function:

Demo: https://repl.it/MyK0

Module:

## mymodule.py

a = None

def do():
  print(a)


def init(_a):
  global a
  a = _a

Main:

## main.py

import mymodule

mymodule.init(123)
mymodule.do()

mymodule.init('foo')
mymodule.do()

Output:

123
foo

Where things can get trickier is if you need to actually redefine some functions because some dynamic internal something is dependent on the input you give. Here's one solution, borrowed from https://stackoverflow.com/a/1676860. Basically, the idea is to grab a reference to the current module by using the magic variable __name__ to index into the system module dictionary, sys.modules, and then define or overwrite the functions that need it. We can define the functions locally as inner functions, then add them to the module:

Demo: https://repl.it/MyHT/2

Module:

## mymodule.py

import sys

def init(a):
  current_module = sys.modules[__name__]
  def _do():
    try:
      print(a)
    except NameError:
      print('failed')
  current_module.do = _do

answered Oct 20 '17 at 16:16

Ken Bellows

6,711
13
50
78

I do not own the module that I'm reloading, so expecting it to have a custom initialising method is a nono. It would be convenient if I could directly overwrite the module class's init though. – Exa Oct 21 '17 at 18:55
@Exa: You don’t own the module, but it changes during execution? – Davis Herring Oct 21 '17 at 19:26
@DavisHerring Indeed. I am building an API wrapper whereby the developer can pass the aforementioned module so it gets reloaded / updated so as to not restart the whole application. ( I know it sound vague but I'll spare you the details ) – Exa Oct 22 '17 at 22:17
@Exa: Then you can surely require that the module developer _not_ do bad things at import time and provide an initialization function, right? – Davis Herring Oct 23 '17 at 00:04
1

You can just use `globals()` rather than `sys.modules`. – Davis Herring Oct 23 '17 at 00:10
@DavisHerring I cannot expect any kind of compliance from the developer when it comes to development practices that I consider "bad" nor do I have the liberty of asking them to "finish" the wrapper so as to become functional. Edit: globals() refers to the global namespace of the module, right? What I want is to embed a single object external to the module's namespace. – Exa Oct 23 '17 at 12:41
@Exa: The `globals()` is for the wrapper module as demonstrated in this answer, not the code being (re)loaded. – Davis Herring Oct 24 '17 at 01:07

Module namespace initialisation before execution

2 Answers2