Injecting variables into an import namespace

Question

To illustrate what I am trying to do, let's say I have a module testmod that lives in ./testmod.py. The entire contents of this module is

x = test

I would like to be able to successfully import this module into Python, using any of the tools available in importlib or any other built in library.

Obviously doing a simple import testmod statement from the current directory results in an error: NameError: name 'test' is not defined.

I thought that maybe passing either globals or locals to __import__ correctly would modify the environment inside the script being run, but it does not:

>>> testmod = __import__('testmod', globals={'test': 'globals'}, locals={'test': 'locals'})
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/jfoxrabi/testmod.py", line 1, in <module>
    x = test
NameError: name 'test' is not defined

I was setting the value of test differently so I could see which dict testmod.x came from if this worked.

Since neither of these seems to work, I am stuck. Is it even possible to accomplish what I am trying to do? I would guess that yes, since this is Python, not Sparta.

I am using Python 3.5 on Anaconda. I would very much prefer not to use external libraries.

Update: The Why

I am importing a module into my program as a configuration file. The reason that I am not using JSON or INI is that I would like to have the full scope of Python's interpreter available to compute the values in the config from expressions. I would like to have certain values that I compute before-hand in the program available to do those calculations.

While I am aware of the fact that this is about as bad as calling eval (I do that too in my program), I am not concerned with the security aspect for the time being. I am, however, quite willing to entertain better solutions should this indeed turn out to be a case of XY.

What about a simple `from module.testmod import x` after wrapping `textmod` into a own module by using an empty `__init__.py` file? — albert, Jul 28 '16 at 21:59
What are you trying to achieve -- i.e. why do you need this? Just trying to clear up [XY](https://meta.stackexchange.com/questions/66377/what-is-the-xy-problem) issues. — tzaman, Jul 28 '16 at 22:00
That sounds like you should define a function in the file you're importing and pass it a dict or something full of the information it needs. — user2357112, Jul 29 '16 at 02:37

Mad Physicist · Accepted Answer · 2022-01-29T09:33:32.463

I came up with a solution based on this answer and the importlib docs. Basically, I have access to the module object before it is loaded by using the correct sequence of calls to importlib:

from importlib.util import spec_from_file_location, module_from_spec
from os.path import splitext, basename

def loadConfig(fileName):
    test = 'This is a test'
    name = splitext(basename(fileName))[0]
    spec = spec_from_file_location(name, fileName)
    config = module_from_spec(spec)
    config.test = test
    spec.loader.exec_module(config)
    return config

testmod = loadConfig('./testmod.py')

This is a bit better than modifying builtins, which may have unintended consequences in other parts of the program, and may also restrict the names I can pass in to the module.

I decided to put all the configuration items into a single field accessible at load time, which I named config. This allows me to do the following in testmod:

if 'test' in config:
    x = config['test']

The loader now looks like this:

from importlib.util import spec_from_file_location, module_from_spec
from os.path import splitext, basename

def loadConfig(fileName, **kwargs):
    name = splitext(basename(fileName))[0]
    spec = spec_from_file_location(name, fileName)
    config = module_from_spec(spec)
    config.config = kwargs
    spec.loader.exec_module(config)
    return config

testmod = loadConfig('./testmod.py', test='This is a test')

After finding myself using this a bunch of times, I finally ended up adding this functionality to the utility library I maintain, haggis. haggis.load.load_module loads a text file as a module with injection, while haggis.load.module_as_dict does a more advanced version of the same that loads it as a potentially nested configuration file into a dict.

score 3 · Answer 2 · answered Jul 28 '16 at 21:57

3

You could screw with Python's builtins to inject your own fake built-in test variable:

import builtins    # __builtin__, no s, in Python 2

builtins.test = 5  # or whatever other placeholder value

import testmod

del builtins.test  # clean up after ourselves

answered Jul 28 '16 at 21:57

user2357112

260,549
28
431
505

Clever, but I was hoping for something less hacky. For example, something that would let me define names that are already in the builtins module without clobbering them permanently or having to make extensive backups. If nothing else crops up in a few days, I will of course accept your answer. – Mad Physicist Jul 29 '16 at 02:37
@MadPhysicist: Well, the variable injection thing you're trying to do is hacky. It's not really a surprise that the way to do it is hacky too. – user2357112 Jul 29 '16 at 02:39
That being said, I believe I found a better solution. I would welcome critiques. – Mad Physicist Jul 29 '16 at 05:12

Injecting variables into an import namespace

2 Answers2

Linked

Related