1

I am puzzled with this behavior where environment variable is not updated correctly for class of a module I import on second or later calls.

I have a module which have some class I initialize to get a function (and totally works fine). To control and Test some behaviors on an automated manner locally (which in prod used by Docker environment variables) have some environment variables which can be set or unset.

To test some behaviors I am trying to make a script that tests some permutations of these environment variables importing the relevant module and initializing an object, reloading the module each call.

Each object initialization shows environment variable value and also if it affected the call as in below code:

#inside MyModule.SubModule

SOME_ENV_VAR = os.getenv("SOME_ENV_VAR", "0")
# convert environment variable to integer
try:
    SOME_ENV_VAR = int(SOME_ENV_VAR)
except ValueError:
    # if non integer given set to 0
    SOME_ENV_VAR = 0
print("SOME_ENV_VAR", SOME_ENV_VAR)

class MyClass:
    def __init__(self):
        # Logic to set Cuda device depending on Environmental variable
        print("Value of SOME_ENV_VAR: ", SOME_ENV_VAR) 

import sys, importlib

def get_object(env_set=True):
    os.environ["SOME_ENV_VAR"] = "1" if env_set else "0"
    importlib.reload(sys.modules['MyModule'])
    from MyModule.SubModule import MyClass

    p = MyClass()
    return p

On first call it is fine and environment variable is set correctly as in logs:

get_object(env_set=False)
#log output
SOME_ENV_VAR 0
Value of SOME_ENV_VAR:  0

But on second (and later calls) when I try to change environment variable, log output show environment variable is set correctly for module but on object gets the previously set environment variable.

get_object(env_set=True)
#log output
SOME_ENV_VAR 1
SOME_ENV_VAR:  0

I tried using multiprocessing too (to create another process), which had same result. Why is this behaviour happening, and how can I fix it some simple logic?

Gorkem
  • 701
  • 8
  • 22
  • What is `MyClass.init()`? I don't see that method in the class. Do you mean `p = MyClass()`? – Barmar Dec 16 '22 at 22:21
  • If you import a module multiple times, it doesn't reload the module. – Barmar Dec 16 '22 at 22:22
  • Instead of reloading the module, put the code that does what you want into a function. Then call that function. – Barmar Dec 16 '22 at 22:24
  • @Barmar thank you , but seems I have wrongly written my problem and there is also another submodule involved, editing the answer – Gorkem Dec 16 '22 at 23:28
  • You mean the question, right? Why do so many posters say that wrong? Is it a language issue? – Barmar Dec 16 '22 at 23:29
  • It doesn't matter if it's a module or a submodule. The top-level code only runs the first time it's imported. You shouldn't put code that needs to run multiple times there, you should put it into a function and call the function. – Barmar Dec 16 '22 at 23:31
  • @Barmar thank you again, I need this kind of structure as I am testing affect of multiple parameters where they can only be passed as environment variables to docker container. – Gorkem Dec 16 '22 at 23:38

1 Answers1

0

Apparently it seems it is due to python not deleting previous instances of "submodule" which this answer https://stackoverflow.com/a/7123146/1562772 explains.

Instead of only reloading the main module, I should be deleting reference to submodule and reoading the actual submodule where the Class exists

def get_object(env_set=True):
    os.environ["SOME_ENV_VAR"] = "1" if env_set else "0"
    
    del sys.modules['MyModule.SubModule'] 
    importlib.reload(sys.modules['MyModule.SubModule']) 
    from MyModule.SubModule import MyClass
    
    p = MyClass()
    return p
Gorkem
  • 701
  • 8
  • 22