Access global variable across multiple modules and threads

Question

I have two python scripts which import each other and access a variable defined in the first script. But changes made to this variable are not seen in the other module.

Code with which the behavior can be reproduced:

File a.py

import time
import b

X = {}

def set():
    global X
    X[time.time()] = 1

def get():
    global X
    return X

if __name__ == '__main__':
    b.monitor()
    while True:
        time.sleep(1)
        set()

File b.py:

import threading
import time

from a import get

def f():
    while True:
        time.sleep(1)
        print get(), 'in b'

def monitor():
    t = threading.Thread(target=f)
    t.setDaemon(True)
    t.start()

Run the program with python a.py. The result is:

{} in b
{} in b
{} in b
...

Expected output:

{"1518106774": 1} in b
{"1518106774": 1, "1518106775": 1} in b
{"1518106774": 1, "1518106775": 1, "1518106776": 1} in b
... // etc

... where the keys are time.time() strings.

I thought the global variable X should be able to be referenced wherever we want to. So why is X not updated in b.f even though the call to set is adding these keys to it?

Answer 1

The problem is that two instances of the variable X are created in such a set-up: one in a.py and another in b.py. The problem also occurs in this example that reduces it to the essence -- it is not related to creating threads:

Problem in most simple form

File a.py:

X = "original value"
import b
if __name__ == '__main__':
    b.monitor()
    print ('A', X)

File b.py:

import a
def monitor():
    print ('B', a.X)
    a.X = "new value"
    print ('B', a.X)

When running a.py the output is:

B original value
B new value
A original value

If the same X were accessed, then the last line of the output would have read:

A new value

Explanation

This happens because import creates new instances of all global names therein. It will do this only upon the first time import a is encountered, and it really is the first time here. The module created by execution of a.py is named __main__, not a. Since this name is different from a, import a will really import the module, and not just reference the already loaded __main__ module. See also the Python documentation for a description of how module names are resolved.

As a consequence the names in these two modules are distinct; import a does not make the __main__ names accessible to the other module.

Solution

A quick solution is to replace the import a statement in b.py with an import __main__ statement, and use __main__.X as the qualified name you want to access in b.py. But this is not regarded the advised way to proceed.

A more robust way is to ensure the global variables are not defined in the executed script, but in an imported module -- one that is imported by both a.py and b.py. The second import will make the names available that were defined when the first import was parsed:

File common.py:

X = "original value"

File a.py:

import common
import b
if __name__ == '__main__':
    b.monitor()
    print ('A', common.X)

File b.py:

import common
def monitor():
    print ('B', common.X)
    common.X = "new value"
    print ('B', common.X)

Now the output is:

B original value
B new value
A new value

Solution applied to your code

File common.py:

import time
X = {}
def set():
    global X
    X[time.time()] = 1

def get():
    global X
    return X

File a.py:

import common
import time
import b

if __name__ == '__main__':
    b.monitor()
    while True:
        time.sleep(1)
        common.set()

File b.py:

import common
import threading
import time

def f():
    while True:
        time.sleep(1)
        print common.get(), 'in b'

def monitor():
    t = threading.Thread(target=f)
    t.setDaemon(True)
    t.start()

Answer 2

a.py

import time
import b

X = {}


def set():
    # global X
    X[time.time()] = 1


def get():
    global X
    return X


if __name__ == '__main__':
    b.monitor()
    while True:
        time.sleep(1)

b.py

import threading
import time
import a


def f():
    while True:
        time.sleep(1)
        a.set()
        print a.get(), 'in b'


def monitor():
    t = threading.Thread(target=f)
    t.setDaemon(True)
    t.start()

and finally here's your answer:

{1518083438.771415: 1} in b
{1518083438.771415: 1, 1518083439.772717: 1} in b
{1518083440.773916: 1, 1518083438.771415: 1, 1518083439.772717: 1} in b

Answer 3

I think the answer lies in the fact that python does not truly allow global variables. A scope of a global is typically only in the same module. So when another module imports a module with globals, the namespace is copied and a variable is created with same original value. An imported Python module will not have direct access to the globals in the module which imports it, nor vice versa.

You may want to read the section 1.2.5 & 1.15.6 from Norman Matloff's Quick and Painless Python PDF Tutorial