How to make main.py's global variable visible in imported module?

Question

When using this main.py

from mymodule import myfunc

GENERALPARAM = "6878623"

for i in xrange(10000):
    myfunc(i)

and this mymodule.py:

def myfunc(a):
    # very fast computation here
    if condition(a):
         log(a, GENERALPARAM)

how to make that GENERALPARAM is available in mymodule.py's myfunc? By using a global variable (how?)?

Of course I could pass it as a parameter:

#main.py    
for i in xrange(10000):
    myfunc(i, GENERALPARAM)

#mymodule.py    
def myfunc(a, GENERALPARAM):
    ...

but I would like to avoid to pass millions of times (for very short computation) the same variable as parameter.

Why would you want to do this? Just pass in the parameter to the function. Globals exist **per module**, not across all modules in Python. You'd have to define the global first before importing the other module so the other module can import `__main__` and reference the global from the `__main__` module, or you'd need to explicitly set the global on the imported module. That's ugly, error prone, and doesn't scale. — Martijn Pieters, Jan 10 '18 at 15:40
Also linked: https://stackoverflow.com/questions/606561/how-to-get-filename-of-the-main-module-in-python — Basj, Jan 10 '18 at 15:54

bruno desthuilliers · Accepted Answer · 2018-01-11T09:47:22.927

TL;DR : There's no clean way to do so, and actually for pretty good reasons. Also, chances are passing it as an argument will actually be as fast if not faster (arguments are local to the function and local lookups are faster than global ones).

More detailed answer:

The "pretty good reasons" are about maintainability and robustness. A function that only depends on it's inputs is easy to understand and test and has a predictable behaviour. A function that depends on a module-level global is already harder to understand and test and becomes much less predictable. A function depending on a global defined in the main script (yes, this can be done in a hackish way, and I'm NOT going to show this way here) is not only totally untestable and unpredictable, but also requires that the importing modules does define this global, which is brittle at best (and actually makes the module unusable in a different context). There's already plenty of litterature on the "globals are evil" topic so I won't explain further...

Now wrt/ the OP motivations, ie

I would like to avoid to pass millions of times (for very short computation) the same variable as parameter.

I assume the OP's main concern here are performances...

In Python, passing an argument to a function has a rather low cost - nothing is copied, what is actually passed is (a reference to) the object itself (cf this reference article from Ned Batcheler on Python's names and values for more details). Also, the argument is a local variable of the function so the name lookup during function execution is the fastest possible - faster than resolving a module global, and much faster than resolving an attribute on a module global. So the clean and obvious solution (passing the objec as argument) will probably be as fast if not faster than the hackish one.

As a matter of fact, I did a quick&crude benchmark of both solutions - alas without the timeit module which would have yielded more accurate results but the very nature of the hack made it impossible to use timeit here (yes, hacks never make your life easier) - using both Python 2.7.6 and Python 3.4.3, and the average difference (using 500000 iterations in the main loop) was quite small and actually less than the average variability between two calls of the same code version. I suspect than with real-life code in the function with multiple lookups of the variable, the clean solution would tend to become faster due to local lookup vs global lookup costs.

The code I used for the benchmark:

Clean solution:

# main2.py
import datetime
start = datetime.datetime.now()

from lib2 import myfunc
GENERALPARAM = ["foo", "bar"]

if __name__ == "__main__":
    for i in range(500000):
        myfunc(i, GENERALPARAM)

    done = datetime.datetime.now()
    print("done in %s" % (done - start))

# lib2.py
def myfunc(i, p): 
    return "{} - {}".format(i, p)

The hackish solution:

# main.py
import datetime
start = datetime.datetime.now()

from lib import myfunc
GENERALPARAM = ["foo", "bar"]


if __name__ == "__main__":
    for i in range(500000):
        myfunc(i)

    done = datetime.datetime.now()
    print("done in %s" % (done - start))


# lib.py
import __main__ as main # this is the hack. DONT DO THIS AT HOME

def myfunc(i): 
    return "{} - {}".format(i, main.GENERALPARAM)

This could just have been left as a comment. Do explain those reasons, and the not-so-clean ways of making it work anyway. Like Abarnert did in the duplicate. — Martijn Pieters, Jan 10 '18 at 15:46
I don't see why the working answer was deleted. I know that it's usually 99% of the time a bad way to do it, but sometimes, such code can be useful for small scripts. — Basj, Jan 10 '18 at 15:47
@Basj I don't know why the "working answer" was deleted either but as far as I'm concerned I'm glad it has been - maintaining code is hard enough so advising hackish "solutions" is the last thing we need. — bruno desthuilliers, Jan 11 '18 at 08:31
@Basj: cf my edited answer for more detailed explanations and a quick&crude benchmark. As you will fnd out, there's nothing to gain from the hack except maintainability problems... — bruno desthuilliers, Jan 11 '18 at 09:49

How to make main.py's global variable visible in imported module?

1 Answers1