importing with * (asterisk) versus as a namespace in python

Question

I know that its bad form to use import * in python, and I don't plan to make a habit of it. However I recently came across some curious behaviour that I don't understand, and wondered if someone could explain it to me.

Lets say I have three python scripts. The first, first_script.py, comprises:

MESSAGE = 'this is from the first script'

def print_message():
    print MESSAGE

if __name__ == '__main__':
    print_message()

Obviously running this script gives me the contents of MESSAGE. I have a second script called second_script.py, comprising:

import first_script

first_script.MESSAGE = 'this is from the second script'

if __name__ == '__main__':
    first_script.print_message()

The behaviour (prints this is from the second script) makes sense to me. I've imported first_script.py, but overwritten a variable within its namespace, so when I call print_message() I get the new contents of that variable.

However, I also have third_script.py, comprising:

from first_script import *

MESSAGE = 'this is from the third script'

if __name__ == '__main__':
    print MESSAGE
    print_message()

This first line this produces is understandable, but the second doesn't make sense to me. My intuition was that because I've imported into my main namespace via * in the first line, I have a global variable called MESSAGES. Then in the second line I overwrite MESSAGES. Why then does the function (imported from the first script) produce the OLD output, especially given the output of second_script.py. Any ideas?

Answer 1

import module, from module import smth and from module import *can have different use cases.

The simpler:

import tools

loads the tools module and adds a reference to it in the local namespace (also named tools). After that you can access any of the tools references by prepending tools to them like for example tools.var1

Variant:

import tools as sloot

Does exactly the same, but you use the alias to access the references from the module (eg: sloot.var1). It is mainly used for module having well known aliases like import numpy as np.

The other way

from tools import foo

directly imports some symbols from the tools module in the current namespace. That means that you can only use the specified symbols by they do not need to be qualified. A nice use case is when you could import a symbol from different modules giving same functionalities. For example

try:
    from mod1 import foo
except ImportError:
    from mod2 import foo

...
foo()      # actually calls foo from mod1 if available else foo from mod2

This is commonly use as a portability trick.

The danger:

from tools import *

It is a common idiom, but may not do what you expect if the module does not document it. In fact, it imports all the public symbols from the module, by default all the symbols that have no initial _ which can contain unwanted things. In addition, a module can declare a special variable __all__ which is assumed to declare the public interface, and in that case only the symbols contained in __all__ will be imported.

Example:

mod.py

__all__ = ['foo', 'bar']

def baz(x):
    return x * 2

def foo():
    return baz('FOO')

def bar():
    return baz('BAR')

You can use (assuming mod.py is accessible)

from mod import *

print(foo())    # should print FOOFOO

# ERROR HERE
x = baz("test") # will choke with NameError: baz is not defined

while

import mod

print(mod.baz("test"))     # will display as expected testtest

So you should only use from tools import * if the documentation of the tools module declares it to be safe and lists the actually imported symbols.

Answer 2

This has to do with Scope. For a very excellent description of this, please see Short Description of the Scoping Rules?

For a detailed breakdown with tons of examples, see http://nbviewer.ipython.org/github/rasbt/python_reference/blob/master/tutorials/scope_resolution_legb_rule.ipynb

Here's the details on your specific case:

The print_message function being called from your third test file is being asked to print out some MESSAGE object. This function will use the standard LEGB resolution order to identify which object this refers to. LEGB refers to Local, Enclosing function locals, Global, Builtins.

1. Local - Here, there is no MESSAGES defined within the print_message function.
2. Enclosing function locals - There are no functions wrapping this function, so this is skipped.
3. Global - Any explicitly declared variables in the outer code. It finds MESSAGE defined in the global scope of the first_script module. Resolution then stops, but i'll include the others for completeness.

4. Built-ins - The list of python built-ins, found here.

   So, you can see that resolution of the variable MESSAGE will cease immediately in Global, since there was something defined there.

Another resource that was pointed out to me for this is Lexical scope vs Dynamic scope, which may help you understand scope better.

HTH

Answer 3

Direct assignment changes the reference of an object, but modification does not. For example,

a = []
print(id(a))
a = [0]
print(id(a))

prints two different IDs, but

a = []
print(id(a))
a.append(0)
print(id(a))

prints the same ID.

In second_script.py, the assignment merely modifies first_script, which is why both first_script.py and second_script.py can locate the same attribute MESSAGE of first_script. In third_script.py, the direct assignment changes the reference of MESSAGE; therefore, after the assignment, the variable MESSAGE in third_script.py is a different variable from MESSAGE in first_script.py.

Consider the related example:

first_script.py

MESSAGE = ['this is from the first script']

def print_message():
    print(MESSAGE)

if __name__ == '__main__':
    print_message()

third_script.py

from first_script import *

MESSAGE.append('this is from the third script')

if __name__ == '__main__':
    print(MESSAGE)
    print_message()

In this case, third_script.py prints two identical messages, demonstrating that names imported by import * can still be modified.