In Python, variables inside if conditions hide the global scope even if they are not executed?

Question

def do_something():
    print 'doing something...'

def maybe_do_it(hesitant=False):
    if hesitant:
        do_something = lambda: 'did nothing'
    result = do_something()
    print result

maybe_do_it()

The result of this code is:

  File "scope_test.py", line 10, in <module>
    maybe_do_it()
  File "scope_test.py", line 7, in maybe_do_it
    result = do_something()
UnboundLocalError: local variable 'do_something' referenced before assignment

But this code prints "did something..." as expected:

def do_something():
    print 'doing something...'

def maybe_do_it(hesitant=False):
    result = do_something()
    print result

maybe_do_it()

How did the function get overridden even though the condition inside the if statement never executed? This happens in Python 2.7 -- is it the same in Python 3?

Answer 1

How did the function get overridden even though the condition inside the if statement never executed?

The decision whether the variable is local or global is made at compile time. If there is an assignment to a variable anywhere in the function, it's a local variable, no matter if the assignment is ever executed.

This happens in Python 2.7 -- is it the same in python 3?

Yes.

By the way, in Python 2, you can override this behavior by using exec (not recommended):

def do_something():
    print 'doing something...'

def maybe_do_it(hesitant=False):
    if hesitant:
        exec "do_something = lambda: 'did nothing'"
    result = do_something()
    print result

maybe_do_it(False)    # doing something...
maybe_do_it(True)    # did nothing

An exec inside a function will, loosely speaking, postpone the decision whether to look up the variable globally or locally to execution time.

Answer 2

As stated in documentation of Python Execution Model:

If a name binding operation occurs anywhere within a code block, all uses of the name within the block are treated as references to the current block. This can lead to errors when a name is used within a block before it is bound. This rule is subtle. Python lacks declarations and allows name binding operations to occur anywhere within a code block. The local variables of a code block can be determined by scanning the entire text of the block for name binding operations.

This is a rule of the language. That is just the way it is. :D

Answer 3

When python compiles to bytecode (makes the *.pyc file)* as there is a do_something = lambda: 'did nothing' line in your function do_something is now treated as a local variable, even if the control flow doesn't take the interpreter there.

The main reasons this is unexpected is that :

1. Contrary to common belief, Python is compiled

2. It's unintuitive.

Fundamentally I think this only becomes a problem if you're implementing bad design. When you reassign do_something from within a function you are playing with the global scope - this is rarely a good idea.

*As has been pointed out, this doesn't actually only apply to Python which is compiled to bytecode (CPython) - it's actually a feature of the language. The details of my explanation (Expressing in terms of bytecode) refers only to CPython.

Answer 4

Yes, it's the same in Python 3. For the most part this is desirable, if not entirely intuitive behavior. Perhaps you have to be Dutch. Many people may be familiar with hoisting (popularized by JavaScript?). It happens in Python, too, except that instead of having an undefined value, Python just raises an UnboundLocalError. Compare:

> // JavaScript example
> var x = 1;
> function foo() {
    if (!x) { // x is declared locally below, so locally x is undefined
      var x = 2;
    }
    return x;
  }
> foo();
2

>>> # Everything else is Python 3
>>> x = 1
>>> def foo():
...   if not x: # x is bound below, which constitutes declaring it as a local
...     x = 2
...   return x
... 
>>> foo()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 2, in foo
UnboundLocalError: local variable 'x' referenced before assignment

So far Python's consistent, but there's a (troubling) workaround:

>>> def foo():
...   if not 'x' in locals():
...     x = 2
...   return x
... 
>>> foo()
2

That works, and we've been told

The local variables of a code block can be determined by scanning the entire text of the block for name binding operations.

But doesn't locals() give us all the local names? Evidently not. In fact, despite the previous statement, Python hints that the local symbol table can change in the description of the locals() builtin:

Update and return a dictionary representing the current [emphasis mine] local symbol table.

I used to think the word current referred to the values, now I think it refers to the keys as well. But ultimately what I think this means is there's no way (short of dumping and parsing the source code for a frame) to enumerate all the names that are declared locally (which is not to say you can't use try/except UnboundLocalError to determine if a specific name is local.)

def foo():
    # Some code, including bindings
    del x, y, z # or any other local names
    # From this point, is it programmatically knowable what names are local?

This, I think, is a fundamental difference between a language with implicit declarations and a language with explicit ones.