Python: Name resolution; order of function def's

Question

I have a very simple example:

#!/usr/bin/env python

#a()  # 1: NameError: name 'a' is not defined
#b()  # 1: NameError: name 'b' is not defined
#c()  # 1: NameError: name 'c' is not defined

def a():
    c()   # note the forward use here...

#a()  #2: NameError: global name 'c' is not defined 
#b()  #2: NameError: name 'b' is not defined
#c()  #2: NameError: name 'c' is not defined

def b():
    a()

#a()   #3: NameError: global name 'c' is not defined    
#b()   #3: NameError: global name 'c' is not defined
#c()   #3: NameError: name 'c' is not defined

def c():
    pass

a()    # these all work OK...   
b()
c()

I have 3 functions named a(), b() and c() defined in a Python source file in alphabetical order. The body of each function definition is a call to one of the other functions. You can see by my comments that I have to have the initial call to the first of these functions BELOW their definitions (in the text file), but you do not necessarily need definition of a function above another function that calls it.

Certainly it seems to be common practice to have the first executable code below all the function definitions (in Python and many other languages), and now I can see why. In C and C++, header files take care of this. In Pascal you must have name definitions prior to their use.

Assume, for example, that you have this in Python:

def a(a_arg):          c(a_arg)
def b(b_arg):          a()
def c(a_arg,b_arg):    b(b_arg)
a(1)

It will fail properly with TypeError: c() takes exactly 2 arguments (1 given) at runtime where the other errors are compile time. (in C, this would compile then fail mysteriously...)

In Perl, since subroutines names are USUALLY resolved at run time, you can have Perl definitions and code in any order:

#!/usr/bin/env perl

a();
b();
c();

sub a{ c(); }
sub b{ a(); }
sub c{ return; }

In C, it is either an error or a warning (implementation dependent) to use a function that has not been prototyped and shouldn't be ignored.

You can have this:

void a(void) { c(); }   /* implicitly assumed to be int c(...) unless prototyped */
void b(void) { a(); }
void c(void) { return; }

int main(void) {
    a();
    return EXIT_SUCCESS;
}

My assumptions and confusion is this: If Python does not resolve subroutines names until runtime, why does the source compile phase fail with the forward declaration of subroutine names that have not been defined yet? Is it documented somewhere (other than by observation of other code) that you cannot have code in a source file above definitions of subroutines?

It seems that Python has elements of dynamic name resolution (the use of c() in a() prior to its definition below in the source file) and elements of static name resolution (the failure of Python to run the call to a() if placed above its definition in the source file.)

Is there a Python version of THIS DOCUMENT that covers the lifecycle of a Perl executable and how names are resolved between source file interpretation and runtime?

Is there a definitive description somewhere on the order of definitions for a Python script that states functions can have forward definitions of other subroutine names but main code cannot?

Edit and conclusion

After some spirited comments, and some research on my part, I have concluded that my question is really more about how names are resolved, and how namespaces, scopes and modules are defined in Python.

From carot-top:

"a callable must be defined before it is called in the current namespace." and this link on scopes and names

From S.Lott:

"When a name is used in a code block, it is resolved using the nearest enclosing scope." and this link to the execution life of a Python script.

From the Python documents:

"A scope defines the visibility of a name within a block." From the Python Execution model

"A module can contain executable statements as well as function definitions." in more about modules

"In fact function definitions are also ‘statements’ that are ‘executed’; the execution of a module-level function enters the function name in the module’s global symbol table." in the footnote thereto.

And my own realization (Duh!) that:

1. Every Python source file is treated as a "module" by Python: "A module is a file containing Python definitions and statements."

2. Unlike Perl (which I have more experience with) Python executes modules as they are being read. Hence the failure of a immediately executable statement referring to a function not yet defined in the same module.

Answer 1

The order of definitions is simply "everything has to be defined before you call it". That's pretty much it.

edit (to include answer in comments, elucidated):

The reason something like

def call_a():
    a()

def a():
    pass

call_a()

works when you've got a() in call_a() before a is even defined as a function is because Python only actually looks up values for symbols on an as-needed basis. When call_a is evaluated, the a() call is basically stored as bytecode instructions to "look up what a is and call it" when the time comes, which isn't until you get down to the actual invocation of call_a() at the bottom.

Here is what the disassembled bytecode of call_a looks like (via dis.dis):

Disassembly of call_a:
  2           0 LOAD_GLOBAL              0 (a)
              3 CALL_FUNCTION            0
              6 POP_TOP
              7 LOAD_CONST               0 (None)
             10 RETURN_VALUE

So basically, when you hit call_a, it loads whatever is stored as a onto the stack, calls it as a function, and then pops the return value off before returning None, which is what implicitly happens for anything that doesn't explicitly return (call_a() is None returns True)

Answer 2

Following the various comments and struggling to understand some Perl concepts from a Python history, let me take a crack at this. Please reset your brain on some of things you learned in Perl. They do not apply in Python. (And vs vs...)

There are no forward declarations in Python. None. Technically, ALL functions are anonymous objects; they just happen to be bound to the name you used to define it. You can re-bind them at will.

The dictionary of these functions can be found using the locals() function like this:

>>> def a(): b()
... 
>>> locals()['a']
<function a at 0x100480e60>
>>> locals()['b']
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
KeyError: 'b'
>>> a()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 1, in a
NameError: global name 'b' is not defined

If Python required b() to be defined BEFORE writing a(), this would be a problem in the Python interpreter. You would need to write all you functions in strict order.

Since all builtin function names are just bounded names, you can easily override builtin functions:

>>> abs(-1)
1
>>> def abs(num): print "HA Fooled you!!!"
... 
>>> abs(-1)
HA Fooled you!!!
>>> abs=__builtins__.abs
>>> abs(-1)
1

It is much more difficult (but possible) to override built-ins in Perl. (The disadvantage here is a mistyped def [builtin]: can unintentionally override the built-in function without warning)

The best description I can refer you to for names and scope in Python is actually the tutorial on Classes -- section 9.2

There is not actually chapter and verse why def need to come before executable code because this is not a true statement. Consider:

#!/usr/bin/env python

def fake_a(): print " a fake"
a=fake_a
a()  
def a():  print "this is a()"
a()

Or even:

def a(): print " first a()"
a()  
def a():  print "second a()"
a()

What is true is that a callable must be defined before it is called in the current namespace. Hence, usually, above the point of being called by executable code in a source text file or module. Each function has its own namespace; calls to other functions not yet defined only fail when that function is called in that local and executing namespace -- when the callable is called from the namespace of the function. This why you can have what looks like a "forward declaration" in your example. The module call of a "forward" callable outside a function fails because the function def has not been executed yet so it is not in the current namespace.

Answer 3

This is exactly the same as in C/C++ etc. You can't use something until it exists. In C/C++ you can't refer to something until it has been declared. Remember that a Python file is processed from top to bottom so if you attempt to call a function or reference a variable that does not exist, then it fails.

Answer 4

http://docs.python.org/reference/executionmodel.html

---

When a name is used in a code block, it is resolved using the nearest enclosing scope. The set of all such scopes visible to a code block is called the block’s environment.

The following constructs bind names: formal parameters to functions, import statements, class and function definitions (these bind the class or function name in the defining block), ...

That's it. There's no concept of "forward" or "reverse" or "declaration" or anything like that.

Answer 5

I know this question has been answered for quite a while now, but I also had trouble understanding this and would like to demonstrate how I found the answer with a simple example:

def a():
     print globals()
     b()
a()

In the example above, globals would display the location for the function that was defined with the name b if your main executable code defined a function with the name b. If not, the globals would be empty and call to a would result in

NameError: global name 'b' is not defined  

such as in the example of the wolf. What this means is that while function names need to be declared before they are called for them to exist in the namespace of the main executing code, the function that the main executing code calls already have the definitions found in main which is why function calls from functions need not follow a strict order.