Does Python scoping rule fits the definition of lexical scoping?

Question

According to my programming language class, in a language that uses lexical scoping

The body of a function is evaluated in the environment where the function is defined, not the environment where the function is called.

For example, SML follows this behavior:

val x = 1
fun myfun () =
    x
val x = 10
val res = myfun()  (* res is 1 since x = 1 when myfun is defined *)

On the other hand, Python does not follow this behavior:

x = 1
def myfun():
    return x
x = 10
myfun()  # 10 since x = 10 when myfun is called

So why is Python described as using lexical scoping?

Answer 1

Your Python myfun is using the x variable from the environment where it was defined, but that x variable now holds a new value. Lexical scoping means functions remember variables from where they were defined, but it doesn't mean they have to take a snapshot of the values of those variables at the time of function definition.

Your Standard ML code has two x variables. myfun is using the first variable.

Answer 2

In Python, just as in SML, or (modern) Lisp, the body of a function is evaluated in the environment where it was defined. So, all three languages are lexically scoped.

In Python and Lisp, environments are mutable. That is, you can assign a new value to an existing variable, and that mutates the environment the variable is part of. Any functions defined within that environment will be evaluated in that environment—which means they will see the new value of the variable.

In SML, environments are not mutable; the environment can't change, there is no new value, so there's no question of whether the function will see that new value.

The syntax can be a bit misleading. In ML, val x = 1 and val x = 10 both define a brand new variable. In Python, x = 1 and x = 10 are assignment statements—they reassign to an existing variable, only defining a new one if there wasn't one of that name yet. (You don't see this in Lisp, where, e.g., let and setq are pretty hard to confuse.)

---

By the way, a closure with mutable variables is functionally equivalent to a mutable object (in the OO sense), so this feature of Lisp (and Python) has traditionally been pretty important.

---

As a side note, Python actually has slightly special rules for the global namespace (and the builtins one above it), so you could argue that the code in your example technically isn't relying on lexical scoping. But if you put the whole thing inside a function and call that function, then it definitely is an example of lexical scoping, so the global issue really isn't that important here.

Answer 3

In addition to the responses of @abarnert and @user2357112, it may help you to consider an SML equivalent of the Python code:

val x = ref 1
fun myfun () = !x
val () = x := 10
val res = myfun ()

The first line declares a variable x that references an integer and sets the referenced cell to 1. The function body dereferences x to return the value in the referenced cell. The third line sets the referenced cell to 10. The function call in the fourth line now returns 10.

I used the awkward val () = _ syntax to fix an ordering. The declaration is added solely for its side effect on x. It is also possible to write:

val x = ref 1
fun myfun () = !x;
x := 10;
val res = myfun ()

The environment is immutable—notably x always points to the same memory cell—but some data structures (reference cells and arrays) are mutable.

Answer 4

Let me start with a rationale that helped me understand the matter (and write this answer).

There are two concepts that overlap each other in the evaluation of variables in Python that may create some confusion:

• One is the (memory) lookup strategy Python deploys to evaluate a variable, the famous BGEL order (Built-in < Global < Enclosing < Local scopes).
• The other is the scope in which an object (function, variable) is defined and/or evaluated.

As the OP notes, Python doesn't look lexically scoped when we can switch the value of our variable ("x"), or from the fact that we can define a function (using "x") without even declaring the variables therein a priori; E.g.,

> def f():
      print(x)
> x = 1
> f()
1

What is lexical scope

So, what is lexical scope after all, and how does it work in Python?

Lexical scope means that an object's (memory) scope will that where it (i.e., the object) was defined.

Let's take a function "f()", for instance. A function has an inner scope and an outer scope. In a lexically scoped language (eg, Python), the outer scope of "f()" will be the scope where the function was defined. Always. Even if you evaluate your function (f()) anywhere else in your code, the outer scope of f() will always be that where it was defined.

So, when Python applies its BGEL symbols evaluation strategy, the "E" is where lexical scope comes into action: the scope where f() was defined.

Lexical scope at work

> def f():
      print(x)

> def g(foo):
      x = 99
      foo()
      print(x)

> x = 1
> g(f)
1
99

We then can see the recursive (BGEL) scope search strategy on evaluating variables is respecting the scopes local-and-above where the function 'f()' was defined.