Function that returns an accumulator in Python

Question

I am reading Hackers and Painters and am confused by a problem mentioned by the author to illustrate the power of different programming languages.

The problem is:

We want to write a function that generates accumulators—a function that takes a number n, and returns a function that takes another number i and returns n incremented by i. (That’s incremented by, not plus. An accumulator has to accumulate.)

The author mentions several solutions with different programming languages. For example, Common Lisp:

(defun foo (n)
  (lambda (i) (incf n i)))

and JavaScript:

function foo(n) { return function (i) { return n += i } }

However, when it comes to Python, the following codes do not work:

def foo(n):
    s = n
    def bar(i):
        s += i
        return s
    return bar

f = foo(0)
f(1)  # UnboundLocalError: local variable 's' referenced before assignment

A simple modification will make it work:

def foo(n):
    s = [n]
    def bar(i):
        s[0] += i
        return s[0]
    return bar

I am new to Python. Why doesn the first solution not work while the second one does? The author mentions lexical variables but I still don't get it.

You can read [nested function variable scoping](https://stackoverflow.com/questions/5218895/python-nested-functions-variable-scoping). — Sohaib Farooqi, Mar 01 '18 at 09:45
A `def` introduces a new scope. You cannot re-bind a variable in an outer scope without `global` or `nonlocal`. — Norrius, Mar 01 '18 at 09:48

MB-F · Accepted Answer · 2018-03-01T10:46:03.313

6

s += i is just sugar for s = s + i.^*

This means you assign a new value to the variable s (instead of mutating it in place). When you assign to a variable, Python assumes it is local to the function. However, before assigning it needs to evaluate s + i, but s is local and still unassigned -> Error.

In the second case s[0] += i you never assign to s directly, but only ever access an item from s. So Python can clearly see that it is not a local variable and goes looking for it in the outer scope.

Finally, a nicer alternative (in Python 3) is to explicitly tell it that s is not a local variable:

def foo(n):
    s = n
    def bar(i):
        nonlocal s
        s += i
        return s
    return bar

(There is actually no need for s - you could simply use n instead inside bar.)

*_{The situation is slightly more complex, but the important issue is that computation and assignment are performed in two separate steps.}

edited Mar 01 '18 at 10:46

answered Mar 01 '18 at 09:57

MB-F

22,770
4
61
116

1

*s += i is just sugar for s = s + i* — note that this is not true for some types. Using += on a list will absolutely mutate it (via `__iadd__`). – Norrius Mar 01 '18 at 10:17
Thanks, now I understand. I use a redundant `s` just to make the comparison between the two cases clearer. BTW, thanks for the corrections of my poor grammar. :) – erlengzi Mar 01 '18 at 10:29
1

@Norrius that is correct, but it makes the explanation easier :) The scoping rules also apply for these types, though. – MB-F Mar 01 '18 at 10:33
"you could simply use `n` instead inside `bar`" by writing `n = n + i`? will Python not make a new, local `n` for it, just the same? – Will Ness Mar 01 '18 at 17:27
@WillNess Sure. Function arguments are just like variables, so you would have to declare it as `nonlocal` inside `bar`. – MB-F Mar 02 '18 at 06:50

jpp · Answer 2 · 2018-03-01T10:43:34.677

1

An infinite generator is one implementation. You can call __next__ on a generator instance to extract successive results iteratively.

def incrementer(n, i):
    while True:
        n += i
        yield n

g = incrementer(2, 5)

print(g.__next__())  # 7
print(g.__next__())  # 12
print(g.__next__())  # 17

If you need a flexible incrementer, one possibility is an object-oriented approach:

class Inc(object):
    def __init__(self, n=0):
        self.n = n
    def incrementer(self, i):
        self.n += i
        return self.n

g = Inc(2)

g.incrementer(5)  # 7
g.incrementer(3)  # 10
g.incrementer(7)  # 17

edited Mar 01 '18 at 10:43

answered Mar 01 '18 at 10:02

jpp

159,742
34
281
339

Yes, but it's not flexible since the increase is fixed to 5. – erlengzi Mar 01 '18 at 10:36
Thanks, that's correct, though more verbose than the one using closure. :) – erlengzi Mar 01 '18 at 10:49
Sometimes, verbose is more pythonic :). But this is more a question of style / preference. I personally think it's cleaner to use classes instead of `global` / `nonlocal` statements. – jpp Mar 01 '18 at 10:52

score -1 · Answer 3 · answered Mar 01 '18 at 09:57

In Python if we use a variable and pass it to a function then it will be Call by Value whatever changes you make to the variable it will not be reflected to the original variable.

But when you use a list instead of a variable then the changes that you make to the list in the functions are reflected in the original List outside the function so this is called call by reference.

And this is the reason for the second option does work and the first option doesn't.

Function that returns an accumulator in Python

3 Answers3