Sharing data among instances of a mapping function in Python

Question

I'm writing a graph-walking script with this function that applies a mapping function given as an argument to every vertex, but can't figure out how to share the closure since Python doesn't have pointers:

def walk_vertices(graph, function, closure):
    for vertex in graph.vertices:
        function(vertex, closure)
    return closure

This could be used, for example, to sum up the values of integer vertices, and in C-based languages, the closure would be the pointer to the running sum. What is a good way of implementing this in Python? Thanks.

score 1 · Accepted Answer · answered Jun 11 '15 at 22:31

Actually i think about languages as python or java as only having pointers to objects.

while function can not rebind to what object closure 'points' in walk_vertices, if the object closure 'points' to is mutable it can of course change it. In your example you talk about a sum. That would of course be an integer or floating point number. These are immutable in python, so closure would point to an object but you can't mutate it:

 x = 5
 def something(ref):
     # you can't change where x points to from here.
     # and because an int is immutable you can't change it.
     ref = 10 # rebinds ref, but not x
 something(x)

 print(x) # still 5

But, if you pass a mutable object you can actually store information. One way to have a very simple mutable object is just to use a list of size 1.

 x = [5]
 def something(ref):
     # you can't change where x points to from here.
     ref = 5 # rebinds ref, but not x
 something(x)

 print(x)    # still [5]

 def something2(ref):
     # ref is a mutable object, so
     ref[0] = 10 # ref points to the same list, but contents of list is now 10
 something2(x)

 print(x)    # now [10]

The same construction works with any mutable object. So a dict or a class are usable as well.

class EmptyClass:
    pass

x = EmptyClass()
x.data = 5
def something(ref):
    ref.data = 10
something(x)
print(x.data)    # now prints 10

To sum it up, python always passes something comparable to pointers. But because it has some types that are immutable you can't always use that to pass data back. You have to pass a mutable object.

Also python has no equivalent of taking the pointer of a local variable. So while everything is a pointer to an object you can't get a pointer to an pointer without having an object in between like in the list case (pointer to list of pointer). What you can do is use 'reflection' to change the value of a local variable via locals()

x = 5
def something(d):
    d['x'] = 10
something(locals())
print(x)   # now prints 10

score 0 · Answer 2 · edited May 23 '17 at 11:51

It's not clear what you mean by the "closure". If you just want the callback to be able to maintain state from call to call, there are plenty of ways to do that, such as with closures in the functional programming sense:

def walk_vertices(graph, callback):
    ...

def f(graph):
    running_sum = 0
    def callback(vertex):
        # Python 3 only
        nonlocal running_sum
        running_sum += vertex.value
    walk_vertices(graph, callback)
    return running_sum

def python2f(graph):
    running_sum = [0]
    def callback(vertex):
        # awkward hack
        running_sum[0] += vertex.value
    walk_vertices(graph, callback)
    return running_sum[0]

But I think the most natural way would be to use a generator. Instead of walk_vertices walking the graph and applying a callback to each vertices, it yields each vertex. Then the calling code can iterate over the vertices with an ordinary loop, without needing to write an awkward callback:

def walk_vertices(graph):
    # Pretend generating vertices is more complicated than this:
    for vertex in graph.vertices:
        yield vertex

running_sum = 0
for v in walk_vertices(graph):
    running_sum += v.value

# or just
vertex_sum = sum(v.value for v in walk_vertices(graph))

Sharing data among instances of a mapping function in Python

2 Answers2