Asynchronous evaluation in ipython parallel

Question

Since the awesome 1.0.0 release I've been playing around with iPython parallel interface. What I'm trying to do is to set up a asynchronous stochastic gradient descent system. The way I see it, I want to send a function to all the nodes and get the results as they come out. From what I was able to implement and glance from the documentation the standard views implemented don't really support that. The get(timeout) method would do that, but you can't really loop through every entry in a <ASync_result> object using a timeout. The way I managed to get it running was the following

c = Client()
calls = []
for i,j in enumerate(args):
    calls.append( c[ i % len( c.ids ) ].apply( f, j ) )

while condition:
    dels = []
    for i,j in enumerate( calls ):
         try:
             print j.get(0.01) #or some other timeout
             dels.append( i ) #I keep track of the calls that have been called
             #do something with the last result, throw a new call
             calls.append( c[ i % len(c.ids) ].apply( f, argument )
         except:
             pass

    for i,d in enumerate( dels ):
         del calls[ d - i ] #delete gotten calls

    #evaluate stopping condition

Now, before you all go screaming that this is horrible code and a stupid way to do that, I know it. I could make this particular way of doing it nicer, but I'm just wondering if there is some built-in way of doing something similar in IPython.parallel.

Thanks in advance to anyone taking the time.

Best, Al.

What's the point of doing it async, when you have a while loop that waits for all the tasks to finish? You just implemented a sync execution on top of the async API :) — Viktor Kerkez, Sep 09 '13 at 11:14
True, but eventually I'd like to implement a loop that keeps throwing new calls and gets them as they come out. I'll update that in the example. — Alex S, Sep 09 '13 at 11:26
My aim is to implement an assynchronous sgd for a neural network. So the calls can be somewhat out-of-sync with the current parameters. And I'd keep calling them until the error changes less than a certain amount as a moving average or something of the like. And damn, Viktor, you sure are quick on the ipython tag! Thanks a lot! — Alex S, Sep 09 '13 at 11:30

score 1 · Accepted Answer · answered Sep 09 '13 at 12:16

You can create multiple async calls, and then iterate through them.

c = Client()
dview = c[:]
asyncs = [dview.map_async(f, [arg]) for arg in args]
while asyncs:
    for async in asyncs[:]:
        if async.ready():
            asyncs.remove(async)
            print async.result[0]

Asynchronous evaluation in ipython parallel

1 Answers1

Linked