2

I would like to use asyncio module in Python to achieve doing request tasks in parallel because my current request tasks works in sequence, which means it is blocking.

I have read the documents of asyncio module in Python, and I have wrote some simple code as follows, however it doesn't work as I thought.

import asyncio
class Demo(object):

    def demo(self):
        loop = asyncio.get_event_loop()
        tasks = [task1.verison(), task2.verison()]
        result = loop.run_until_complete(asyncio.wait(tasks))
        loop.close()
        print(result)

class Task():
   @asyncio.coroutine
   def version(self):
       print('before')
       result = yield from differenttask.GetVersion() 
       # result = yield from asyncio.sleep(1)
       print('after')

I found out that all the example they give use asyncio function to make the non-blocking works, how to make own function works as a asyncio?

What I want to achieve is that for a task it will execute the request and doesn't wait the response then it switch to next task. When I tried this: I get RuntimeError: Task got bad yield: 'hostname', which hostname is one item in my expected result.

so as @AndrewSvetlov said, differentask.GetVersion() is a regular synchronous function. I have tried the second method suggested in similar post, --- the one Keep your synchronous implementation of searching...blabla

@asyncio.coroutine
def version(self):
    return (yield from asyncio.get_event_loop().run_in_executor(None, self._proxy.GetVersion()))

And it still doesn't work, Now the error is Task exception was never retrieved future: <Task finished coro=<Task.version() done, defined at /root/syi.py:34> exception=TypeError("'dict' object is not callable",)>

I'm not sure if I understand if it right, please advice.

Community
  • 1
  • 1
ZhouQuan
  • 1,037
  • 2
  • 11
  • 19
  • What isn't working? What specific problems are you having? – dirn Mar 08 '16 at 13:08
  • What I want to achieve is that for a task it will execute the request and doesn't wait the response then it switch to next task. I have error : `RuntimeError: Task got bad yield: 'hostname' ` hostname is one item in my result. – ZhouQuan Mar 08 '16 at 13:33
  • `differenttask.GetVersion()` should be coroutine to be used with `await` syntax. Looks like you have a regular function. – Andrew Svetlov Mar 09 '16 at 11:24
  • @AndrewSvetlov Yes, GetVersion() is a regular synchronous function. I have tried the second way suggested in [task got bad yield](http://stackoverflow.com/questions/30172821/python-asyncio-task-got-bad-yield?rq=1), which makes a regular function as async function if I understand it right. But it still doesn't work...I will add my changes in my post. – ZhouQuan Mar 09 '16 at 13:16

1 Answers1

1

Change to 

@asyncio.coroutine
def version(self):
    return (yield from asyncio.get_event_loop()
            .run_in_executor(None, self._proxy.GetVersion))

Please pay attention self._proxy.GetVersion is not called here but a reference to function is passed into the loop executor.

Now all IO performed by GetVersion() is still synchronous but executed in a thread pool.

It may have benefits for you or may not. If the whole program uses thread pool based solution only you need concurrent.futures.ThreadPool perhaps, not asyncio. If the most part of the application is built on top of asynchronous libraries but only relative small part uses thread pools -- that's fine.

Andrew Svetlov
  • 16,730
  • 8
  • 66
  • 69
  • I got this error `TypeError: coroutines cannot be used with run_in_executor()`, do you know why? And I have checked `concurrent.futures.ThreadPool` and as you suggested, I think it suits to my scenarios better than `asyncio`. – ZhouQuan Mar 09 '16 at 14:07
  • `self._proxy.GetVersion` should be a regular function for `run_in_executor()` usage. Your `version()` wrapper should be a coroutine to be used with `yield from` statement. – Andrew Svetlov Mar 09 '16 at 14:39