async and await with Python

Question

I have worked with async/await in C# before as in this question, but it is a concept that I need to go to the basics for a refresh from time to time.

This time I am working in Python, but this question is more about the concept of Async Await than the implementation.

Basically I have a server to which I send a request and this server sends a response. If it is only once I have

async def send_and_get_response(self,request):
   self.remote_server.send_request(request)
   response= await self.remote_server.get_response()
   return response

No problem there.

However, imagine that I want to send not one but several requests to the server, and I want to await for the responses. I don't want to send one request after I get the response, but all at once. I don't want to miss responses either.

Correct me if I am wrong but the nature of async/await is that if I put code after await this code won't get executed until the wait is complete right?

So this won't do right?

   async def send_and_get_response(self,request1,request2):
       self.remote_server.send_request(request1)
       response1= await self.remote_server.get_response()
       self.remote_server.send_request(request2)
       response2= await self.remote_server.get_response()
       return [response1,response2]

I imagine that in the above case, request2 will get send only after response1 has arrived, am I wrong?

So what to do? I thought something like

 async def send_and_get_response(self,request1,request2):
       self.remote_server.send_request(request1)
       self.remote_server.send_request(request2)
       response1= await self.remote_server.get_response()
       response2= await self.remote_server.get_response()
       return [response1,response2]

but this somehow does not feel right. What if response1 arrives immediately before I could await it? What if response2 arrives before than response1?

So how about this one:

async def send_and_get_response(self,request):
   self.remote_server.send_request(request)
   response= await self.remote_server.get_response()
   return response

response1=send_and_get_response(request1)
response2=send_and_get_response(request2)

Will this work?

I repeat, what I want to accomplish is to send two (or more) requests to the server and await the response from all the requests. They can come in a different order.

Edit

How about this one

    async def send_and_get_response(self,request):
       self.remote_server.send_request(request)
       response= await self.remote_server.get_response()
       return response
    
 results=  await asyncio.gather(
            send_and_get_response(request1),
            send_and_get_response(request2),
            send_and_get_response(request3),
        )

Will this work? If it does, how do I get the responses?

Edit 2

I have checked other resources and I wonder how to make the following work

async def process_when_done(tasks):
    for res in asyncio.as_completed(tasks):
        print("Result %s" % await res)    
    
tasks2= [ send_and_get_response(request1),
        send_and_get_response(request2),
        send_and_get_response(request3)]   
    
process_when_done(tasks2)

In this case I am trying to process every process as soon as it finishes

score 1 · Answer 1 · answered May 25 '20 at 01:40

1

I imagine that in the above case, request2 will get send only after response1 has arrived, am I wrong?

You are correct.

but this somehow does not feel right. What if response1 arrives immediately before I could await it? What if response2 arrives before than response1?

It could be right, as you want to wait for all the responses(Which means you are actually waiting for the longest response). It may not be right if you need to do something with the response immediately:

async def send_and_get_response(self,request1,request2):
    self.remote_server.send_request(request1)
    self.remote_server.send_request(request2)
    response1= await self.remote_server.get_response()
    # do something here with response1
    # if response2 arrives first, then we are wasting time.
    response2= await self.remote_server.get_response()
    # do something here with response2
    return [response1,response2]

will this work?

No, send_and_get_response is an async function, you need to await send_and_get_response, or you do nothing, but if you await it, then there is no difference from the first implementation.

Actually we use asyncio.gather for this purpose in Python.

answered May 25 '20 at 01:40

Sraw

18,892
11
54
87

I see, so the last one will not work. I see it now. ;( I will check your link, but do you have some idea how to do what I wish to do? – KansaiRobot May 25 '20 at 02:05
1

Your second implementation actually works. Or you can follow the link to use a more pythonic way. – Sraw May 25 '20 at 02:06
I have added one more try with `asyncio.gather`. Could you take a look please? – KansaiRobot May 25 '20 at 13:23
Yeah, it makes sense. – Sraw May 25 '20 at 14:51
thanks. Are the responses going to come in the order they have been called? or in the order they come from the server? – KansaiRobot May 25 '20 at 23:25
1

"The order of result values corresponds to the order of awaitables in aws." See the document. – Sraw May 25 '20 at 23:35
Thanks. I have researched a bit more and draft another solution to process the responses as soon as they came. what do you think? – KansaiRobot May 25 '20 at 23:55
I think nothing to it... It makes sense though. Actually you can try your code. – Sraw May 26 '20 at 00:22

async and await with Python

Edit

Edit 2

1 Answers1