I want to use http client as a class member, but del function could not call await client.aclose().
e.g.:
import httpx
class Foo(object):
def __init__(self):
self.client = httpx.AsyncClient()
def __del__(self):
await self.client.aclose()
refer: https://www.python-httpx.org/async/#opening-and-closing-clients how to safely aclose?
Although this is an older question, I may have something compelling to share as I had a similar situation. To @Isabi's point (Answered 2020-12-28), you need to use an event loop to decouple the client from your operations and then manually control it's lifecycle.
In my case, I need more control over the client such that I can separate the Request from the sending and when the client is closed so I can take advantage of session pooling, etc. The example provided below shows how to use http.AsyncClient as a class member and close it on exit.
In figuring this out, I bumped into an Asyncio learning curve but quickly discovered that it's ... actually not too bad. It's not as clean as Go[lang] but it starts making sense after an hour or two of fiddling around with it. Full disclosure: I still question whether this is 100% correct.
The critical pieces are in the __init__, close, and the __del__ methods. What, to me, remains to be answered, is whether using a the http.AsyncClient in a context manager actually resets connections, etc. I can only assume it does because that's what makes sense to me. I can't help but wonder: is this even necessary?
import asyncio
import httpx
import time
from typing import Callable, List
from rich import print
class DadJokes:
headers = dict(Accept='application/json')
def __init__(self):
"""
Since we want to reuse the client, we can't use a context manager that closes it.
We need to use a loop to exert more control over when the client is closed.
"""
self.client = httpx.AsyncClient(headers=self.headers)
self.loop = asyncio.get_event_loop()
async def close(self):
# httpx.AsyncClient.aclose must be awaited!
await self.client.aclose()
def __del__(self):
"""
A destructor is provided to ensure that the client and the event loop are closed at exit.
"""
# Use the loop to call async close, then stop/close loop.
self.loop.run_until_complete(self.close())
self.loop.close()
async def _get(self, url: str, idx: int = None):
start = time.time()
response = await self.client.get(url)
print(response.json(), int((time.time() - start) * 1000), idx)
def get(self, url: str):
self.loop.run_until_complete(self._get(url))
def get_many(self, urls: List[str]):
start = time.time()
group = asyncio.gather(*(self._get(url, idx=idx) for idx, url in enumerate(urls)))
self.loop.run_until_complete(group)
print("Runtime: ", int((time.time() - start) * 1000))
url = 'https://www.icanhazdadjoke.com'
dj = DadJokes()
dj.get_many([url for x in range(4)])
Since I've been using Go as of late, I originally wrote some of these methods with closures as they seemed to make sense; in the end I was able to (IMHO) provide a nice balance in between separation / encapsulation / isolation by converting the closures to class methods.
The resulting usage interface feels approachable and easy to read - I see myself writing class based async moving forward.
The problem might be due to the fact that client.aclose() returns an awaitable, which cannot be awaited in a normal def function.
It could be worth giving a try with asyncio.run(self.client.aclose()). Here it might occur an exception, complaining that you are using a different event loop (or the same, I don't know much of your context so I can't tell) from currently running one. In this case you could get the currently running event loop and run the function from there.
See https://docs.python.org/3/library/asyncio-eventloop.html for more information on how you could accomplish it.
