2

In the link https://zetcode.com/python/httpx/, it has the following example for stream

import httpx

url = 'https://download.freebsd.org/ftp/releases/amd64/amd64/ISO-IMAGES/12.0/FreeBSD-12.0-RELEASE-amd64-mini-memstick.img'

with open('FreeBSD-12.0-RELEASE-amd64-mini-memstick.img', 'wb') as f:
    with httpx.stream('GET', url) as r:
        for chunk in r.iter_bytes():
            f.write(chunk)

Is it a way to stream the data asynchronously? e.g.

async def stream(call_back):
   async with httpx.stream('GET', url) as r:
       for chunk in await? r.iter_bytes():
           await call_back(chunk)
ca9163d9
  • 27,283
  • 64
  • 210
  • 413

1 Answers1

3

This should do the work,

async def stream(cb):
    async with httpx.AsyncClient() as client:
        async with client.stream('GET', url) as resp:
            async for chunk in resp.aiter_bytes():
                await cb(chunk)

The problem is that each chunk is pretty small, like 3K bytes.

ca9163d9
  • 27,283
  • 64
  • 210
  • 413
  • 1
    I was just working on a [similar task](https://stackoverflow.com/a/71398460/17865804) and would like to complement this answer with the [relevant HTTX documentation](https://www.python-httpx.org/async/#streaming-responses). As for the chunk size, I don't think one can have actual control over the chunk size. The chunk size is actually controlled by the server/client sending the content. Even if you used, for example, `iter_raw(1024 * 1024)`, it wouldn't really mean that you have control over how the chunks are being sent to your application (see the linked answer above for more details). – Chris Aug 26 '22 at 14:21