Read a stream with for-await using JavaScript in the browser

Question

I am using the following code to download something large upon user gesture in the browser using fetch with progress indication:

const url = 'https://source.unsplash.com/random';
const response = await fetch(url);
const total = Number(response.headers.get('content-length'));
let loaded = 0;
const reader = response.body.getReader();
let result;
while (!(result = await reader.read()).done) {
  loaded += result.value.length;
  // Display loaded/total in the UI
}

I saw a snippet in a related question which lead me to believe this could be simplified into:

const url = 'https://source.unsplash.com/random';
const response = await fetch(url);
const total = Number(response.headers.get('content-length'));
let loaded = 0;
for await (const result of response.body.getReader()) {
  loaded += result.value.length;
  // Display loaded/total in the UI
}

getReader returns a ReadableStreamDefaultReader which comes from the Streams API which is a web API as well as a Node API which makes finding only web related information really hard.

In the snippet above the code fails with response.body.getReader(...) is not a function or its return value is not async iterable. I checked the object prototype and indeed I don't think it has Symbol.asyncIterator on it so yeah no wonder the browser failed to iterate over it.

So the code in that question must have been wrong, but now I wonder, is there a way to take a stream like this and iterate over it using for-await? I suppose you need wrap it in an async generator and yield the chunks in a similar way to the first snippet, right?

Is there an existing or planned API which makes this more streamlined, something closer to the second snippet, but actually working?

score 4 · Accepted Answer · answered Jul 11 '19 at 14:43

4

The ReadableStream itself implements an async iterable

 for await (const result of response.body) {
   loaded += result.value.length;
   // Display loaded/total in the UI
 }

answered Jul 11 '19 at 14:43

Jonas Wilms

132,000
20
149
151

Can you turn the code block in your answer into a runnable snippet? I tried it and have not had success with it: https://jsfiddle.net/y9f2cpro/ – Tomáš Hübelbauer Jul 11 '19 at 16:54
2

Well, seems to be very new (April, this year), probably not supported yet. – Jonas Wilms Jul 11 '19 at 16:59
I see that Microsoft Edge does not define a `Symbol.asyncIterator` on `ReadableStream`, meaning it doesn't allow `for await` to be used on readable streams. Yet. I assume same goes for Chromium. – Armen Michaeli Sep 20 '21 at 09:55

score 2 · Answer 2 · answered Sep 28 '20 at 19:01

The response.body.getReader() .read() returns {value:..., done: Boolean} meets exactly what asyncIterator needs; before the ReadableStream itself got asyncIterator implementation, you can easily polyfill it:

if (!response.body[Symbol.asyncIterator]) {
  response.body[Symbol.asyncIterator] = () => {
    const reader = response.body.getReader();
    return {
      next: () => reader.read(),
    };
  };
}
for await (const result of response.body) {
  loaded += result.length;
  console.log(((loaded / total) * 100).toFixed(2), '%');
}

See https://jsfiddle.net/6ostwkr2/

To add two thoughts to this, (1) we should probably add a "return" method to the iterator that calls `reader.releaseLock()` and (2) it's worth noting that this can be added to the prototype of ReadableStream making this available globally. Example: https://gist.github.com/sstur/29d7de58d3c7073781e26b46ee397dac — sstur, Jul 17 '22 at 15:26

Read a stream with for-await using JavaScript in the browser

2 Answers2