2

I'm using fetch to retrieve a URL. This is in code that is acting as a proxy and if it gets a response with content-encoding: gzip, I want to get the raw data so I can send it back to the consumer without decoding it.

But I see no way to do this. Response.blob, Response.arrayBuffer, Response.body.getReader(), .... these all seem to decode the content.

So, for example, in my test case, I end up with a 15K array of bytes instead of the actual response body which was only 4k.

Is there a way to get the raw contents of the response without it being decoded?

anthumchris
  • 8,245
  • 2
  • 28
  • 53
donaddon
  • 413
  • 2
  • 13
  • 2
    You may want to chime in https://github.com/whatwg/fetch/issues/1524 if you have a "pretty compelling use case" ;-) – Kaiido Dec 02 '22 at 00:50
  • 2
    In the whatwg issue you stated that you are able to do so with XHR. Could you please share how you do that? I don't think XHR is able to do that either, and thus what you were doing with XHR might actually be possible with `fetch()` too. – Kaiido Dec 05 '22 at 03:25

1 Answers1

2

The browser automatically decodes compressed, gzip-encoded HTTP responses in its low-level networking layer before surfacing response data to the programatic Javascript layer. Given that the compressed bytes are already transmitted over the network and available in users' browsers, that data has already been "sent to the customer".

The Compression Streams API can performantly create a Gzip-encoded, ArrayBuffer, Blob, etc:

const [ab, abGzip] = await Promise.all((await fetch(url)).body.tee().map(
  (stream, i) => new Response(i === 0
    ? stream
    : stream.pipeThrough(new CompressionStream('gzip'))
  ).arrayBuffer()
))

Full example

https://batman.dev/static/74634392/

A gzip-encoded SVG image is provided to users as a downloadable file link. The original SVG image is retrived using fetch() from a gzip-encoded response.

anthumchris
  • 8,245
  • 2
  • 28
  • 53