3

Summary: I would like to make a Range header request from GitHub pages. However, in some browsers this is failing- possibly due to Gzip compression issues. It works in Chrome (v74) but not in FF (v66), Mac OS.

Goal: I would like to reliably make this request in all browsers, such as by forcing the response type to be encoded as text whenever a range request is made.

It is not clear to me whether this behavior is dictated by the browser, the server, or some combination of the two. Knowing origins could help to define a fix- working with Github pages would be nice but not mandatory.

It is also not clear to me whether this represents a bug, or if so, where. (in browser, in spec, etc)

Sample test case: Possibly because this involves server-side gzip encoding, the sample test case doesn't reproduce locally. You'll need to enter these commands in the JS console at https://abought.github.io/weetabix/ to reproduce.

fetch('https://abought.github.io/weetabix/example_data/McDonald_s.csv', {headers: {range: 'bytes=1-100'}} ).then(resp => resp.text());

In chrome, this fetches the response text. In firefox, it gives a "decoding error".

If I omit resp.text, Firefox can complete the request- the decoding error is in reading the body, rather than any other code. Copying as curl shows that FF adds a --compress flag and Chrome does not.

Investigations If the byte range is 0-100, the request works in FF. If the range is 1-100, it fails. This section of the file is all ASCII characters.

If I inspect the response headers (Array.from(r.headers.entries())), FF has an extra "content-encoding: gz flag" that I think is causing the issue. (eg, gzip makes no sense without the secret decoder instructions)

I tried adding 'accept-encoding': 'identity' to the fetch request, but it is a forbidden header and modifying it via code has no effect.

abought
  • 2,652
  • 1
  • 18
  • 13

1 Answers1

6

The specs have changed quite recently here. Here is the link to the PR.

TLDR; They now ask the UA that the Acccept-Encoding/Identity header be added to all the Range-requests.

[§5.15]

If httpRequest’s header list contains Range, then append Accept-Encoding/identity to httpRequest’s header list.

Firefox has not yet followed up here, but a bug report has been filled.

For the time being, the Range requests in Firefox indeed are made with the Gzipped data, and thus, you must not break the bytes integrity (for instance the range 0-100 is decode-able in Firefox).

Kaiido
  • 123,334
  • 13
  • 219
  • 285
  • 1
    Thanks for the helpful links- particularly the ticket with browser status! Though it seems there is some role in the server sending back an unreadable response, GitHub's support page explicitly excludes any requests related to fetch (and probably this interaction of headers accordingly). It seems the combination of range + compression is not overly well covered by some specs. – abought Apr 30 '19 at 14:08