1

I have an API endpoint whose response comes from a secondary server through a transparent proxy. Let us say that normally the answer is

{ "hello": "world" }

Due to the architecture (and yes, it is bad practice, but I haven't authority over this), the "transparent proxy" violates standards, injecting HTML messages in whatever passes through (PNG images included!). The headers are sent from the backend and are not touched by the proxy.

Occasionally, in the obvious scenario, the answer is, almost literally,

<div class="red">ERROR: session token not found</div>
{ "hello": "foobar" }

What is happening is that the backend answers with HTTP/200 and application/json, because it doesn't (and shouldn't) care about the HTTP token. It simply answers with something that's logically garbage (imagine an ATM that, if your PIN is correct, tells you your balance; but if it is incorrect, answers "Balance: 0", because the unauthenticated customer has a balance of 0).

On these occasions, as expected, the front end jQuery throws a fit since the answer is definitely not valid JSON, and I am informed through a .catch(xhr) fallback, and parsing its errorText I can recover the error text. So, I can tell the user that there was a problem, and what the problem was. From the front-end and application points of view, everybody lives happily ever after.

My small problem here is when debugging, because Firefox accepts the answer and displays it as if it was valid JSON, hiding the HTML as if it didn't exist:

{
    "hello": "foobar"
}

even if, toggling the [x] RAW switch in the same Firefox Developer Tools window, the non-JSON HTML preamble is displayed as expected, so I can see it's not JSON, albeit, jumbled as it is, I can't see much else:

<div class="red">ERROR: session token not found</div>
{"hello":"foob...

How can I ensure that Firefox Developer Tools will actually treat an invalid JSON as invalid, or show it visually (a flag, a red background...), without having to manually toggle the RAW switch off (to check for errors) and on (to inspect the contents) at each request? Which is unquestionably doable (indeed, it's what I'm doing now), but really irksome.


Example to reproduce

  • firefox_json_mangle_test.html
<!DOCTYPE html>
<html lang="en">
<head>
<title>JSONFail</title>
<meta charset="utf-8">
</head>
<body>
    <button onclick="json_test(0, '200 OK')">OK</button>
    <button onclick="json_test(1, '200 OK')">FAIL 200</button>
    <button onclick="json_test(2, '500 Some fail')">FAIL 500</button>
<script>
let XHR = (url) => {
    console.debug(';; XHR', url);
    return new Promise(function(resolve, reject) {
        const xhr = new XMLHttpRequest();
        xhr.open('GET', url, true);
        xhr.onload = function() {
            if (xhr.status == 200) {
                resolve(xhr);
            } else {
                reject(xhr);
            }
        };
        xhr.send(null);
    });
};
let json_test = (state, http_status) => {
    let xhr = XHR('json_serve.php?state=' + state
        + '&http_status=' + escape(http_status)
    );
    xhr.then(
        xhr => console.info(xhr)
    ).catch(
        xhr => console.error(xhr)
    );
};
</script>
</body>
</html>
  • json_serve.php (In same directory as html file)
<?php

header($_SERVER["SERVER_PROTOCOL"] . " " . $_GET['http_status']);
header('Content-Type: application/json;charset=utf-8');

if ($_GET['state'] == '0') {
    echo '{"hello": "world"}';
} else {
    echo '<div class="red">ERROR: session token not found</div>' . "\n";
    echo '{"hello": "foobar"}';
}
user3342816
  • 974
  • 10
  • 24
LSerni
  • 55,617
  • 10
  • 65
  • 107
  • i guess the header says that its json (content type application/json)? – flaxon Aug 24 '21 at 15:43
  • Is the server status response 200 OK? – user3342816 Aug 24 '21 at 15:45
  • @flaxon yes, the backend is unaware that someone else added (*incorrectly*) HTML prefix to its output. – LSerni Aug 25 '21 at 13:30
  • @user3342816 unfortunately yes, because that header is sent from the backend. The proxy simply injects HTML garbage that borks the JSON. I'm okay with that, I'd just wish I could spot these calls more easily instead of opening them one by one. – LSerni Aug 25 '21 at 13:32
  • 1
    Ah. OK. That's not good, but neither is FF's handling of mangled json. Added a minimal example (tested for myself). Roll-back if you do not like it ;). `⇅ Network` tab is enough of course, but added some more verbose logging. – user3342816 Aug 25 '21 at 14:44
  • By example code, I tried to change content header to `text/plain` and `text/html` but the `Response` is still displayed as `JSON` ... The html page itself is displayed as is (display view, HTML markup in raw. So it seems it does some *document detection* no matter the type. In all fairness the JSON view is nice even on mangled response - but as I see it `Raw` should have been default view on invalid (with the option to switch *to* JSON). – user3342816 Aug 25 '21 at 15:07
  • 1
    @user3342816 yeah, "the JSON view is nice even on mangled response" -- but that's exactly the problem, I can't easily see when a response is mangled from the FF Tools view unless I wiggle the RAW switch on each and every request (wonderful edit, by the way: thanks!). – LSerni Aug 25 '21 at 15:09
  • OK. Have looked at this further. From what I gather the rationale behind it ***might*** be that `Raw` is a persistent option per window-session. As in if I select `Raw` I get raw view on all network requests. That is in `⇅ Network` tab only though. Hence; if I select raw-view all responses I click on is raw. This way I can see *JSON view* or *Raw-view* on all. If it switched by parsing result that be a downside again if I really wanted that view. In `❏ Console` however this is not persistent. As it *try hard* to represent JSON this get precedence even if data is bad. – user3342816 Aug 25 '21 at 16:49
  • There *should* at least be a notification if JSON is bad. How this is solved in the best way is something worthy a debate - and should at least be a bug report (which is the only way to resolve it natively as-is from what I can see). One option is to create an extension but that is perhaps not a route you are willing to go. Made a test where I check response on network requests for devtools and log error on bad format - but that is in `❏ Console` . – user3342816 Aug 25 '21 at 16:56

1 Answers1

1

This has been reported as bug 1708356. It is apparently an intentional (if overzealous) feature, meant to facilitate viewing JSON responses with XSSI mitigation sentinels. The discussion under the bug report does however suggest the feature is about to be limited.

Firefox ESR 78 is not affected, so until the ticket is resolved, you may consider downgrading; otherwise though, there is little you can do. Doing web development under the ESR version shouldn’t hurt either; since some users do prefer using the ESR version, it should pay off to keep compatibility with it.

user3840170
  • 26,597
  • 4
  • 30
  • 62