Webextension: How to transparently modify XMLHttpRequests

Question

My goal

… is to have a web extension (in Firefox for now) that intercepts and modified XMLHttpRequests issues by a site as transparently as possible. Best case is that the extension is undetectable by the site even when explicitly looking for it. For example, I want to be able to automatically redact/replace critical data before it is send out or enforce caching of large images despite the original page disabling that explicitly.

Original approach

Use a background script and browser.webRequest with blocking to intercept all requests.

browser.webRequest.onBeforeRequest.addListener(
  modifySend,
  {
    urls: ["<all_urls>"],
  },
  ["blocking", "requestBody"]
);

browser.webRequest.onResponseStarted.addListener(
  modifyReceive,
  {
    urls: ["<all_urls>"],
  },
  ["blocking", "responseHeaders"]
);

This works to some degree. While I can view all requests, changing the content of sends (POST) is not possible and (some?) changed headers of received are ignored by Firefox. For example I was not able to overwrite cache-control, the original value was still effective.

Content scripts

Since I'm only interested in XMLHttpRequests why not have a content script modify those as suggested in another question. Unfortunately, content scripts are isolated from the page they run in and injected scripts are detectable. The problem of changing response headers is also not solved.

Questions

What is the proper way to intercept and modify XMLHttpRequests? Is it even possible to the degree I want? And as an extension: How to I provide large data blobs as a response, for example if I do my own caching when I cannot persuade the browser to ignore "no caching" headers?

Answer 1

In the end, I used two separate approaches. One for providing large data blobs and one for XMLHttpRequest interception.

Content blobs

The easiest option seems to provide the data as web_accessible_resources and then use the extension's background script to enforce redirects:

function serveLocally(details)
{
  const localRes = browser.runtime.getManifest().web_accessible_resources;
  let ret = {};

  localRes.forEach(file => {
    if (details.url.endsWith(file))
    {
      ret.redirectUrl = browser.runtime.getURL(file);
    }
  });
  return ret;
}

Interception

Here I use a content script which injects a script into the page. The injected script overwrites the open function of XMLHttpRequest:

content.js

function injectJS(file) {

  let D = document;
  let s = D.createElement('script');

  s.type = "text/javascript";
  s.src = browser.runtime.getURL(file);
  s.onload = function() {
    s.parentNode.removeChild(s);
  };

  (D.head || D.documentElement).appendChild(s);
}

injectJS("inject.js");

inject.js (separate file and among the web_accessible_resources)

let realOpen = window.XMLHttpRequest.prototype.open;

window.XMLHttpRequest.prototype.open = function() {
  let method = arguments[0];
  let url = arguments[1];
  ...  
  realOpen.apply(this, arguments);
};