How can I intercept XMLHttpRequests from a Greasemonkey script?

Question

I would like to capture the contents of AJAX requests using Greasemonkey.

Does anybody know how to do this?

Answer 1

The accepted answer is almost correct, but it could use a slight improvement:

(function(open) {
    XMLHttpRequest.prototype.open = function() {
        this.addEventListener("readystatechange", function() {
            console.log(this.readyState);
        }, false);
        open.apply(this, arguments);
    };
})(XMLHttpRequest.prototype.open);

Prefer using apply + arguments over call because then you don't have to explicitly know all the arguments being given to open which could change!

Answer 2

How about modifying the XMLHttpRequest.prototype.open or send methods with replacements which set up their own callbacks and call the original methods? The callback can do its thing and then call the callback the original code specified.

In other words:

XMLHttpRequest.prototype.realOpen = XMLHttpRequest.prototype.open;

var myOpen = function(method, url, async, user, password) {
    //do whatever mucking around you want here, e.g.
    //changing the onload callback to your own version


    //call original
    this.realOpen (method, url, async, user, password);
}  


//ensure all XMLHttpRequests use our custom open method
XMLHttpRequest.prototype.open = myOpen ;

Answer 3

Tested in Chrome 55 and Firefox 50.1.0

In my case I wanted to modify the responseText, which in Firefox was a read-only property, so I had to wrap the whole XMLHttpRequest object. I haven't implemented the whole API (particular the responseType), but it was good enough to use for all of the libraries I have.

Usage:

    XHRProxy.addInterceptor(function(method, url, responseText, status) {
        if (url.endsWith('.html') || url.endsWith('.htm')) {
            return "<!-- HTML! -->" + responseText;
        }
    });

Code:

(function(window) {

    var OriginalXHR = XMLHttpRequest;

    var XHRProxy = function() {
        this.xhr = new OriginalXHR();

        function delegate(prop) {
            Object.defineProperty(this, prop, {
                get: function() {
                    return this.xhr[prop];
                },
                set: function(value) {
                    this.xhr.timeout = value;
                }
            });
        }
        delegate.call(this, 'timeout');
        delegate.call(this, 'responseType');
        delegate.call(this, 'withCredentials');
        delegate.call(this, 'onerror');
        delegate.call(this, 'onabort');
        delegate.call(this, 'onloadstart');
        delegate.call(this, 'onloadend');
        delegate.call(this, 'onprogress');
    };
    XHRProxy.prototype.open = function(method, url, async, username, password) {
        var ctx = this;

        function applyInterceptors(src) {
            ctx.responseText = ctx.xhr.responseText;
            for (var i=0; i < XHRProxy.interceptors.length; i++) {
                var applied = XHRProxy.interceptors[i](method, url, ctx.responseText, ctx.xhr.status);
                if (applied !== undefined) {
                    ctx.responseText = applied;
                }
            }
        }
        function setProps() {
            ctx.readyState = ctx.xhr.readyState;
            ctx.responseText = ctx.xhr.responseText;
            ctx.responseURL = ctx.xhr.responseURL;
            ctx.responseXML = ctx.xhr.responseXML;
            ctx.status = ctx.xhr.status;
            ctx.statusText = ctx.xhr.statusText;
        }

        this.xhr.open(method, url, async, username, password);

        this.xhr.onload = function(evt) {
            if (ctx.onload) {
                setProps();

                if (ctx.xhr.readyState === 4) {
                     applyInterceptors();
                }
                return ctx.onload(evt);
            }
        };
        this.xhr.onreadystatechange = function (evt) {
            if (ctx.onreadystatechange) {
                setProps();

                if (ctx.xhr.readyState === 4) {
                     applyInterceptors();
                }
                return ctx.onreadystatechange(evt);
            }
        };
    };
    XHRProxy.prototype.addEventListener = function(event, fn) {
        return this.xhr.addEventListener(event, fn);
    };
    XHRProxy.prototype.send = function(data) {
        return this.xhr.send(data);
    };
    XHRProxy.prototype.abort = function() {
        return this.xhr.abort();
    };
    XHRProxy.prototype.getAllResponseHeaders = function() {
        return this.xhr.getAllResponseHeaders();
    };
    XHRProxy.prototype.getResponseHeader = function(header) {
        return this.xhr.getResponseHeader(header);
    };
    XHRProxy.prototype.setRequestHeader = function(header, value) {
        return this.xhr.setRequestHeader(header, value);
    };
    XHRProxy.prototype.overrideMimeType = function(mimetype) {
        return this.xhr.overrideMimeType(mimetype);
    };

    XHRProxy.interceptors = [];
    XHRProxy.addInterceptor = function(fn) {
        this.interceptors.push(fn);
    };

    window.XMLHttpRequest = XHRProxy;

})(window);

Answer 4

You can replace the unsafeWindow.XMLHttpRequest object in the document with a wrapper. A little code (not tested):

var oldFunction = unsafeWindow.XMLHttpRequest;
unsafeWindow.XMLHttpRequest = function() {
  alert("Hijacked! XHR was constructed.");
  var xhr = oldFunction();
  return {
    open: function(method, url, async, user, password) {
      alert("Hijacked! xhr.open().");
      return xhr.open(method, url, async, user, password);
    }
    // TODO: include other xhr methods and properties
  };
};

But this has one little problem: Greasemonkey scripts execute after a page loads, so the page can use or store the original XMLHttpRequest object during it's load sequence, so requests made before your script executes, or with the real XMLHttpRequest object wouldn't be tracked by your script. No way that I can see to work around this limitation.

Answer 5

I spent quite some time figuring out how to do this. At first I was just overriding window.fetch but that stopped working for some reason - I believe it has to do with Tampermonkey trying to sandbox window (??) and I also tried unsafeWindow with the same results.

So. I started looking into overriding the requests at a lower level. The XMLHttpRequest (also that class name upper case lower case ew...) Sean's answer was helpful to get started but didn't show how to override the responses after interception. The below does that:

let interceptors = [];

/*
 * Add a interceptor.
 */
export const addInterceptor = (interceptor) => {
  interceptors.push(interceptor);
};

/*
 * Clear interceptors
 */
export const clearInterceptors = () => {
  interceptors = [];
};


/*
 * XML HTPP requests can be intercepted with interceptors.
 * Takes a regex to match against requests made and a callback to process the response.
 */
const createXmlHttpOverride = (
  open
) => {
  return function (
    method: string,
    url,
    async,
    username,
    password
  ) {
    this.addEventListener(
      "readystatechange",
      function () {
        if (this.readyState === 4) {
          // Override `onreadystatechange` handler, there's no where else this can go.
          // Basically replace the client's with our override for interception.
          this.onreadystatechange = (function (
            originalOnreadystatechange
          ) {
            return function (ev) {
              // Only intercept JSON requests.
              const contentType = this.getResponseHeader("content-type");
              if (!contentType || !contentType.includes("application/json")) {
                return (
                  originalOnreadystatechange &&
                  originalOnreadystatechange.call(this, ev)
                );
              }

              // Read data from response.
              (async function () {
                let success = false;
                let data;
                try {
                  data =
                    this.responseType === "blob"
                      ? JSON.parse(await this.response.text())
                      : JSON.parse(this.responseText);
                  success = true;
                } catch (e) {
                  console.error("Unable to parse response.");
                }
                if (!success) {
                  return (
                    originalOnreadystatechange &&
                    originalOnreadystatechange.call(this, ev)
                  );
                }

                for (const i in interceptors) {
                  const { regex, override, callback } = interceptors[i];

                  // Override.
                  const match = regex.exec(url);
                  if (match) {
                    if (override) {
                      try {
                        data = await callback(data);
                      } catch (e) {
                        logger.error(`Interceptor '${regex}' failed. ${e}`);
                      }
                    }
                  }
                }

                // Override the response text.
                Object.defineProperty(this, "responseText", {
                  get() {
                    return JSON.stringify(data);
                  },
                });

                // Tell the client callback that we're done.
                return (
                  originalOnreadystatechange &&
                  originalOnreadystatechange.call(this, ev)
                );
              }.call(this));
            };
          })(this.onreadystatechange);
        }
      },
      false
    );

    open.call(this, method, url, async, username, password);
  };
};

const main = () => {
  const urlRegex = /providers/; // Match any url with "providers" in the url.

  addInterceptor({
    urlRegex,
    callback: async (_data) => {
      // Replace response data.
      return JSON.parse({ hello: 'world' });
    },
    override: true
  });

  XMLHttpRequest.prototype.open = createXmlHttpOverride(
    XMLHttpRequest.prototype.open
  );
};

main();

Answer 6

Based on proposed solution I implemented 'xhr-extensions.ts' file which can be used in typescript solutions. How to use:

1. Add file with code to your solution

2. Import like this

   import { XhrSubscription, subscribToXhr } from "your-path/xhr-extensions";
   

3. Subscribe like this

   const subscription = subscribeToXhr(xhr => {
     if (xhr.status != 200) return;
     ... do something here.
   });
   

4. Unsubscribe when you don't need subscription anymore

   subscription.unsubscribe();
   

Content of 'xhr-extensions.ts' file

    export class XhrSubscription {

      constructor(
        private callback: (xhr: XMLHttpRequest) => void
      ) { }

      next(xhr: XMLHttpRequest): void {
        return this.callback(xhr);
      }

      unsubscribe(): void {
        subscriptions = subscriptions.filter(s => s != this);
      }
    }

    let subscriptions: XhrSubscription[] = [];

    export function subscribeToXhr(callback: (xhr: XMLHttpRequest) => void): XhrSubscription {
      const subscription = new XhrSubscription(callback);
      subscriptions.push(subscription);
      return subscription;
    }

    (function (open) {
      XMLHttpRequest.prototype.open = function () {
        this.addEventListener("readystatechange", () => {
          subscriptions.forEach(s => s.next(this));
        }, false);
        return open.apply(this, arguments);
      };
    })(XMLHttpRequest.prototype.open);

Answer 7

Not sure if you can do it with greasemonkey, but if you create an extension then you can use the observer service and the http-on-examine-response observer.