61

It seems to be difficult problem (or impossible??). I want to get and read HTTP Response, caused by HTTP Request in browser, under watching Chrome Extension background script. We can get HTTP Request Body in this way

chrome.webRequest.onBeforeRequest.addListener(function(data){
    // data contains request_body
},{'urls':[]},['requestBody']);

I also checked these stackoverflows

Is there any clever way to get HTTP Response Body in Chrome Extension?

Community
  • 1
  • 1
otiai10
  • 4,289
  • 5
  • 38
  • 50

5 Answers5

35

I can't find better way then this anwser.

Chrome extension to read HTTP response

The answer told how to get response headers and display in another page.But there is no body info in the response obj(see event-responseReceived). If you want to get response body without another page, try this.

var currentTab;
var version = "1.0";

chrome.tabs.query( //get current Tab
    {
        currentWindow: true,
        active: true
    },
    function(tabArray) {
        currentTab = tabArray[0];
        chrome.debugger.attach({ //debug at current tab
            tabId: currentTab.id
        }, version, onAttach.bind(null, currentTab.id));
    }
)


function onAttach(tabId) {

    chrome.debugger.sendCommand({ //first enable the Network
        tabId: tabId
    }, "Network.enable");

    chrome.debugger.onEvent.addListener(allEventHandler);

}


function allEventHandler(debuggeeId, message, params) {

    if (currentTab.id != debuggeeId.tabId) {
        return;
    }

    if (message == "Network.responseReceived") { //response return 
        chrome.debugger.sendCommand({
            tabId: debuggeeId.tabId
        }, "Network.getResponseBody", {
            "requestId": params.requestId
        }, function(response) {
            // you get the response body here!
            // you can close the debugger tips by:
            chrome.debugger.detach(debuggeeId);
        });
    }

}

I think it's useful enough for me and you can use chrome.debugger.detach(debuggeeId)to close the ugly tip.

sorry, mabye not helpful... ^ ^

Tonio Liebrand
  • 17,189
  • 4
  • 39
  • 59
liyangready
  • 366
  • 4
  • 6
  • 2
    use `--silent-debugger-extension-api` command line switch to lunch the browser to get rid of yellow bar , it must be the first instance of the browser – Pranoy Sarkar Nov 02 '17 at 15:53
  • 1
    I keep getting undefined for response, do you have any idea? – Romulus Urakagi Ts'ai Dec 24 '17 at 14:30
  • I found the reason is "No resource with given identifier found" (requestId of getResponseBody is wrong), but I have no idea why it is wrong :( – Romulus Urakagi Ts'ai Dec 24 '17 at 15:10
  • 3
    I made a content script from the above code, and added a console.log and alert in the response body part, but nothing happens. Where should the code be placed? – user5155835 Jan 02 '18 at 09:16
  • 5
    I'm getting the error cannot 'attach' to chrome://url. Anyway around this and only attach to specific urls? – evanjmg Apr 18 '18 at 15:25
  • 4
    is this ran from a background script or from a content script? – spy May 28 '18 at 01:19
  • 4
    @evanjmg You can query the url if you put "tabs" in the permissions array in `manifest.json`. In `chrome.tabs.query`, use `function (tabArray) { currentTab = tabArray[0]; if(!currentTab.url.startsWith("chrome:")){ chrome.debugger.attach({ //debug at current tab tabId: currentTab.id }, version, onAttach.bind(null, currentTab.id)); } }` – Paul Feb 23 '19 at 05:58
  • hello! were you able to catch all the requests this way? I wrote a script to catch all requests, it works fine with iframes, but not all requests are there.. – spielerds Jun 23 '19 at 14:18
  • 2
    Hi, If any one need more commands to get network information follow this link https://chromedevtools.github.io/devtools-protocol/tot/Network – Ernesto Alfonso Mar 31 '20 at 10:26
10

There is now a way in a Chrome Developer Tools extension, and sample code can be seen here: blog post.

In short, here is an adaptation of his sample code:

chrome.devtools.network.onRequestFinished.addListener(request => {
  request.getContent((body) => {
    if (request.request && request.request.url) {
      if (request.request.url.includes('facebook.com')) {

         //continue with custom code
         var bodyObj = JSON.parse(body);//etc.
      }
    }
  });
});
URL87
  • 10,667
  • 35
  • 107
  • 174
JohnP2
  • 1,899
  • 19
  • 17
  • 26
    From the article itself: The drawback for this method is that we have to keep the Chrome DevTools open all the time because DevTools extensions are only activated when DevTools is open. – mikey Nov 11 '20 at 05:19
9

This is definitely something that is not provided out of the box by the Chrome Extension ecosystem. But, I could find a couple of ways to get around this but both come with their own set of drawbacks.

The first way is:

  1. Use a content script to inject our own custom script.
  2. Use the custom script to extend XHR's native methods to read the response.
  3. Add the response to the web page's DOM inside a hidden (not display: none) element.
  4. Use the content script to read the hidden response.

The second way is to create a DevTools extension which is the only extension that provides an API to read each request.

I have penned down both the methods in a detailed manner in a blog post here.

Let me know if you face any issues! :)

Tarun Dugar
  • 8,921
  • 8
  • 42
  • 79
  • Your article doesn't say how to send response to the background page, because injecting 'chrome.runtime.sendMessage' doesn't work. – Andrew Mar 03 '19 at 21:27
  • @Andrew ill check, the article was meant for just reading response body. Can I ask where you are using 'chrome.runtime.sendMessage'? – Tarun Dugar Mar 04 '19 at 11:22
  • I was trying to use it inside that injected content script, but it (obviously) doesn't work. Article mentioned that 'you can pass to BG' so it would be good to know how you can do that. – Andrew Mar 05 '19 at 20:47
  • @Andrew you can use `chrome.runtime.sendMessage` inside a content script: https://developer.chrome.com/extensions/content_scripts. Probably, something else is going wrong. – Tarun Dugar Mar 06 '19 at 05:46
  • You cannot use chrome messaging api inside injected script the way article injects it with a self running function. There is no messaging between that script and extension. – Andrew Mar 06 '19 at 09:50
  • @Andrew yes, because the injected script is not the content script. The way you can achieve this is by using appending the data from the injected script to a DOM element and then using the content script to access that data by reading the corresponding DOM element. I hope this clears it up? – Tarun Dugar Mar 09 '19 at 05:28
  • Yes, that's the way. Also the content will be added 'dynamically' so when content script tries to 'read it' this should be taken into account. – Andrew Mar 09 '19 at 09:36
  • @Andrew true, for that you can call a function inside the content script continuously wrapped inside a requestIdleCallback to avoid the browser from crashing. The function would just check for the contents of the DOM element where we are going to append the intercepted data. – Tarun Dugar Mar 09 '19 at 10:09
  • with the arrival of chrome extension manifest v3 and their restriction to inject JS inline scripts, what are the solutions now, looking into the future? – Andrew Apr 10 '21 at 16:16
5

To get a XHR response body you can follow the instructions in this answer.

To get a FETCH response body you can check Solution 3 in this article and also this answer. Both get the response body without using chrome.debugger.

In a nutshell, you need to inject the following function into the page from the content script using the same method used for the XHR requests.

const constantMock = window.fetch;
 window.fetch = function() {
    return new Promise((resolve, reject) => {
        constantMock.apply(this, arguments)
            .then((response) => {
                if (response) {
                    response.clone().json() //the response body is a readablestream, which can only be read once. That's why we make a clone here and work with the clone
                    .then( (json) => {
                        console.log(json);
                        //Do whatever you want with the json
                        resolve(response);
                    })
                    .catch((error) => {
                        console.log(error);
                        reject(response);
                    })
                }
                else {
                    console.log(arguments);
                    console.log('Undefined Response!');
                    reject(response);
                }
            })
            .catch((error) => {
                console.log(error);
                reject(response);
            })
    })
}

If response.clone().json() does not work, you can try response.clone().text()

General Grievance
  • 4,555
  • 31
  • 31
  • 45
Kalo
  • 103
  • 1
  • 7
  • The question is about interception of all requests from the page with extension. Your answer is about interception of own requests. Own requests with fetch, for example, could be processed with ".then" function easily. – Sergey NN Mar 30 '23 at 07:53
  • @SergeyNN My answer is about interception of all requests. Read carefully and try to understand! – Kalo Mar 31 '23 at 09:37
1

I show my completed code if it can be some help. I added the underscore to get the request url, thanks

//background.js
import _, { map } from 'underscore';

var currentTab;
var version = "1.0";

chrome.tabs.onActivated.addListener(activeTab => {
    currentTab&&chrome.debugger.detach({tabId:currentTab.tabId});
    currentTab = activeTab;
    chrome.debugger.attach({ //debug at current tab
        tabId: currentTab.tabId
    }, version, onAttach.bind(null, currentTab.tabId));
});

function onAttach(tabId) {
    chrome.debugger.sendCommand({ //first enable the Network
        tabId: tabId
    }, "Network.enable");
    chrome.debugger.onEvent.addListener(allEventHandler);
}

function allEventHandler(debuggeeId, message, params) {
    if (currentTab.tabId !== debuggeeId.tabId) {
        return;
    }
    if (message === "Network.responseReceived") { //response return
        chrome.debugger.sendCommand({
            tabId: debuggeeId.tabId
        }, "Network.getResponseBody", {
            "requestId": params.requestId
            //use underscore to add callback a more argument, passing params down to callback
        }, _.partial(function(response,params) {
            // you get the response body here!
            console.log(response.body,params.response.url);
            // you can close the debugger tips by:
            // chrome.debugger.detach(debuggeeId);
        },_,params));
    }
}

I also find there is a bug in chrome.debugger.sendCommand. If I have two requests with same URI but different arguments. such as:

The second one will not get the corrected responseBody, it will show:

Chrome Extension: "Unchecked runtime.lastError: {"code":-32000,"message":"No resource with given identifier found"}

But I debugger directly in background devtools, it get the second one right body.

chrome.debugger.sendCommand({tabId:2},"Network.getResponseBody",{requestId:"6932.574"},function(response){console.log(response.body)})

So there is no problem with tabId and requestId. Then I wrap the chrome.debugger.sendCommand with setTimeout, it will get the first and second responseBody correctly.

if (message === "Network.responseReceived") { //response return            
console.log(params.response.url,debuggeeId.tabId,params.requestId)
        setTimeout(()=>{
            chrome.debugger.sendCommand({
                tabId: debuggeeId.tabId
            }, "Network.getResponseBody", {
                "requestId": params.requestId
                //use underscore to add callback a more argument, passing params down to callback
            }, _.partial(function(response,params,debuggeeId) {
                // you get the response body here!
                console.log(response.body,params.response.url);
                // you can close the debugger tips by:
                // chrome.debugger.detach(debuggeeId);
            },_,params,debuggeeId));
        },800)
    
}

I think the setTimeout is not the perfect solution, can some one give help? thanks.

Anson Hwang
  • 117
  • 1
  • 8