Acessing Network Request and Response

Question

Am trying to get all the sub-resources or urls or fetch/xhr (i hope am communicating well) from a webpage.

I've tried using the chrome.webRequest API but it did'nt get me all the sub-resources/fetch/xhr called on the page unlike when i tried using chrome.debugger which gets me exactly what I want, allow me to showcase the code.

chrome.webRequest.onBeforeRequest.addListener(
    (details) => {
        console.log("all resource URL:", details.url, details.initiator, "\n");
    },
    { urls: ["<all_urls>"] }
);

chrome.action.onClicked.addListener((tab) => {
    console.log("action button has been clicked\n")
    chrome.debugger.attach({ tabId: tab.id }, '1.0', () => {
        // Send the Network.enable command to enable network tracking
        chrome.debugger.sendCommand({ tabId: tab.id }, 'Network.enable', {}, () => {
            // Add an event listener for the Network.requestWillBeSent event
            chrome.debugger.onEvent.addListener((source, method, params) => {
                if (source.tabId === tab.id && method === 'Network.responseReceived') {
                    // Add the request to the list of sub-resource fetches
                    if (params.response.mimeType === 'application/json' && params.response.url.includes("url")) {
                        console.log(params.response, "\n")
                        console.log(params.response.url)
                    }
                }
            });
        });
    });
})

And yes, unlike the webRequest version I have narrowed my search in the chrome.debugger to json responses, which is actually what I want but the debugging warning would not be nice for the users. So here are my troubles;

Is there a way, still using the working solution of chrome.debugger to get away from the warnings, cos that essentially the problem with that approach. I don't suppose the users would like that, right?

2a) I prefer the chrome.webRequest approach, very clean, but how do i make it give me all fetch/xhr like i see on my network tab?

2b) I did notice it fetched me a url. Which is actually a js script, which is the initiator for the url i want, how to i get down into the script to get that particular url. I saw it in the "Request Iniator chain" of the js script.

If I'm getting it right. You want to get all the urls of resources loaded by a specific page? — doc-han, Aug 10 '23 at 14:53

score -1 · Answer 1 · answered Aug 10 '23 at 14:52

onBeforeRequest should be able to do what you need. However, you need to ensure your extension has all the necessary permissions. This includes the "webRequest" permission to access the API, plus host permissions which let you intercept specific URLs.

For example, if you need to intercept requests on all websites, then your manifest file might contains properties like this:

{
  "permissions": ["webRequest"],
  "host_permissions": ["*://*/*"]
}

If possible, you should narrow down host_permissions to only those sites you actually need. However, for sub-resources, it's important to note that you need permission for the URL being requested and the URL which initiated the request.

Acessing Network Request and Response

1 Answers1