How to get list of network requests done by HTML

Question

How can i get the list of network requests using Javascript done by the HTML, as seen in the chrome devtools.

For example: enter image description here

This is the devtools for google.com. I want to, using javascript get all these request in a list. is this possible? if yes how?

It's part of a website, the reference to the chrome devtools was simply to explain what kind of info i want to get in the javascript... — Nachshon Schwartz, Dec 16 '13 at 21:23
you can get some of them from the DOM, like images and css images, and css sheet urls, iframes, flash, etc, others like ajax are unreachable from js. — dandavis, Dec 16 '13 at 21:28

jfriend00 · Accepted Answer · 2016-11-03T06:34:16.827

28

Some browsers have implemented a version of the not-yet-standard, Resource Timing API where you can collect some of this information.

Some browsers may have some of this info available to browser extensions as part of their developer tools support, but that would require the installation of a custom extension, not something that could be done from a regular web page.

For very specific operations where you control the calling code or you know the calling code, it is possible to instrument some things. For example, if you know that all ajax calls go through one particular function, you can hook that function and it's completion handlers and monitor all ajax calls.

edited Nov 03 '16 at 06:34

answered Dec 16 '13 at 21:28

jfriend00

683,504
96
985
979

1

This is correct. Any sort of inspection at this level would have to be done in an extension (at least in the case of Chrome). – Matt Dec 16 '13 at 22:07
still no way to do it without a browser extension? even with something like node? – oldboy Jan 19 '20 at 05:13
@oldboy - It depends upon what you're really looking for. You can use node.js and get a parser like cheerio and parse the HTML to discover all the references to resources that are in the page. Or, you could even use something like puppeteer to run the page (including Javascript) and then examine the content. You won't be access the Chrome network page from node. Perhaps you should fully describe your problem and what you're trying to achieve in a new question? – jfriend00 Jan 19 '20 at 05:43
im looking to actually obtain all of the individual files that are served to the client. i cant even open a webpage within, say, an electron app's window, and then obtain (via somescript.js) the files that are served thereto? i created a question [here.](https://stackoverflow.com/questions/59807327/get-access-to-or-acquire-files-on-network-tab-in-developer-console/59807436?noredirect=1#59807436) please feel free to share your thoughts and knowledge!! <3 – oldboy Jan 19 '20 at 06:29
I'm not really interested in the extension, my interest is about watching over network requests, from my own extension or from an internal script. See https://stackoverflow.com/questions/72585423/is-it-possible-to-watch-over-xhr-fetch-requests-happening-in-the-browser for more details – Vadorequest Jun 11 '22 at 16:16

score 14 · Answer 2 · answered Jun 07 '18 at 03:51

14

As I understand it, you can consult the list of requests via JavaScript. It is? "I do not know how."

But one solution that can help is this ...

You intercept all requisitions with the code below. If your JavaScript runs very early in loading the page you will be able to get most of the requests from the list.

See how cool this article.

XMLHttpRequest.prototype.realSend = XMLHttpRequest.prototype.send;
XMLHttpRequest.prototype.send = function(value) {
    this.addEventListener("progress", function(){
        console.log("Loading. Here you can intercept...");
    }, false);
    this.realSend(value);
};

answered Jun 07 '18 at 03:51

Sergio Cabral

6,490
2
35
37

Wouldn't it be necessary to remove the event listener once the request ended? – Xoundboy Aug 07 '20 at 11:03
I understand that it is not necessary. Because after reaching the final status, the instance does not trigger any more events. – Sergio Cabral Apr 15 '21 at 13:04
1

Use `this.realSend.apply(this, arguments);` to call with all arguments provided – Ethan O'Brien May 12 '22 at 17:00

score 12 · Answer 3 · answered Dec 19 '13 at 15:37

12

You can use Resource Timing API to get all relevant information (domain lookups, cache hits, redirects, etc.) about each resource being loaded on your website.

You can read about it here. There is also a bookmarklet that generates a page load waterfall using this API.

Resource Timing API is available in Chrome, Chromium, Chrome Mobile and IE10. Firefox team seems to be working on it.

answered Dec 19 '13 at 15:37

Konrad Dzwinel

36,825
12
98
105

6

I find your answer more useful than just saying, "It can't be done". Thanks. – JohnnyJS Feb 25 '16 at 10:35

score 12 · Answer 4 · edited May 10 '20 at 17:11

I have written the code using the Resource Timing API

function captureNetworkRequest(e) {
    var capture_network_request = [];
    var capture_resource = performance.getEntriesByType("resource");
    for (var i = 0; i < capture_resource.length; i++) {
        if (capture_resource[i].initiatorType == "xmlhttprequest" || capture_resource[i].initiatorType == "script" || capture_resource[i].initiatorType == "img") {
            if (capture_resource[i].name.indexOf('www.demo.com OR YOUR URL') > -1) {
                capture_network_request.push(capture_resource[i].name)
            }
        }
    }
    return capture_network_request;
}

score 2 · Answer 5 · answered Dec 16 '13 at 21:59

2

You could get the URLs of requests to be made when the page loads but retrieving any sort of statistics on load times is unrealistic. Query elements which make these kind of resource requests such as script, link or img.

For example:

var urls = Array.prototype.map.call(
    document.querySelectorAll("link, img, script, iframe"), // Elements which request external resources
    function(e) { // Loop over and return their href/src
        return e.href || e.src; 
    }
);

answered Dec 16 '13 at 21:59

AdrianCooney

727
5
13

Well, you'd of course have to repeat the requests if you wanted any real statistics about them. – AdrianCooney Dec 16 '13 at 22:05
I meant redirects done by the elemnts themselves... like if the source of an image is a page that redirects to the image – Nachshon Schwartz Dec 16 '13 at 22:11

How to get list of network requests done by HTML

5 Answers5

Linked