Multiple URLs copy in Sources/Network tab

Question

Is it possible to extract multiple resources' URLs in Sources or Network tab of Chrome Dev Tools?
When I want to get URL of a single resource, I can do it with context menu function Copy link address

I can switch to this resource from Network to Sources tab and vice versa, but what if I have a need to get URLs of multiple resources at once? It is very cumbersome to copy them manually if resultset consists of 200-300 resources.

What I've tried so far:

To copy the whole folder from a sources tab, but from this answer I found out it is not possible for now.
To use $(selector) as specified in the Console reference, in a form of
```
$('img')
```
in case we need to fetch image URLs.

The complexity of this approach is that it's often hard to distinguish target images on a page that has hundreds of them, and furthermore, multiple versions of the same image (views, previews, small-sized icons and etc.) I.e. to match the element inside the tag with the needed resource is not that easy, as it seems. Also not all the file types have dedicated tags (as in the case with img).

Maybe I should use src tag with some modifiers? Any other suggestions?

did you try 'copy all as HAR' and then storing the result as a variable in the console then running some simple code like ```copy(a.log.entries.map(e => e.request.url).join('\n'))``` ? — Elisha Habinsky, May 06 '20 at 17:59

wOxxOm · Accepted Answer · 2023-01-07T09:16:48.340

61

make sure Network panel is active
switch devtools Dock side in the menu to a detached (floating) window

Next time you can press CtrlShiftD to toggle docking.

in the now detached devtools press CtrlShifti or ⌘⌥i on MacOS,
which will open devtools-on-devtools in a new window

Run the following code in this new window:

copy(UI.panels.network.networkLogView.dataGrid.rootNode().flatNodes.map(n => n.request().url()).join('\n'))

It'll copy the URLs of all requests that match current filter to clipboard.

Hint: save the code as a Snippet and run it in devtools-on-devtools window via the commands palette, CtrlP or ⌘P then type the snippet's name.

In old Chrome the code was different:

copy(UI.panels.network._networkLogView._dataGrid._rootNode._flatNodes.map(n => n._request._url).join('\n'))
copy(UI.panels.network.networkLogView.dataGrid.rootNode().flatNodes.map(n => n.request().urlInternal).join('\n'))

edited Jan 07 '23 at 09:16

answered Dec 17 '16 at 19:14

wOxxOm

65,848
11
132
136

Excellent! That's exactly what I want. – Suncatcher Dec 18 '16 at 11:19
I added the variant with `_flatNodes` to your answer, as only displayed nodes are not very useful. When we have 200+ requests they surely cannot be places within single screen area and one should run this snippet N times moving scroll to the next portion, however `_flatNodes` do it all. – Suncatcher Dec 18 '16 at 11:21
1

@Pacerier, No, it's a "hack". I've just examined the [source code](https://cs.chromium.org/chromium/src/third_party/WebKit/Source/devtools/front_end/) and the actual values in devtools-for-devtools. – wOxxOm Oct 11 '17 at 18:40
@wOxxOm, You mean there's no tutorial page at all and you just found out there's a devtools for devtools all by yourself? – Pacerier Oct 11 '17 at 18:42
1

The devtools-for-devtools invocation spell itself was mentioned somewhere. – wOxxOm Oct 11 '17 at 18:43
@wOxxOm, Btw is there a way to get the actual HTTP response body using script? I'd only see `_responseHeaderValues` , `_responseHeaders`, `_responseHeadersText`, `_responseReceivedTime`. – Pacerier Oct 11 '17 at 19:05
1

As per the source code, it's `Promise.all(UI.panels.network._networkLogView._dataGrid._rootNode._flatNodes.map(node => node._request.contentData().then(data => data.content))).then(console.log)` – wOxxOm Oct 11 '17 at 19:18
Note, copypasting from comments includes nonprintable characters. – wOxxOm Oct 11 '17 at 19:19
3

on one computer - the accepted answer works, on another - it fails, both seem to be running up to date chrome. 'ReferenceError - UI is not defined'. any idea why? – Joe Jul 20 '18 at 13:03
@Joe, sounds like you didn't invoke devtools-for-devtools. – wOxxOm Jul 20 '18 at 13:06
yes, I did do Ctrl-Shift-i, but wondered whether it did the right thing. is there a way to make sure I am running devtools-for-devtools? the console looks similar in both cases. I wondered it's some extension I failed to install on one pc but not the other...? – Joe Jul 20 '18 at 13:11
The exact sequence is described in this answer. In the end you'll have two devtools windows. – wOxxOm Jul 20 '18 at 13:18
1

@wOxxOm thanks, I got it working, having two devtools windows was what I was missing. – Joe Jul 20 '18 at 16:25
In Chrome on OSX, the keyboard shortcut to open devtools-for-devtools is Cmd-Option-i (Cmd-Alt-i) – Tom Jul 04 '19 at 12:13
It used to work but now I am getting "Uncaught TypeError: Cannot read properties of undefined (reading '_dataGrid') at :1:40" – user1156544 Nov 20 '21 at 16:24
4

For me in the latest chrome I had to do it like this: `copy(UI.panels.network.networkLogView.dataGrid.rootNode().flatNodes.map(n => n.request().url()).join('\n'))` – frankhommers Dec 08 '21 at 21:26
It was useful for me to filter out data urls also: copy(UI.panels.network.networkLogView.dataGrid.rootNode().flatNodes.map(n => n.request().url()).filter(url => !url.startsWith('data:')).join('\n')) – Justin Wignall Apr 25 '22 at 14:33
here is the one that worked for me: `UI.panels.network.networkLogView.dataGrid.rootNode().children.map(n => n.request().url()).join('\n')` – Raphael Apr 14 '23 at 16:48

score 2 · Answer 2 · answered Nov 22 '18 at 02:15

I found the above method too clunky, its way easier to use fiddler:

open fiddler (possibly install it)
filter on the domain name that you are interested in
clear the screen
refresh the web page
Select the fiddler output
right click and select copy just the URL's

score 0 · Answer 3 · answered Dec 01 '20 at 01:32

Based on @wOxxOm, this worked for me:

var nodes = UI.panels.network._networkLogView._dataGrid._rootNode._flatNodes,
    urls = [];
nodes.forEach(function() {
    var req = arguments[0]._request;
    if (req !== undefined) {
        urls.push(req.url());
    }
});

score 0 · Answer 4 · answered Mar 04 '22 at 14:25

0

A selection and simple copy (Ctrl+C) work for me. I select URLs in the Url column by the mouse. Then I use the context menu to copy the list to the clipboard. The clipboard contents then I can paste to Excel and get the URL list. It adds some empty lines though.

answered Mar 04 '22 at 14:25

Sergey Beloglazov

60
5

Works only for visible urls when selecting in multipage list. Ctrl+A solves this but adds garbage to selection – Estus Flask Aug 29 '22 at 08:54

score 0 · Answer 5 · answered Mar 07 '23 at 12:26

0

Based on @wOxxOm and @onassar. This worked for March 2023.

window.nodes = UI.panels.network.networkLogView.dataGrid.rootNode().flatNodes;
window.urls = [];
window.nodes.forEach(function() {
    let req_ = arguments[0].request();
    if (req_ !== undefined /*&& req_.statusCode === 404*/) {
        window.urls.push(req_.url());
    }
});
copy(urls.join('\n'));  //copy to clipboard

answered Mar 07 '23 at 12:26

xexsus

1

As it’s currently written, your answer is unclear. Please [edit] to add additional details that will help others understand how this addresses the question asked. You can find more information on how to write good answers [in the help center](/help/how-to-answer). – Community Mar 13 '23 at 02:33

Multiple URLs copy in Sources/Network tab

5 Answers5

Linked

Related