I have some code which uses the new ES6 fetch API and nests the fetch calls. The first fetch gets a document, which contains some anchors. And for each anchor I do another fetch. This is the code.
I do a fetch, take the response text, parse it as HTML and pass the DOM to another function.
I select some anchors from the document and for each I check, if it is suitable and then I do the first logging. This logs six different URLs.
Now I do for each of the six URLs a nested fetch. The code is almost the same as for the first fetch. I just log the response URL to make sure, that I have different responses. This second logging shows also 6 different URLs. Once again I parse the response text and pass the DOM to another function.
And now it gets funny. The first thing I do with my DOM is printing the document URL. And this prints six times the same URL. Why does it print six times the same URL although four lines ago six different URLs have been logged?
There must be some kind of side effect, but I do not understand where it is. I have different responses and I create for each response a new DOM parser. How can there be a side effect?
// 1.
fetch('https://eu.battle.net/d3/de/item/two-handed/')
.then(response => response.text())
.then(text => (new DOMParser()).parseFromString (text, 'text/html'))
.then(doc => {
// 2.
let urls = [];
doc.querySelectorAll('li.open a')
.forEach(node => {
if (!node.href.match(/flail|mighty-weapon|scythe/g)) {
console.log ("1: " + node.href); // Prints 6 different URLs.
// 3.
fetch(node.href)
.then(response => {
console.log ("2: " + response.url); // Prints 6 different URLs.
return response.text()
})
.then(text => (new DOMParser()).parseFromString (text, 'text/html'))
.then(doc => {
// 4.
console.log ("3: " + doc.URL); // Prints first URL six times.
doc.querySelectorAll('.top .ui-pagination a')
.forEach(node => {
urls.push(node.href);
});
});
}
});
return urls;
});
