17

I've been using d3.js v4 for sometime now and I've learned that Mike Bostock has replaced the d3.queue in the v5 release with the Promise native JavaScript object. I would like to check with you if this code that I have written is properly queuing (asynchronously) these URL's:

var makeRequest = function() {
    "use strict";

    var bli = [
        "http://stats.oecd.org/sdmx-json/data/BLI2013/all/all",
        "http://stats.oecd.org/sdmx-json/data/BLI2014/all/all",
        "http://stats.oecd.org/sdmx-json/data/BLI2015/all/all",
        "http://stats.oecd.org/sdmx-json/data/BLI2016/all/all",
        "http://stats.oecd.org/sdmx-json/data/BLI/all/all"
    ];

    var promises = [];

    bli.forEach(function(url) {
        promises.push(
            new Promise(function(resolve, reject) {
                d3
                    .json(url)
                    .then(function(response) {
                        resolve(response);
                    })
                    .catch(function(error) {
                        console.log("Error on: " + url + ". Error: " + error);
                        reject(error);
                    });
            })
        );
    });

    Promise.all(promises).then(function(values) {
        console.log(values);
    });
};

makeRequest();

The code seems to function properly, but, is this proper code or is there a better way (a best practice approach) for queuing with Promise.all and d3.js? Is the catch error properly implemented?

Gerardo Furtado
  • 100,839
  • 9
  • 121
  • 171
Marco A. Ferra
  • 193
  • 1
  • 1
  • 7
  • 5
    I just rolled back this question, removing the `d3v5` tag. To the editor, please read this: https://meta.stackoverflow.com/a/338840/5768908. We had a lot of work removing the `d3v4` tag, and apparently this process will start again with this new `d3v5` tag! This is the recommended action: questions should have only the [tag:d3.js] tag, regardless the version. Then, if the question is specific to a given version, OP has to specify the version on the title or on the question's body. – Gerardo Furtado Mar 28 '18 at 13:08

3 Answers3

27

You can simplify that code a lot: you don't net to use new Promise with d3.json, since d3.json will itself create the promise.

So, you can just do:

var files = ["data1.json", "data2.json", "data3.json"];
var promises = [];

files.forEach(function(url) {
    promises.push(d3.json(url))
});

Promise.all(promises).then(function(values) {
    console.log(values)
});

Or, if you're into the code golf, even shorter:

var files = ["data1.json", "data2.json", "data3.json"];

Promise.all(files.map(url => d3.json(url))).then(function(values) {
    console.log(values)
});

Since I cannot use JSON files in the S.O. snippet, check the console in this bl.ocks: https://bl.ocks.org/GerardoFurtado/f08993c9c729b0b3452ef1803ad9dcbf/c4b45c5acce6033085a667cbb7d34203d15de0f0

Gerardo Furtado
  • 100,839
  • 9
  • 121
  • 171
  • Excellent! But, if if a single URL gives an error - if it doesn't exist for example - the Promise doesn't return the results from the other URL's, even if they are correct (your code and mine suffer from this). Is there a way to return the values from the URL's that returned data or the Promise.all function assumes that there must not be any error? – Marco A. Ferra Mar 28 '18 at 12:46
  • @MarcoA.Ferra That's a different problem, related specifically to `Promise.all()`, and not related to D3. So, to avoid your question being "too broad", I suggest you to post this issue as a *new* question, with the `promise` tag and **without** the `d3.js` tag. – Gerardo Furtado Mar 28 '18 at 12:49
  • I'll do that. Thank you for the right answer above. – Marco A. Ferra Mar 28 '18 at 12:52
  • @MarcoA.Ferra No worries. There is also the **[tag:es6-promise]** tag, use that as well, together with **[tag:javascript]** of course. – Gerardo Furtado Mar 28 '18 at 12:53
  • @MarcoA.Ferra For that you might try p-queue https://github.com/sindresorhus/p-queue – Union find Mar 28 '18 at 16:17
  • how would this work with a mix of filetypes? ie json, js, csv... thanks. – v3nt Oct 21 '20 at 18:09
  • 1
    @v3nt that's too complex for answering in a comment, I suggest you post a new question. – Gerardo Furtado Oct 21 '20 at 21:57
1

Here's an approach with ES6 async functions and ES6 array destructuring:

async function chart() {
  const [first, second] = await Promise.all([
    d3.json('data1.json'),
    d3.json('data2.json'),
  ])
  console.log('data2.json :', second)
}

chart()
xy2
  • 6,040
  • 3
  • 14
  • 34
0

You can also add a formatting function for your data as such if you want to clean up your data to your preference. .then() will have your data in a nice array which you can use later.

const myData = d3.csv("data.csv", formatterFunction)
.then(data => /* do whatever*/ )

function formatterFunction(row){
  // do formatting
  return // formatted data
   }
theBird
  • 31
  • 3