HTTP fetch info IN ORDER and add to array in the order specified

Question

I have tried it all. I want to fetch from my server some data in a loop, and add the ressponses in an array in the order I run the for loop.

I am thinking the answer is in recursion, but tried it several times and failed.

I am using AXIOS:

let currentPhraseWordAssign = [];

function addPartOfSpeechAndWordPhrasesToFullScript () {
    assignPartOfSpeechAndAddToPhraseArray(["phrase", "run"]).then(() => {console.log(currentPhraseWordAssign)});
}


async function assignPartOfSpeechAndAddToPhraseArray (phrase) {
    if(currentPhraseWordAssign.length >= phrase.length) {
        console.log(currentPhraseWordAssign);
        currentPhraseWordAssign = [];
        return;
    }
    let word = phrase[currentPhraseWordAssign.length];
    axios({
        method: 'get',
        url: `http://localhost:9200/english_minus_verbs/_search`,
        headers: {
            'Content-Type': 'application/json'
        },
        data: {
            _source: ["part_of_speech"],
            query: {
                term: {
                    word: word,
                }
            }
        }
    }).then((r) => {
        try {
            currentPhraseWordAssign.push({word: word, partOfSpeech: r.data.hits.hits[0]._source.part_of_speech});
        }catch (e) {
            currentPhraseWordAssign.push({word: word, partOfSpeech: 'verb'});
        }
    }).then(()=>{
        assignPartOfSpeechAndAddToPhraseArray(phrase);
    });
}

Maybe that code is completely wrong, it is anyway like the 7th time I write it... I tried with promises, async/await, sync, and now recursion...

Are you sure you're meant to be sending a GET request with payload? That's not typical — Phil, Jun 16 '22 at 02:17
@Phil I am using elasricsearch (opensearch), the API uses a body in this case. — Xavi Font, Jun 16 '22 at 02:18
@Phil Good to know, I will look into it and see if it is better in some use cases. I was rying to get data atm but maybe POST can help with getting data somehow I guess. — Xavi Font, Jun 16 '22 at 02:45
If you ever want to make requests from a browser, you won't be able to use the GET method. Just something to keep in mind — Phil, Jun 16 '22 at 02:47

jfriend00 · Accepted Answer · 2022-06-16T02:24:44.863

You can run the axios calls in sequence (one after the other) like this:

async function lookupWords(arrayOfWords) {
    let currentPhraseWordAssign = [];
    for (let word of arrayOfWords) {
        const data = await axios({
            method: 'get',
            url: `http://localhost:9200/english_minus_verbs/_search`,
            headers: {
                'Content-Type': 'application/json'
            },
            data: {
                _source: ["part_of_speech"],
                query: {
                    term: {
                        word: word,
                    }
                }
            }
        });
        currentPhraseWordAssign.push({word: word, partOfSpeech: r.data.hits.hits[0]._source.part_of_speech});        
    }
    return currentPhraseWordAssign;
}

Or, you can run them in parallel and collect the results in order:

function lookupWords(arrayOfWords) {
    return Promise.all(arrayOfWords.map(word => {
        return axios({
            method: 'get',
            url: `http://localhost:9200/english_minus_verbs/_search`,
            headers: {
                'Content-Type': 'application/json'
            },
            data: {
                _source: ["part_of_speech"],
                query: {
                    term: {
                        word: word,
                    }
                }
            }
        }).then(r => {
            return {word: word, partOfSpeech: r.data.hits.hits[0]._source.part_of_speech};
        });
    }));
}

If you run them in parallel (which may finish sooner), you have to make sure that the target server will accept however many simultaneous requests as your array is long. Some servers will limit or reject too many simultaneous requests. To run semi-in-parallel where you have N requests in flight at the same time (how you would process a large array in parallel without too many requests in flight at once), you would use something like mapConcurrent() to control the execution.

What a wizard! That is so good, it worked out of the box... One question... does the "for (let word of arrayOfWords) {} " do anything better than a regular for loop or for each? I have seen it multiple times on async functions... any reason why that is that way? — Xavi Font, Jun 16 '22 at 02:37
@XaviFont - `for/of` is just a more modern way to run a `for` loop when the thing you are iterating is an "iterable" such as an array. It has much more flexible flow of control than `.forEach()` and `.forEach()` is not `async` aware so it wouldn't work at all with the first code block here. — jfriend00, Jun 16 '22 at 02:57

HTTP fetch info IN ORDER and add to array in the order specified

1 Answers1