What's the correct way to iterate through array and sending multiple ajax requests

Question

I currently have some ajax that sends an array to php for processing. The array size at most would be around 1000 elements which will be enough for the server to time out while the php is processing. I would like to chunk the array and send it in batched ajax requests (or maybe just one at a time) that will each finish without timing out.

Below is my basic code structure. I'm trying to use promises and iterate through the chunks:

Here's the basic structure of the working function (non-promise version):

function ajax_function(big_array) {
    $.ajax({
        type: 'POST',
        url: ajax.ajax_url,
        data: {
            'action': 'process_big_array',
            'array_to_process': big_array
        },
        beforeSend: function () {
            xxx
        },
        dataType: 'json',
        success: process_big_array_chunk_handler,
        error: function (xhr, desc, err) {
            console.log(xhr);
            console.log('Details: ' + desc + '\nError: ' + err);
        }
    });
}

Here's work in progress trying to add the loop and promise.

function ajax_function(big_array) {
    var big_array_chunks = make it into chunks somehow;
    $.each(big_array_chunks, function (big_array_chunk) {
        var request = $.ajax({
            type: 'POST',
            url: ajax.ajax_url,
            data: {
                'action': 'process_big_array_chunk',
                'array_to_process': big_array_chunk
            },
            dataType: 'json'
        }).done(function (data) {
            process_big_array_chunk_handler(data);
        }).fail(function (xhr, desc, err) {
            console.log(xhr);
            console.log('Details: ' + desc + '\nError: ' + err);
        });
    });
}

Trying to get my head around how this all fits together:

each iteration of loop
responses from php
how to use my response handler when it's all done

fyi, this is being done within the context of WordPress if that matters.

Response to Bergi:

function ajax_function(big_array) {
    var big_array_chunks = make big_array into chunks somehow;
    var request = $.ajax({
        type: 'POST',
        url: ajax.ajax_url,
        data: {
            'action': 'process_big_array_chunk',
            'array_to_process': big_array_chunk
        },
        dataType: 'json'
    }).then(function (data) {
        process_big_array_chunk_handler(data);
    });
}

[Don't use `$.each`](https://stackoverflow.com/q/37576685/1048572), and don't use `success`/`error`/`done`/`fail` callbacks put only the promise `then` method — Bergi, Jul 06 '18 at 17:53
@Bergi Thanks I modified code above. Like so? Where will I be doing my iteration through my array chunks? — egauvin, Jul 06 '18 at 18:06
Currently I'm studying this example by @jfriend00 very carefully. It seems to be very promising (no pun intended). https://stackoverflow.com/questions/47125620/using-jquery-deferred-and-promise-inside-loop This might be exactly what I need. Still trying to figure it out. — egauvin, Jul 06 '18 at 19:08
I marked yours a dup of a [different answer of mine](https://stackoverflow.com/questions/33378923/make-several-requests-to-an-api-that-can-only-handle-20-request-a-minute/33379149#33379149) that contains a function named `pMap()` which lets process a big array while keeping only N requests in flight at a time which sounds like it's what you want to do. — jfriend00, Jul 06 '18 at 22:51
All you have to do is wrap your ajax into a function that returns a promise that resolves when the ajax call is done and resolves with the desired value. `pMap()` will then manage when to call it and what data to pass to it and will return you a promise that resolves to an array of results when its all done. Your function should contain `return $.ajax(...)` so it's returning the ajax promise directly. — jfriend00, Jul 06 '18 at 22:52
@jfriend00 THANKS! I will study this and try to get my head around it. For some reason promises confuse me. Thanks for your explanation. Very helpful! This looks like it should be exactly what I need. — egauvin, Jul 07 '18 at 13:25
@jfriend00 I'm trying to figure out how to implement the pMap function. Let's say I have a working ajax that accepts an array and I want to refactor so the ajax runs once for each element of the array and at the end I get my response at for everything together just like the ajax that was taking the whole array. Does this make sense? Sorry to be so dense. — egauvin, Jul 07 '18 at 15:44
I didn't realize your ajax call takes a chunk from the array. That's not exactly what [pMap](https://stackoverflow.com/a/33379149/816620) does so I reopened your question and will work on an answer. — jfriend00, Jul 07 '18 at 16:03
Let us [continue this discussion in chat](https://chat.stackoverflow.com/rooms/174568/discussion-between-egauvin-and-jfriend00). — egauvin, Jul 07 '18 at 19:01

jfriend00 · Accepted Answer · 2018-07-07T16:18:54.303

The simplest solution I can think of is to just pre-chunk the array and then just use the .reduce() design pattern for serializing promises.

function processBigArray(bigArray, chunkSize) {
    // pre-split array into chunks
    var chunkArray = [];
    for (var i = 0; i < bigArray.length; i+=chunkSize) {
        chunkArray.push(bigArray.slice(bigArray, i, chunkSize));
    }

    var results = [];

    // use .reduce() design pattern for chaining and serializing
    // a sequence of async operations
    return chunkArray.reduce(function(p, chunk) {
        return p.then(function() {
            return $.ajax({
                type: 'POST',
                url: ajax.ajax_url,
                data: {
                    'action': 'process_big_array',
                    'array_to_process': chunk
                },
                beforeSend: function () {
                    xxx
                },
                dataType: 'json',
            }).then(function(data) {
                results.push(data);
            });
        });
    }, Promise.resolve()).then(function() {
        // depending upon what your ajax returns, results might be an array of arrays
        // that you would want to flatten
        console.log(results);
        return results;
    });
}

// usage
processBigArray(someBigArray, 50).then(function(results) {
    // process results here
}).catch(function(err) {
    // deal with error here
    console.log(err);
});

What's the correct way to iterate through array and sending multiple ajax requests

1 Answers1