1

When opening a page I make an Ajax request. This requests takes about 10 sec to load and returns about 11MB of data.

The response of this Ajax requests needs to be set in the DOM by InnerHTML. For Internet Explorer this 11MB is way to much to handle at once, so I split that 11MB into 40 small blocks.

This is my code:

$.get('url')
.done(function( data ) {
    splitdata = data.split('<!-- START -->');

    data = '';

    var length = splitdata.length;

    for (var i = 0; i < length; i++)
    {
        var brin        = String(splitdata[i].split('\n')[0].replace("<!-- ",'').replace(' -->', ''));
        var percentage  = Math.round(((i+1)*100)/(length + 2));
        var data        = splitdata[i];

        if(brin.length != 0)
        {
            (function(brin, percentage, data) {
                setTimeout(function() {
                    console.log(brin);
                    console.log(percentage+'%');

                    document.getElementById('tab_'+brin).innerHTML = data;
                    //$('#tab_'+brin).html(splitdata[i]);
                }, 0);
            })(brin, percentage, data)

            $('#loadingbar').css('width', percentage+'%');
        }
    }

    ajaxBegrotingDone(callback);
})
.fail(function() {
    ajaxBegrotingFail(callback);
});

Now I have two questions: 1. All browsers have some trouble with that innerHTML part. The page "hangs" and so my progressbar is not updated. Is there a solution for this problem?

  1. Now I need to wait for 10 sec till my Ajax request is done. After that the JavaScript parsing (and so the innerHTML starts). I would be nice to split realtime, is that possible?

It is not possible to make 40 small ajax requests because the data is very complex and results of part 1 are the basis of calculations made in two and so on.

Luuk Schakenraad
  • 51
  • 5
  • You are parsing it realtime. It's just that, you're modifying the document so quickly the browser doesn't have time to render between your updates. You could add a delay between each update using setTimeout with an increasing delay amount, but ultimately the best solution is to not have so much data. – Kevin B Nov 12 '13 at 19:26
  • If you can modify the server side and the data can be split into smaller pieces then maybe you can "stream" the data to the client [jquery ajax, read the stream incrementally?](http://stackoverflow.com/questions/7740646/jquery-ajax-read-the-stream-incrementally) – Andreas Nov 12 '13 at 19:34
  • You and your JS are better at parsing 11MB of HTML that the browser itself? I doubt that. If you need something inside the 11MB, don't append it to the DOM, parse the string. – Rudie Nov 12 '13 at 20:08

2 Answers2

1

Solved my own question, thanks to Boothinator. The code needs some improvements, but it works as expected:

$   .ajax({
                url:            'url',
                dataType:       'html',
                processData:    false,
                xhr:            function()
                                {
                                    var xhr         = $.ajaxSettings.xhr();
                                    var data        = [];

                                    xhr.addEventListener("progress", function(evt)
                                    {
                                        if(data.length > 0)
                                        {
                                            splitdata = xhr.responseText.split(data.join('<!-- START -->'))[1].split('<!-- START -->');
                                        }
                                        else
                                        {
                                            splitdata = xhr.responseText.split('<!-- START -->');
                                        }

                                        var length = splitdata.length;

                                        for (var i = 0; i < length-1; i++)
                                        {
                                            if(splitdata[i] !== undefined)
                                            {
                                                var brin        = String(splitdata[i].split('\n')[0].replace("<!-- ",'').replace(' -->', ''));

                                                if(brin.length != 0)
                                                {
                                                    var percentage  = Math.round(((data.length+1)*100)/(40 + 2));

                                                    data.push(splitdata[i]);

                                                    console.log(brin);
                                                    console.log(data.length);
                                                    console.log(percentage);

                                                    document.getElementById('tab_'+brin).innerHTML = splitdata[i];
                                                    //$('#tab_'+brin).html(data[i]);

                                                    $('#loadingbar').css('width', percentage+'%');
                                                }
                                            }
                                        }
                                    }, false);

                                    return xhr;
                                }
Luuk Schakenraad
  • 51
  • 5
0

You should update your progress asynchronously, but do the final processing after all the data is loaded, as you are now.

For that, you should update your code to use the $.ajax() method so you can listen to the progress event of the XMLHttpRequest.

$.ajax({
    url: 'url',
    xhr: function()
    {
        var xhr = $.ajaxSettings.xhr();
        //Download progress
        xhr.addEventListener("progress", function(evt){
        if (evt.lengthComputable) {
            var percentage = 100 * evt.loaded / evt.total;
            $('#loadingbar').css('width', percentage+'%');
            }
        }, false);
        return xhr;
    },
    complete: /* done() callback */,
    error: ajaxBegrotingFail(callback)
});

This code will allow you to receive the data from the server asynchronously so that the browser does not hang, and will update the progress bar on the slow process of loading the data.

You could even provide a second progress bar for your processing after the data load is complete, or reuse the same progress bar.

boothinator
  • 121
  • 3