0

I am trying to populate a DataTable (DataTables.net) from a javascript array of objects. While I don't have problem doing so if the data is relatively small, such as less than 6000 rows--and data can be as large as 20,000 rows. But when the data is larger then I get Google Chrome to hang and I get the message to Wait or exit. Firstly, my current code:

var data = [];
for (var i = 0; i < arr_Members_in_Radius.length; i++) {
    data.push([arr_Members_in_Radius[i].record_id);
}
var search_results_table_2 = $('#tbl_search_results').DataTable({
    destroy: true,
    data: data
});

The above code hangs when data is large. So, following this link How to rewrite forEach to use Promises to stop "freezing" browsers? I implemented a Promise based approach:

function PopulateDataTable(current_row) {
    search_results_table.row.add([current_row.record_id, current_row.JOA_Enrollment, current_row.mbr_full_name,
    current_row.member_address, current_row.channel_subchannel, current_row.joa_clinic, current_row.serviceaddress]).draw();
};
var wait = ms => new Promise(resolve => setTimeout(resolve, ms));
arr_Members_in_Radius.reduce((p, i) => p.then(() => PopulateDataTable(i) ).then(() => wait(1)), Promise.resolve());

and while that seems to solve the problem of the browser hanging, the datatable gets updated one row at a time--which is very time consuming and makes the datatable unusuable (scrolling issues, searching, sorting etc) until all/sufficient of the 6000 rows are loaded. It would be nice if datatable gets loaded at least 100+ at a time in the PopulateDataTable() function call. I wonder how will I able to do that. Or please suggest a different approach.

Thank you!

IrfanClemson
  • 1,699
  • 6
  • 33
  • 52
  • 2
    Javascript is not designed to handle datasets that size. If you have 20k entities to display in a DataTable I would strongly suggest you transfer the logic to the server side and implement a method of filtering and paging the entities before transmitting the data to the client. – Rory McCrossan Jun 22 '21 at 14:11
  • Pagination is not feasible; the data contains coordinates which have to be displayed on a map at once. It is more than DataTable. But good suggestion. Thank you. PS. It is a .NET .ashx.cs file which is providing the data from a Postgresql database, btw. – IrfanClemson Jun 22 '21 at 15:26

1 Answers1

0

It turns out that the data table was indeed able to handle 20,000 rows of data without any issue. The problem was--and to some extent still is-- that the subsequent call to another function load large data on a map (a Leaflet.js map) was causing the browser to hang. So I guess too much data processing too close to each other. Here is how I fixed it. The function to call the display on the map is now waiting for 3 seconds before calling. I will tweak my code to increase/reduce the timeout based on the amount of data. Not an elegant solution but the browser doesn't hang anymore and I get to display all the data per user selections.

setTimeout(function () {
 createRouteOnMap(arr_Members_in_Radius);
}, 3000);
IrfanClemson
  • 1,699
  • 6
  • 33
  • 52