How to pass large array value from javascript to php

Question

My requirement is to make multiple xml request and to store the data in an array so that if i get say for example 4000 records I can display 40 records in a page and can do pagination. So if i goto page2 I can get the data from the array instead of making the xml request again.

Currently I am using a php array variable to store large amount of data and on click a button javascript function will call the same php page again using ajax request through query string. The problem is that the query string can't hold large arrays.

Here is my code -

1. Passing php array to javascript variable

   echo '<script type="text/javascript">/* <![CDATA[ */';
   echo 'var allProd = '.json_encode($allProd);
   echo '/* ]]> */</script>';
   

2. Using form calling javascript function through button click

   <form id="new_page" name="new_page" method="post" action="" >
       <td>&nbsp;&nbsp;Jump to page&nbsp;<input name="neggPage" id="neggPage"  class="pageText" style="height:20px; width:40px"/></td>
       <input type="hidden" name="hideProd" value="<?php echo htmlentities(serialize($allProd)); ?>" />
       <td><input type="Submit" name="neggBut" value="Go" class="pageButton" onClick="get_results(document.getElementById('neggPage').value, <?php echo $sInd; ?>, <?php echo $rndTot; ?>, <?php echo $totEnt; ?>, allProd);">
       Total pages <?php echo $rndTot; ?></td>
       <td height="40">&nbsp;</td>
      </tr></table>
    </form>
   

3. Javascript function

   <script>
   function get_results(PageNo, SelIndex, totPage, totEnt, allProd)
   {
   var allProd1 = '';
   for (i=0; i < allProd.length; i++)
   {
       allProd1 += '&q[' + i + ']=' + encodeURIComponent(JSON.stringify(allProd[i]));
   }
   
   if (PageNo > totPage || isNaN(PageNo))
   {
       alert ('Please enter the available pages');
   }
   else
   {
   var neggProd = sessionStorage.getItem('Product1');
   var cntryCode = sessionStorage.getItem('Cntry'); 
   var sortType;
   var ordType;
   if (SelIndex == 0) {sortType = 'bestmatch';}
   else if (SelIndex == 1) {sortType = 'bestseller';}
   else if (SelIndex == 2) {
       sortType = 'pricehigh';
       ordType = 'dsc';
   }
   else if (SelIndex == 3) {
       sortType = 'pricelow';
       ordType = 'asc';
   }
   
   document.getElementById("sl_api").innerHTML="";
   document.getElementById("sl_api1").setAttribute("style","background:     url('KartEazy/loading.gif') #edeeee no-repeat center center; background-size:20px 20px;");
   
   if (window.XMLHttpRequest)
     {// code for IE7+, Firefox, Chrome, Opera, Safari
     xmlhttp=new XMLHttpRequest();
     }
   else
     {// code for IE6, IE5
     xmlhttp=new ActiveXObject("Microsoft.XMLHTTP");
     }
   xmlhttp.onreadystatechange=function()
     {
     if (xmlhttp.readyState==4 && xmlhttp.status==200)
       {
       document.getElementById("sl_api").innerHTML= xmlhttp.responseText;
       }
     }
   
   xmlhttp.open("GET",'KartEazy/Skimlinks/Skimlinks.php?PageNum=' + PageNo + '&neggProd=' + neggProd + '&cntryCode=' + cntryCode + '&sortType=' + sortType + '&ordType=' + ordType + '&totEnt=' + totEnt + allProd1, true);
   xmlhttp.send();
   setInterval(function()    {
   document.getElementById("sl_api1").setAttribute("style","background: #edeeee;")},4500);
   }
   }
   </script>
   

4. In the same page now I am trying to get the array

   if(isset($_REQUEST['q']))
   {
   $totEnt = $_REQUEST['q'];
   }
   

I tried using hidden form variable but even $_POST also not accepting large arrays. Could any of you please give me a solution or suggestion. It will be very helpful for me.

Even storing the data in database or session variable will not solve my requirement. Because if I use database I can't retrieve data for multiple queries and even php session variable will be overwritten if the user uses multiple tabs for the same url.

Answer 1

I was in a similar situation where I was passing several MEG of data from Javascript to PHP.

I don't have the code with me, so I can only give you a p-code solution, but it will give you the idea.

// JavaScript:

var bigstring = "...."; the data to pass
var chunkSize = 100*1024; 

// http://stackoverflow.com/questions/7033639/javascript-split-large-string-in-n-size-chunks
var chunks = splitString(bigstring);

var chunkIdx = -1; // Prime the pump

function sendNextChunk() {
  chunkIdx++;


  if (chunkIdx < chunks.length) {
    // Your AJAX call here
    var data = {
      piece: chunkIdx
      ofpieces: chunks.length;
    }

    // set your onsuccess method to call
    // sendNextChunk again
  }
}

sendNextChunk(); // start it.

Now, for your PHP code:

// Read in each chunk, and the piece number.
// The way I've got the code written, then come in sequentional order
// so open up a temporary file and append the data.
// when piece === ofpieces
// you have the whole file and you can then process it.