4

I tried to fetch >30,000 Records as a JSON Data, I Can't able to get the proper response. But I tried the PHP Service directly to print the JSON, it prints the actual record without any bug. But in AngularJS , I can't able to fetch the actual number of records.

My Sample JSON Data

{
   "records":[
      {
         "ldat":"2014-08-13",
         "eid":"HSL018",
         "dr":"55420",
         "cr":"0",
         "bal":"55420"
      },
      {
         "ldat":"2014-10-11",
         "eid":"HBL056",
         "dr":"0",
         "cr":"35000",
         "bal":"20420"
      },
      {
         "ldat":"2014-10-26",
         "eid":"HBL001",
         "dr":"0",
         "cr":"420",
         "bal":"20000"
      },


      ----- 30,000 Records -----

      ,
      {
         "ldat":"2015-11-01",
         "eid":"HDL001",
         "dr":"0",
         "cr":"20000",
         "bal":"0"
      }
   ]
}

while on debugging, I got only partial output from the Response. 

{
   "records":[
      {
         "ldat":"2014-08-13",
         "eid":"HSL018",
         "dr":"55420",
         "cr":"0",
         "bal":"55420"
      },
      {
         "ldat":"2014-10-11",
         "eid":"HBL056",
         "dr":"0",
         "cr":"35000",
         "bal":"20420"
      },
      {
         "ldat":"2014-10-26",
         "eid":"HBL001",
         "dr":"0",
         "cr":"420",
         "bal":"20000"
      },


      ----- 4,000 Records -----

      ,
      {
         "ldat":"2015-11-01",
         "eid":"HDL0

Let I know is there is any limits are there in the Response? How to get the entire record as a response? Kindly assist me...

The Response Header Details which I was got

HTTP/1.1 200 OK
Date: Thu, 11 Feb 2016 01:31:45 GMT
Server: Apache
Access-Control-Allow-Origin: *
Expires: Thu, 19 Nov 1981 08:52:00 GMT
Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Pragma: no-cache
Vary: Accept-Encoding
Content-Encoding: gzip
Keep-Alive: timeout=10, max=500
Connection: Keep-Alive
Transfer-Encoding: chunked
Content-Type: application/json; charset=UTF-8

My PHP Service Script

<?php
    ob_start(); // start normal buffer
    ob_start("ob_gzhandler"); // start gzip buffer
    echo $content;
    ob_end_flush(); // output gzipped content

    $gzippedContent = ob_get_contents(); // store gzipped content to get size
    header('Content-Length: '.strlen($gzippedContent));
    ob_end_flush(); // flush gzipped content
?>

Use the following PHP Script to test the task, it will produce the 35,000 records.

PHP Script : Statement.php

<?php
    header("Access-Control-Allow-Origin: *");
    header("Content-Type: application/json; charset=UTF-8");

    $content = "";
    for($i=1; $i<=35000; $i++) {
        if ($content != "") {
            $content .= ",";
        }
        $content .= '{"slr":' . $i . ',';
        $content .= '"uid":' . rand(100,10000) . ',';
        $content .= '"name":"Balamanigandan",';
        $content .= '"cell":"9999999999",';
        $content .= '"balance":10000,';
        $content .= '"ltrans":"2015-01-01",';
        $content .= '"msg":" Opening Balance"}';
    }
    $content = '{"records":[' . $content . ']}';

    ob_start(); // start normal buffer
    ob_start("ob_gzhandler"); // start gzip buffer
    echo $content;
    ob_end_flush(); // output gzipped content

    $gzippedContent = ob_get_contents(); // store gzipped content to get size
    header('Content-Length: '.strlen($gzippedContent));
    ob_end_flush(); // flush gzipped content
?>

The AngularJS Sample Script to fetch the records

var stmt= angular.module('Statement', []);

stmt.controller('StmtCtrl', function($scope, $http, $cacheFactory) {

    var request = $http({
        method: "post",
        url: "Statement.php",
        headers: { 'Content-Type': 'application/x-www-form-urlencoded' }
    });

    request.success(function (response) 
    {   
        $scope.data = response;
    });

});

Note: I don't get any error, but I'm getting the partial output as a Response.

B.Balamanigandan
  • 4,713
  • 11
  • 68
  • 130
  • 1
    This might help you http://stackoverflow.com/questions/815961/how-to-determine-the-content-length-of-a-gzipped-file?rq=1 – amighty Feb 16 '16 at 17:10
  • 2
    Can you paste the error that you get. – oseintow Feb 18 '16 at 20:11
  • @oseintow There is no semantic error. I'm not able to get the actual records, because of Content Length of the Response. – B.Balamanigandan Feb 18 '16 at 22:20
  • What exactly happens when it doesn't work? – Kevin B Feb 18 '16 at 22:33
  • @KevinB I Can't able to get the proper response. But I tried the Service directly to print the JSON (Sample URL: https://www.domain.com/Statement.php) using the browser, it prints the actual record without any bug. But in AngularJS , I can't able to fetch the actual number of records. – B.Balamanigandan Feb 18 '16 at 22:37
  • 1
    If you can't get the proper response, that wouldn't be an angular problem. You either aren't describing the problem clearly or this is an issue on the php side of things. Can you see the correct response in your network tab? – Kevin B Feb 18 '16 at 22:39
  • @B.Balamanigandan Why are you using a POST request? You are fetching JSON. You should be using a GET request. I don't think you'll get any response back from a POST other than a 200 if it succeeds. – linstantnoodles Feb 20 '16 at 23:17
  • @linstantnoodles I tried in both action methods POST and GET. But I can't able to get the proper response. Note: I don't get any error, but I'm getting the partial output as a Response. – B.Balamanigandan Feb 21 '16 at 04:14
  • @KevinB - can you please assist me, then where is the hole (i.e., bug) ? If I tried to load the URL in browser, then I'm getting the entire results. But in ajax http call, I can't able to get the response with full results, but you are saying it's not a angular issue, then what's the real issue in this scenario ???? Kindly assist me. – B.Balamanigandan Jun 25 '16 at 03:03

2 Answers2

0

You are sending the Content-Length header after the content itself (and that header does not appear in the header details you listed). Headers should be sent first. Maybe that's the reason?

David
  • 33,444
  • 11
  • 80
  • 118
-1

30k results seems to be a very large result set. Do you really need all these data, can you paginate the result?

If you can see your json without Angular in a browser, I suspect the problem come from Angular or your web browser. Too much data to retreive, can freeze your browser. Chrome provide great tools to monitor your browser, give it a try to see if you don't reach browser limit.

metfan
  • 98
  • 2