Large size JSON response from PHP truncated by browser

Question

I have a hosted PHP application which echoes a large JSON string and Mozilla suddenly started to show the below error in dev tools:

SyntaxError: JSON.parse: bad Unicode escape at line 1 column 1048577 of the JSON data

(Tried chrome & Mozilla, it fails in both, only Chrome doesn't show any errors)

If the response size is reduced, then the problem disappears.

At the front the code is something like this (removed some stuff for clarity)

function DataManager() {

$http.get("datamanager.php")
        .success(function (data) {
            $scope.fulldata = data.fulldata;
            $scope.datagroups = data.datagroups;        
            //More code.......      
        })
        .error(function (data, status, headers, config) {
            //Code removed.....
        });

}

At the back, there is a PHP script running which makes an "echo" and returns the JSON data.

 $fulldata = json_encode($treedata);
$datagroups = json_encode($datagroups);

header('Content-Type: application/json');

echo '{ "fulldata" : ' . $fulldata . ","
. ' "datagroups" : ' . $datagroups
. "}";
die;

Added error_log checks, saved the JSON result to a file using file_put_contents and it's perfectly fine. Uploaded it to online JSON check tools and got no errors.

Tried to find any PHP/Apache restrictions about response size but no luck. Checked the DB/JSON data for possible strange characters or anything that might look suspicious but couldn't find anything.

My host uses fail2ban & mod_security , could they affect the PHP response size?

Pagination style (Page1.....Page X) is out of the question, the full data is needed. We started already to make changes in order to send data in chunks of less than 1M.

Any ideas are welcome.

not sure if this will fix your error, but you could try to add the Content-Length HTTP Header. So you can let the browser know how much data it can expect. — mwallisch, Jan 21 '19 at 16:16
Thanks, already tried the Content-Length , sorry I forgot to mention it. — Greko2009, Jan 21 '19 at 16:18
Pagination is out of the question , what we are doing now is change the code and send the data in chunks of less than 1M each. — Greko2009, Jan 21 '19 at 16:20
FYI... `echo json_encode(array('fulldata' => $treedata, 'datagroups' => $datagroups));` — AbraCadaver, Jan 21 '19 at 16:26
"Pagination is out of the question, we are instead implementing pagination." — Sammitch, Jan 21 '19 at 19:37
@Sammitch , by pagination I meant the usual Page 1.....Page X , updated the description accordingly — Greko2009, Jan 22 '19 at 14:06

score 0 · Answer 1 · answered Jan 21 '19 at 16:23

Consider this previous response on the question of maximum response-size (this being server-side, but clients are similar):

Limit on the length of the data that a webserver can return in response to a GET request

However, from a design standpoint, I suggest that your JSON-based API should be designed to be "paginated" so that it is not necessary to expect to return more than about a megabyte of data at one time, or less. "Don't get near the fences."

Large size JSON response from PHP truncated by browser

1 Answers1