2

Let's say I'm doing an AJAX call to get some JSON data and it returns a 300MB+ JSON string. After it finishes downloading, I need to parse it.

JSON.parse(this.responseText);

In Chrome (v56) I'll get this error: “Uncaught SyntaxError: Unexpected end of JSON input”

If I download that same JSON string and open it up in Notepad.exe, after a minute or two of waiting, I can navigate to the bottom and it appears as if the entire JSON string looks good (it ends with the proper closing tags).

Is there a limitation to JSON.parse()? It would surprise me if Chrome is simply unable to convert the JSON string to a proper JSON object because it's such a large size. While this is sort of being used in a stress-test scenario, I would like to see this working with JSON strings as large as 500MB. Note: Using JSON.parse(this.responseText), I have no issues with 200MB JSON strings.

(I understand that there is supposedly no limitation on JSON size. I am not asking that question. I am asking if there is an issue with JSON.parse at a certain file size or certain memory amount or if there is a different line of code that needs to be run)

Anyone know of a solution (client-side is ideal as opposed to paginating the JSON or breaking it up server side and making multiple calls)? Thoughts?

UPDATE:

If I grab a smaller JSON string and do not parse it after the AJAX response, I can do a console.log(this.responseText.length); It returns 11514 in the DevTools.

For the the ~300MB JSON string, console.log(this.responseText.length); returns 0.

AJAX call:

     getData: function(url){
        var xhr = new XMLHttpRequest();

        xhr.onreadystatechange = function(){
            if(this.readyState === 4 && this.status === 200){
                console.log(this.responseText.length);
                return false; //prevent from going to next line to parse

                var data = JSON.parse(this.responseText);
            }
        };

        xhr.onerror = function(){
            console.log('there was an error:', this);
        };

        xhr.open('GET', url, true);
        xhr.send();
    }

Thoughts? I can't be the only one who's run into this issue.

Shazboticus S Shazbot
  • 1,289
  • 2
  • 16
  • 27
  • 1
    Have you confirmed that the entire JSON string was returned? – sideroxylon Feb 11 '17 at 01:31
  • Possible duplicate of [Is there a limit on how much JSON can hold?](http://stackoverflow.com/questions/1262376/is-there-a-limit-on-how-much-json-can-hold) – SaggingRufus Feb 11 '17 at 01:31
  • @sideroxylon I did my best to confirm it. The most obvious answer is that the string is incomplete but it doesn't seem to be as far as I can tell. – Shazboticus S Shazbot Feb 11 '17 at 01:33
  • In light of your update, one question - is `JSON.parse()` or `console.log()` in the success/done function of your AJAX function? – sideroxylon Feb 11 '17 at 02:07
  • Yes... adding updating original question to show AJAX request – Shazboticus S Shazbot Feb 11 '17 at 02:14
  • Maybe try taking the server out of it, just to test: ` if(this.readyState === 4) {console.log(this.responseText.length);}` – sideroxylon Feb 11 '17 at 02:27
  • Impossible to answer without the JSON. – epascarello Feb 11 '17 at 02:31
  • @sideroxylon tested with your above code - says 0 – Shazboticus S Shazbot Feb 11 '17 at 02:41
  • Sorry, I'm out of ideas (and have nothing I can test) - it appears to be a file size issue, so I'd look at the server settings, to see if there is anything truncating the file, and I'd also console log the response (not the length), and see what's there. Maybe also get the response status, or even try a synchronous AJAX request: `xhr.open('GET', url, false)`. – sideroxylon Feb 11 '17 at 02:45
  • @sideroxylon - Whole page goes dark and freezes. Chrome gives a warning right before it happens: Chrome: "Synchronous XMLHttpRequest on the main thread is deprecated because of its detrimental effects to the end user's experience. For more help, check https://xhr.spec.whatwg.org/." I appreciate your help and effort. I've been doing front-end dev for a long time and I wouldn't ask on here unless I tried everything I could first. – Shazboticus S Shazbot Feb 11 '17 at 02:49
  • Did you ever find a solution to this? Suffering from the same problem, only in chrome. Large json files throw this error, Firefox can handle it though. – J. Minjire Dec 21 '21 at 07:26

2 Answers2

0

It's possible there's an error in the JSON somewhere if chrome suggests it ends all of a sudden. It might be worth throwing it into a json parser like http://json.parser.online.fr/ and seeing if it has any issue with it.

Simply investigating a json string in notepad is probably not going to give you the right answer.

dayvidwhy
  • 76
  • 1
  • 5
0

Maybe use a smaller JSON file, or store it in your webserver filesystem and do a server-side render.

wilsonhobbs
  • 941
  • 8
  • 18
  • this is for stress testing an application - we won't see a 300MB JSON string (hopefully) in production - but 100MB+ is common daily. The data is used on the front-end itself to render visual content. – Shazboticus S Shazbot Feb 11 '17 at 02:43