How to parse JSON and throw if loss of fidelity?

Question

Some values in JSON cannot be represented in JavaScript with full fidelity. For example:

9999999999999999999999999

I am working on a protocol/application that requires interoperability and we use JSON as our data interchange format. In my JavaScript implementation, I would like the JSON parser to throw on those inputs.

I have created a simple (and wrong) function to do this.

function safeDecodeJson(str) {
  decoded = JSON.parse(str);
  reencoded = JSON.stringify(decoded);
  if (str != reencoded) {
    throw new RangeError();
  }
  return decoded;
}

Here is a test case:

jsonString = "9999999999999999999999999";
safeDecodeJson(jsonString);

It does throw the RangError.

My issue is that this safeDecodeJson function only works if the input is minimal. Is there a more robust way to implement this function?

To be super specific, I am concerned about a "non-injective attack" on the input JSON file. My system requires that logically-distinct JSON inputs (like 9999999999999999999999999 and 9999999999999999999999998) have distinct representations in JavaScript. Or the function must throw.

Maybe [`BigInt`](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/BigInt)? — Jack Bashford, Mar 12 '19 at 01:51
This is unrelated, but you seem to be creating a lot of global variables by skipping declaring your local variables. I'm not sure if you're doing that in your real code or just in this question, but it's worth checking out because those globals can quickly become a maintenance nightmare. — Paul, Mar 12 '19 at 15:12
This is an interesting question, but it seems much more difficult than I would assume at first. Related links: https://esdiscuss.org/topic/arbitrary-precision-numbers-in-json , https://stackoverflow.com/a/47916876/772035 — Paul, Mar 12 '19 at 15:36

score 0 · Answer 1 · answered Mar 12 '19 at 02:06

0

It is not really odd. Javascript uses floating point internally. Here is the explanation of highest value.

In other words you can't use more than 53 bits. In some implementations you may be limited to 31. Try using a bignum library, or if you only need to deal with integers, a biginteger library.

answered Mar 12 '19 at 02:06

Abdullah Aziz

700
5
11

BigInt is actually native JS now! – jhpratt Mar 12 '19 at 02:13
JavaScript always uses 64 bit doubles. If an implementation uses anything less than that it is not JavaScript. 53 bits are used for precision, but the other 11 bits are just as important in distinguishing different numbers. – Paul Mar 12 '19 at 15:08

score 0 · Answer 2 · answered Mar 12 '19 at 02:49

The answers that have been given are not the answer to the question. read the question. Is there a more robust way to implement this function? I would ask if you are able to render the number, when serializing to JSON, in a string form so that the JavaScript JSON parser doesn't attempt to parse it. Then you can, after the JSON.parse manually handle the odd cases - where you can parse individual string'd numbers manually and then perhaps do the reverse to see if they are the same. Using BigInt or whatever is fine, but that's not what you were asking.

Paul · Answer 3 · 2019-03-14T16:53:49.743

This is my best attempt, based off this answer to a similar question. It uses your logic of converting to a number and back to a string and checking if it matches the original string and makes it apply to all numbers in the JSON.

Unfortunately that logic is itself a bit flawed since there are several ways to represent the same number in JSON. It will throw the error for numbers that aren't in the representation given by Number#toString(), I.E. 1e1 is one way of representing 10, but it will throw the RangeError below. If you can guarantee that your numbers will be represented in the same format as Number#toString(), then this should work for you:

const tests = [
  // Success cases
  `{"foo":[10,25]}`,
  `{"\\\\\\\"":[10,25]}`,
  `{"99999999999999999999999999":"99999999999999999999999999"}`,
  `{"foo":[10,25],"bar":{"baz":{"bark":[1,2,3]}}}`,

  // RangeError cases
  `{"foo":99999999999999999999999999}`,
  `{"99999999999999999999999999":99999999999999999999999999}`,
  `{"foo":[10,25],"bar":{"baz":{"bark":[1,2,3,1e1]}}}`,
];

tests.forEach( test => {
  try { 
    console.log( 'Success', JSON.stringify( safeDecodeJson( test ) ) );
  } catch ( e ) {
    console.log( 'Error', e.message );
  }
} );

function safeDecodeJson( str ) {
  const prefix = '^({{SafeJsonDecode}})^:';
  const pre_decode = str.replace( /((?:[^"]*?(?:"(?:[^"\\]|\\.)*?")?)*?)((?:[:,\[]|^)[\s\n]*)(-?(0|([1-9]\d*))(\.\d+)?([eE][-+]?\d*)?)/gsy, `$1$2"${prefix}$3"` );
  return JSON.parse( pre_decode, ( key, value ) => {
    if ( typeof value !== 'string' || ! value.startsWith( prefix ) )
      return value;
    const numeric_string = value.substr( prefix.length );
    if ( '' + +numeric_string !== numeric_string )
      throw new RangeError( `\`${numeric_string}\` out of range or not in canonical form` );
    return +numeric_string;
  } );
}

If you want to you could use an arbitrary precision number library, such as bignumber.js, to parse the numbers instead of throwing a RangeError. With that you can also detect that 1e1 is the same as 10:

const result1 = safeDecodeJson( '9999999999999999999999999' );

// If out of range the result will be a BigNumber
console.log( BigNumber.isBigNumber( result1 ) );
console.log( result1 );

const result2 = safeDecodeJson( '1e1' );

// If exactly representable by a JavaScript number, the result will be a number, not a BigNumber
console.log( BigNumber.isBigNumber( result2 ) );
console.log( result2 );

function safeDecodeJson( str ) {
  const prefix = '^({{SafeJsonDecode}})^:';
  const pre_decode = str.replace( /((?:[^"]*?(?:"(?:[^"\\]|\\.)*?")?)*?)((?:[:,\[]|^)[\s\n]*)(-?(0|([1-9]\d*))(\.\d+)?([eE][-+]?\d*)?)/gsy, `$1$2"${prefix}$3"` );
  return JSON.parse( pre_decode, ( key, value ) => {
    if ( typeof value !== 'string' || ! value.startsWith( prefix ) )
      return value;
    const numeric_string = value.substr( prefix.length );
    if ( '' + +numeric_string !== numeric_string ) {
      const big = new BigNumber( numeric_string );
      if ( ! new BigNumber( +numeric_string ).isEqualTo( big ) )
        return big;
    }
    return +numeric_string;
  } );
}

<script src="https://cdnjs.cloudflare.com/ajax/libs/bignumber.js/8.1.1/bignumber.min.js"></script>

You could also just return numbers as a BigNumber (just always return new BigNumber( numeric_string ) instead of conditionally returning +numeric_string.

How to parse JSON and throw if loss of fidelity?

3 Answers3