Why do divisions not work in jsonpath-ng?

Question

Jsonpath-ng offers basic arithmetic as in:

from jsonpath_ng import jsonpath
from jsonpath_ng.ext import parse

jsonpath_expr = parse('$.foo * 2')
target = {'foo': 2}
result = jsonpath_expr.find(target)
result = [match.value for match in result]
print(result)

result: [4]

However, if I change the expression to $.foo / 2, then I get a Parse Error:

Traceback (most recent call last):
  File "test.py", line 4, in <module>
    jsonpath_expr = parse('$.foo / 2')
  File "C:\Users\micha\AppData\Roaming\Python\Python38\site-packages\jsonpath_ng\ext\parser.py", line 172, in parse
    return ExtentedJsonPathParser(debug=debug).parse(path)
  File "C:\Users\micha\AppData\Roaming\Python\Python38\site-packages\jsonpath_ng\parser.py", line 32, in parse
    return self.parse_token_stream(lexer.tokenize(string))
  File "C:\Users\micha\AppData\Roaming\Python\Python38\site-packages\jsonpath_ng\parser.py", line 55, in parse_token_stream
    return new_parser.parse(lexer = IteratorToTokenStream(token_iterator))
  File "C:\Users\micha\AppData\Roaming\Python\Python38\site-packages\ply\yacc.py", line 333, in parse
    return self.parseopt_notrack(input, lexer, debug, tracking, tokenfunc)
  File "C:\Users\micha\AppData\Roaming\Python\Python38\site-packages\ply\yacc.py", line 1201, in parseopt_notrack
    tok = call_errorfunc(self.errorfunc, errtoken, self)
  File "C:\Users\micha\AppData\Roaming\Python\Python38\site-packages\ply\yacc.py", line 192, in call_errorfunc
    r = errorfunc(token)
  File "C:\Users\micha\AppData\Roaming\Python\Python38\site-packages\jsonpath_ng\parser.py", line 69, in p_error
    raise Exception('Parse error at %s:%s near token %s (%s)' % (t.lineno, t.col, t.value, t.type))
Exception: Parse error at 1:6 near token / (SORT_DIRECTION)

I can sometimes work around this issue by dividing by the inverse value, so I would do $.foo * 0.5 to get the result [1.0]. But this doesn't work if both sides of the equation are numeric values of different types (int or float). So 2 * 0.5 and 0.5 * 2 will result in a Parse error, but 2.0 * 0.5 will not.

How do I get divisions to work? And why can I not multiply a float by an integer?

Answer 1

Those are both grammar bugs.

1. Extended JSON paths are allowed to be suffixed with a bracketed expression containing a list of "sorts"; each sort starts with / or \. To make that work, the extended lexer recognises those two symbols as the token SORT_DIRECTION, which takes precedence over the recognition of / as an arithmetic operator. Consequently the use of / as an arithmetic operator is not allowed by the parser. (In fact, the problem goes deeper than that, but that's the essence.)

2. For some reason, the grammar author chose to separate NUMBER (actually, integer) and FLOAT in arithmetic expressions, which means that they had to enumerate the possible combinations. What they chose was:

   jsonpath : NUMBER operator NUMBER
            | FLOAT operator FLOAT
            | ID operator ID
            | NUMBER operator jsonpath
            | FLOAT operator jsonpath
            | jsonpath operator NUMBER
            | jsonpath operator FLOAT
            | jsonpath operator jsonpath
   

   There are other problems with this grammar, but the essence here is that it permits NUMBER operator NUMBER and FLOAT operator FLOAT but does not permit NUMBER operator FLOAT or FLOAT operator NUMBER, which is what you observe. However, path expressions can work with either NUMBER or FLOAT.