How do you check for nan in python jmespath?

Question

Assume import jmespath. Let's say I have some data like:

In [3]: data=[{'foo':10}, {'foo':float('nan')}, {'foo':32}]

In [4]: data
Out[4]: [{'foo': 10}, {'foo': nan}, {'foo': 32}]

I want to use jmespath to count the number of nodes that are nan. I can't seem to get it to. I can count the non-nan nodes like so:

jmespath.search('length([?@.foo.abs(@) >= `0`])',data)

I have tried comparing to nan and NaN but that doesn't work, and frankly it shouldn't because nan doesn't equal nan. However, there is no is_nan() function that I can see. I suppose I could convert to string and then compare them? That seems like a last resort. Is there some is_nan I don't see documented?

Maybe dumb idea depending on your use-case, but you could subtract from the data length the non-nan length (change `>` to `>=` though). — wim, Feb 12 '18 at 02:18
@wim i meant to have >=, which i edited. it's not that straightfoward to do arithmetic in jmespath as far as i can tell but i thought about that. I need to look into if arithmetic is a thing — frankc, Feb 12 '18 at 02:39
JSON does not know NAN: https://stackoverflow.com/q/1423081/7311767 — Stephen Rauch, Feb 12 '18 at 05:10
@Stephen Raunch, that link seems to claim that its treated as null but interestingly there is an is_null() method in jmespath, but it didn't not identify nans as null. I think ultimately i will try to add a custom function into the language — frankc, Feb 12 '18 at 17:01

dreftymac · Answer 1 · 2019-03-23T01:10:17.033

Context

python 2.x or python 3.x
jmespath query

Problem

how to construct a query that counts the instances of nan (not a number)

Solution

use Jmespath not-expression ! (exclaimation point) in conjunction with the abs() function
use jmespath length() function to count the results

Example

jmespath.compile('''[*].foo|[? !(abs(@) >= `0`)]|length([*])''').search(data)

Details

Jmespath type() function treats nan as number
Using not-expression in combination with abs(@) greater-than or equal-to zero returns true for nan

Pitfalls

Jmespath errors out with invalid type for value if an incompatible type is passed into abs() (such as string).