2

I am looking for a way to parse a JSON file for a specific node and get that node's line number in the file. I would like to use the Jayway JSONPath library to support extended JSONPath queries.

For example (from jsonpath.com), here's some JSON:

{
  "firstName": "John",
  "lastName" : "doe",
  "age"      : 26,
  "address"  : {
    "streetAddress": "naist street",
    "city"         : "Nara",
    "postalCode"   : "630-0192"
  },
  "phoneNumbers": [
    {
      "type"  : "iPhone",
      "number": "0123-4567-8888"
    },
    {
      "type"  : "home",
      "number": "0123-4567-8910"
    }
  ]
}

and here's a jsonPath: $.phoneNumbers.[?(@.type=='iPhone')]

I would like to have a way to say that this node is on line 11 in the json file. I don't know ahead of time what the json contents might be or the jsonPath. Both are dynamic.

So far, I've tried to parse the json into a tree and traverse it up to the node to get the parser's current location, but the parser must always run to the end of the file before the jsonPath executes. Any other ideas?

tophersmith116
  • 432
  • 1
  • 5
  • 19
  • Line number is ultimately irrelevant, and you're not guaranteed to have more than one line number anyway. It's very likely that all whitespace is stripped and the JSON is a single line. – Christopher Schneider Aug 25 '20 at 19:34
  • While broadly true, this is a part of a static analysis engine. JSONPath are rules and I would like to report on where this rule was violated(found). The JSON is likely to be somehow formatted, but not necessarily pretty-printed. – tophersmith116 Aug 25 '20 at 19:51
  • hey @tophersmith116 , are you able to find a solution to this? I exactly have same requirement. And i am trying to figure this out since last 20 days, but no help yet. It would be great if you can assist. Thanks. – Ankit Ostwal Aug 26 '21 at 04:29
  • @AnkitOstwal I've posted my code that solved this, which has a thread-safety warning. Take a look and see if it helps – tophersmith116 Aug 26 '21 at 18:26
  • Hey @tophersmith116, thanks for the code. Let me check this and see if it helps my problem. I will keep you updated. – Ankit Ostwal Aug 27 '21 at 09:19
  • Hi @tophersmith116, Code dosent work with all scenarios, i mean it works for simple JSON. But not for complex JSON. I have JSON in which you can't make node as unique. Just wanted to know from you, what alterations, i can do to make it work. I can share my json as well (https://stackoverflow.com/questions/68993406/is-it-possible-to-get-line-number-based-on-jsonpath-for-a-given-json) . Help is apperciated. Thanks. – Ankit Ostwal Aug 31 '21 at 05:55

2 Answers2

2

I eventually found a solution that involves using Jackson's JsonFactory and JsonParser. It's kludge-y to say the least, but it uses the JsonParser's knowledge of its parser's line number to get the JsonNode's position and works pretty well.

I'll paste the code here, but the code is also available at watchtower github

Calling class:

void findLineNumber() throws Exception{
    CustomParserFactory customParserFactory = new CustomParserFactory();
    ObjectMapper om = new ObjectMapper(customParserFactory);
    factory = new CustomJsonNodeFactory(om.getDeserializationConfig().getNodeFactory(),
            customParserFactory);
    om.setConfig(om.getDeserializationConfig().with(factory));
    config = Configuration.builder()
            .mappingProvider(new JacksonMappingProvider(om))
            .jsonProvider(new JacksonJsonNodeJsonProvider(om))
            .options(Option.ALWAYS_RETURN_LIST)
            .build();

    File filePath = ...;
    JsonPath jsonPath = ...;
    DocumentContext parsedDocument = JsonPath.parse(filePath, config);
    ArrayNode findings = parsedDocument.read(jsonPath);
    for (JsonNode finding : findings) {
        JsonLocation location = factory.getLocationForNode(finding);
        int lineNum = location.getLineNr();
        //Do something with lineNum
    }
}

CustomJsonNodeFactory.java

public class CustomJsonNodeFactory extends JsonNodeFactory {

    private static final long serialVersionUID = 8807395553661461181L;

    private final JsonNodeFactory delegate;
    private final CustomParserFactory parserFactory;

    /*
     * "Why isn't this a map?" you might be wondering. Well, when the nodes are created, they're all
     * empty and a node's hashCode is based on its children. So if you use a map and put the node
     * in, then the node's hashCode is based on no children, then when you lookup your node, it is
     * *with* children, so the hashcodes are different. Instead of all of this, you have to iterate
     * through a listing and find their matches once the objects have been populated, which is only
     * after the document has been completely parsed
     */
    private List<Entry<JsonNode, JsonLocation>> locationMapping;

    public CustomJsonNodeFactory(JsonNodeFactory nodeFactory,
        CustomParserFactory parserFactory) {
        delegate = nodeFactory;
        this.parserFactory = parserFactory;
        locationMapping = new ArrayList<>();
    }

    /**
     * Given a node, find its location, or null if it wasn't found
     * 
     * @param jsonNode the node to search for
     * @return the location of the node or null if not found
     */
    public JsonLocation getLocationForNode(JsonNode jsonNode) {
        return this.locationMapping.stream().filter(e -> e.getKey().equals(jsonNode))
                .map(e -> e.getValue()).findAny().orElse(null);
    }

    /**
     * Simple interceptor to mark the node in the lookup list and return it back
     * 
     * @param <T>  the type of the JsonNode
     * @param node the node itself
     * @return the node itself, having marked its location
     */
    private <T extends JsonNode> T markNode(T node) {
        JsonLocation loc = parserFactory.getParser().getCurrentLocation();
        locationMapping.add(new SimpleEntry<>(node, loc));
        return node;
    }

    @Override
    public BooleanNode booleanNode(boolean v) {
        return markNode(delegate.booleanNode(v));
    }

    @Override
    public NullNode nullNode() {
        return markNode(delegate.nullNode());
    }

    @Override
    public NumericNode numberNode(byte v) {
        return markNode(delegate.numberNode(v));
    }

    @Override
    public ValueNode numberNode(Byte value) {
        return markNode(delegate.numberNode(value));
    }

    @Override
    public NumericNode numberNode(short v) {
        return markNode(delegate.numberNode(v));
    }

    @Override
    public ValueNode numberNode(Short value) {
        return markNode(delegate.numberNode(value));
    }

    @Override
    public NumericNode numberNode(int v) {
        return markNode(delegate.numberNode(v));
    }

    @Override
    public ValueNode numberNode(Integer value) {
        return markNode(delegate.numberNode(value));
    }

    @Override
    public NumericNode numberNode(long v) {
        return markNode(delegate.numberNode(v));
    }

    @Override
    public ValueNode numberNode(Long value) {
        return markNode(delegate.numberNode(value));
    }

    @Override
    public ValueNode numberNode(BigInteger v) {
        return markNode(delegate.numberNode(v));
    }

    @Override
    public NumericNode numberNode(float v) {
        return markNode(delegate.numberNode(v));
    }

    @Override
    public ValueNode numberNode(Float value) {
        return markNode(delegate.numberNode(value));
    }

    @Override
    public NumericNode numberNode(double v) {
        return markNode(delegate.numberNode(v));
    }

    @Override
    public ValueNode numberNode(Double value) {
        return markNode(delegate.numberNode(value));
    }

    @Override
    public ValueNode numberNode(BigDecimal v) {
        return markNode(delegate.numberNode(v));
    }

    @Override
    public TextNode textNode(String text) {
        return markNode(delegate.textNode(text));
    }

    @Override
    public BinaryNode binaryNode(byte[] data) {
        return markNode(delegate.binaryNode(data));
    }

    @Override
    public BinaryNode binaryNode(byte[] data, int offset, int length) {
        return markNode(delegate.binaryNode(data, offset, length));
    }

    @Override
    public ValueNode pojoNode(Object pojo) {
        return markNode(delegate.pojoNode(pojo));
    }

    @Override
    public ValueNode rawValueNode(RawValue value) {
        return markNode(delegate.rawValueNode(value));
    }

    @Override
    public ArrayNode arrayNode() {
        return markNode(delegate.arrayNode());
    }

    @Override
    public ArrayNode arrayNode(int capacity) {
        return markNode(delegate.arrayNode(capacity));
    }

    @Override
    public ObjectNode objectNode() {
        return markNode(delegate.objectNode());
    }

}

CustomParserFactory.java (Note that this removes thread-safety, which can be kind of a big deal):

public class CustomParserFactory extends JsonFactory {

    private static final long serialVersionUID = -7523974986510864179L;
    private JsonParser parser;

    public JsonParser getParser() {
        return this.parser;
    }

    @Override
    public JsonParser createParser(Reader r) throws IOException, JsonParseException {
        parser = super.createParser(r);
        return parser;
    }

    @Override
    public JsonParser createParser(String content) throws IOException, JsonParseException {
        parser = super.createParser(content);
        return parser;
    }    
}
tophersmith116
  • 432
  • 1
  • 5
  • 19
  • 1
    Hi @tophersmith, this works like a charm. Thanks to you, it solved my more than 3 weeks problem. Thanks once again. – Ankit Ostwal Aug 27 '21 at 11:23
  • Hi @tophersmith, i found on extensive testing that, for simpler json your solution is giving correct result. but as i gor for complex json's solution fails. – Ankit Ostwal Aug 30 '21 at 18:50
  • Hi @tophersmith, if you can throw some insight on this. it would be great. I am trying to figure out. But unable to find solution. Help is appreciated. Thanks. – Ankit Ostwal Aug 31 '21 at 05:03
  • Without more info, I can't really help you. This works well for my cases, but I don't know what you mean by "complex" cases. I'm using the info available in the parser... – tophersmith116 Sep 01 '21 at 15:28
0

I guess you could use the pretty print version of the json, which should return you the json formatted with the line breaks, and work from there.

Jorge V
  • 11
  • 1