extracting data from big JSON

Question

We have a very big JSON, close to 1500 fields. We want to read most of the JSON fields (Most of the time I have to read whole JSON and get 80% of data). What could be the most performance optimized way ? should we use JObject.Parse and then use JsonPath or should we use JsonConvert.DeserializeObject and then use linq queries or are there any other better ways

It looks like you are already aware of a few options, why won't you just try them and measure time using stopwatch to find the most performant way? — J. Tuc, Mar 08 '17 at 09:28
May be this link will be helpful to you http://stackoverflow.com/questions/4611031/convert-json-string-to-c-sharp-object — J.SMTBCJ15, Mar 08 '17 at 09:53

Liu · Accepted Answer · 2017-03-08T10:06:16.053

1

Since you mentioned it's a big JSON, I will suggest you to use a stram reader so you don't need to load the whole JSON into memory

    using (var reader = new JsonTextReader(new StreamReader(stream)))
    {
        while(!(reader.TokenType.Equals(JsonToken.EndObject) && reader.Depth == 0))
        {
            if (reader.TokenType.Equals(JsonToken.PropertyName))  
                 if (!DoSomething(reader))
                    break;
            if (!reader.Read())
                break;
        }
    }

In your DoingSomething method, you can for example, read value from one field

    private bool DoSomething(JsonTextReader reader)
    {
        if(reader.Path.Equals("FieldName_You_Are_Looking_For"))
        {
            string jsonText = JsonTextReaderToJsonText(reader);
            // Do something to read the value
        }
        // return true to continue reading, return false to stop reading
        return true;
    }

Hope it can help you

edited Mar 08 '17 at 10:06

answered Mar 08 '17 at 09:38

Liu

970
7
19

Thanks Jianping Liu, Most of the time I have to read whole JSON and get 80% of data – user3615869 Mar 08 '17 at 10:01
memory wise, stream reader is more efficient than loading entire data into memory. – Liu Mar 08 '17 at 10:03
The only limitation i see that i need to add so many If conditions for checking each path any better way for path checking and retrieving – user3615869 Mar 09 '17 at 11:13

extracting data from big JSON

1 Answers1