Quickly finding out if element contains JSON formatted data in C# .Net 6

Question

What would be the most efficient way in C# .Net 6 to determine if an element or string contains JSON formatted data. Will have billions of rows to search through and didn't know if using JSON.Net, System.JSON or a regex expression, or other way would be the least computationally costly. Ideally a basis for a .containsJson method that short circuits a fully qualified JSON document with the high likelihood it is JSON formatted data.

The input value you have here is a string which you don't know if it is json string or not. And you need to figure that out using the quickest way possible? Is that correct problem statement? — Chetan, Dec 15 '21 at 12:53
Yes, we will have billions of rows of string data, sometimes blobs from a database and they have lost all relationship with the indexes. Need to quickly sort out if they are json data or not. Once that is done should be able to one by one go though and figure out data types or so I hope. — SuperDave, Dec 15 '21 at 13:04
Does this answer your question? https://stackoverflow.com/questions/14977848/how-to-make-sure-that-string-is-valid-json-using-json-net — Andrew Corrigan, Dec 15 '21 at 13:08
That is a good example of what I am trying to avoid which is a costly Try Catch block. But yes its a long those lines. — SuperDave, Dec 15 '21 at 13:34

score 1 · Accepted Answer · edited Dec 26 '21 at 16:11

Out of the box Json.NET is faster than DataContractJsonSerializer and JavaScriptSerializer. Json.NET supports deserializing directly from stream, deserialize your JSON using a StreamReader reading the JSON string one piece at a time instead of having the entire JSON string loaded into memory.

HttpClient client = new HttpClient();

using (Stream s = client.GetStreamAsync("http://www.test.com/large.json").Result)
using (StreamReader sr = new StreamReader(s))
using (JsonReader reader = new JsonTextReader(sr))
{
    JsonSerializer serializer = new JsonSerializer();

    // read the json from a stream
    // json size doesn't matter because only a small piece is read at a time from the HTTP request
    Person p = serializer.Deserialize<Person>(reader);
}

To keep an application consistently fast, it is important to minimize the amount of time the .NET framework spends performing garbage collection. Allocating too many objects or allocating very large objects can slow down or even halt an application while garbage collection is in progress.

To minimize memory usage and the number of objects allocated, Json.NET supports serializing and deserializing directly to a stream. Reading or writing JSON a piece at a time, instead of having the entire JSON string loaded into memory, is especially important when working with JSON documents greater than 85kb in size to avoid the JSON string ending up in the large object heap.

Please check below link to get better idea:

Sounds like a good thing to try! – SuperDave Dec 15 '21 at 13:50 — SuperDave, Dec 15 '21 at 13:50

Quickly finding out if element contains JSON formatted data in C# .Net 6

1 Answers1