0

I'm trying to do the following..

Not represent JSON in a POCO

Find and Deserialize only the value of a particular property hidden deep within a large JSON using Utf8JsonReader.

Filter for JSON tokens : working

Following is the code that is working.

using System;
using System.IO;
using System.Text;
using System.Text.Json;
namespace ProcessData
{
    class Program
    {

        private static readonly byte[] s_pagetypeUtf8 = Encoding.UTF8.GetBytes("type");
        private static readonly byte[] s_transactionorderidUtf8 = Encoding.UTF8.GetBytes("orderid");
        private static readonly byte[] s_transactionorderamountUtf8 = Encoding.UTF8.GetBytes("amount_to_pay");

static void Main(string[] args)
        {
            string fileName = "D:\\500GBNP\\Projects\\temp1\\temp1.json";
            using var fs = new FileStream(fileName, FileMode.Open, FileAccess.Read);
            using var sr = new StreamReader(fs, Encoding.UTF8);
            string line = String.Empty;
            bool success =false;
            string orderid="";
            decimal orderamount=0;
            while ((line = sr.ReadLine()) != null)
                {
                    ReadOnlySpan<byte> jsonReadOnlySpan = Encoding.UTF8.GetBytes(line);
            
                    var reader = new Utf8JsonReader(jsonReadOnlySpan);

                    while (reader.Read())
                    {
                        JsonTokenType tokenType = reader.TokenType;

                        switch (tokenType)
                        {
                            case JsonTokenType.PropertyName:
                                if (reader.ValueTextEquals(s_pagetypeUtf8))
                                {
                                    // Assume valid JSON, known schema
                                    reader.Read();
                                    if (reader.GetString().Equals("Success Confirmation"))
                                    {
                                        success=true;
                                        
                                    
                                    }
                                }
                                break;
                        }
                    }
                }
            Console.WriteLine($"{success}");
        }

    }
}

The json file contains the following text and is successfully able to find if the line contains page type value Success Confirmation :

{"contexts":{"page":{"type":"Success Confirmation"},"transaction":{"orderid":100002210,"shipping_cost":70,"subtotal":200,"subtotal_incl_tax":200,"tax_cost":0,"amount_to_pay":270,"delivery":{"postcode":"400666","state":"West Bengal","city":"kolkata","country":"IN"},"billing":{"postcode":"400666","state":"West Bengal","city":"kolkata","country":"IN"},"product_details":{"no_of_items":1,"order_items":[{"name":"Designer Red Folral Earrings with White Beads","id":5384,"quantity":1,"price":200,"tax":0,"price_incl_tax":200,"actual_price":400,"percent_discount":50,"category":["New Arrivals","Modern Jewellery","Women","Earrings & Studs","Earrings & Studs","Danglers","View All","Hot Deals","Puja Collection 2016"]}]},"payment_mathod":"Cash On Delivery"}}

What I need to do is if page type is Success Confirmation, get the value of orderid and amount_to_pay. This is where I'm getting stuck.

I tried the following..

if (reader.GetString().Equals("Success Confirmation"))
                                    {
                                        success=true;
                                        reader.Read(); 
                                        if (reader.TokenType == JsonTokenType.EndObject)
                                            {
                                                break;
                                            }
                                            reader.Read();
                                        if (reader.ValueTextEquals(s_transactionorderidUtf8))
                                            {
                                                reader.Read();
                                                orderid=reader.GetString();
                                            }

                                        if (reader.ValueTextEquals(s_transactionorderamountUtf8))
                                            {
                                                reader.Read();
                                                orderamount=reader.GetDecimal();
                                            }
                                    Console.WriteLine($"{orderid}:{orderamount}");
                                    
                                    }

Without the if (reader.TokenType == JsonTokenType.EndObject) codeblock, it throws an error

> Cannot compare the value of a token type 'EndObject' to text.

With the code block it just moves forward

Any help is sincerely appreciated.

Please do ask if I have not been able to explain or you need any further info.

Thanks

Arnab
  • 2,324
  • 6
  • 36
  • 60
  • [Parsing a JSON file with .NET core 3.0/System.text.Json](https://stackoverflow.com/a/55429664/3744182) shows how to stream through a file with `Utf8JsonReader` then deserialize some nested object. Does that meet your needs? – dbc Apr 27 '21 at 15:15
  • @dbc I did see this article.. but I thought it required POCO objects like TestData in var obj = jsonStreamReader.Deserialize(); – Arnab Apr 27 '21 at 15:33
  • As far as github.com/evil-dr-nick/utf8jsonstreamreader is concerned the test code at https://github.com/evil-dr-nick/utf8jsonstreamreader/blob/master/Utf8JsonStreamReader.Test/Utf8JsonStreamReaderTests.fs is written in fsharp and I could not understand the same – Arnab Apr 27 '21 at 15:40
  • `Utf8JsonStreamReader` wraps a `Utf8JsonReader` and has exposes members such as `TokenType` and `GetString()`. It doesn't expose `ValueTextEquals()` but that could trivially be added. – dbc Apr 27 '21 at 15:44
  • @dbc I would appreciate if you could provide an example of Utf8JsonStreamReader without POCO object and get three values - type, orderid and amount_to_pay. Thanks – Arnab Apr 27 '21 at 15:56

1 Answers1

1

What I need to do is if page type is Success Confirmation, get the value of orderid and amount_to_pay.

I think you're overthinking it. You could just map the classes:

class Order {
    public Context contexts { get;set;}
};

class Context {
    public Page page {get; set;}
    public Transaction transaction {get; set;}
}

class Page {
    public string type {get; set;}
}

class Transaction 
{
    public long orderid {get;set;}
    public decimal amount_to_pay {get;set;}
}

Then with

var o = JsonSerializer.Deserialize<Order>(json);

Console.WriteLine(o.contexts.page.type);
Console.WriteLine(o.contexts.transaction.orderid);
Console.WriteLine(o.contexts.transaction.amount_to_pay);

the result is:

Success Confirmation
100002210
270

Option B. With JsonDocument

If you don't wish to create classes then you can use JsonDocument.

using JsonDocument doc = JsonDocument.Parse(json);

var context = doc.RootElement.GetProperty("contexts");
Console.WriteLine(context.GetProperty("page").GetProperty("type").GetString());
Console.WriteLine(context.GetProperty("transaction").GetProperty("orderid").GetUInt64());
Console.WriteLine(context.GetProperty("transaction").GetProperty("amount_to_pay").GetDecimal());

Result:

Success Confirmation
100002210
270
tymtam
  • 31,798
  • 8
  • 86
  • 126
  • Thanks but unfortunately, POCO is just not viable. The json segment I have shown is part of a large json. There are too many of them of diff kinds and they can change. I have no control over the team which does this. I'm aware of JsonDocument and was going to use it if unable to get utf8jsonreader working. Reading extremely large JSON data sets is a forte of the later and is true in my case – Arnab Apr 28 '21 at 06:17
  • Why is poco not viable? If json changes you've in a loosing position, no matter what solution you choose. – tymtam Apr 28 '21 at 06:26
  • What is extremely large json? – tymtam Apr 28 '21 at 06:26
  • Every json line could go upto 4MB and files 10s of GB – Arnab Apr 28 '21 at 09:35