Newtonsoft.Json.JsonSerializationException: 'Unexpected token when deserializing object: Comment with Dynamic Object

Question

I am trying to deserialize the JSON to a object using Newtonsoft.Json. Our input json request has commented line.

{
    "paymentMethodToken": "bb3vph6",
    "paymentMethodTypes": "ACH",
    "transactionAmount": "1090.9",
    "transactionType": "Charge",
    "transactionID": "3532464245",
    "merchantAccountID": "643765867",
    "insuredName": "First Last",
    "firstName": "First",
    "lastName": "Last",
    // "sourceUserKey": "example@gmail.com"
    "paymentMethodTokenType": "Durable",
    "paymentType": "Recurring",
    "sourceTransactionId": "OrderACH300"
}

we wanted to ignore/skip the comments while deserializing. I believe Json.Net does this by default.

But I am getting error as

Newtonsoft.Json.JsonSerializationException: 'Unexpected token when deserializing object: Comment. Path 'lastName', line 11

Below is the code I am using for deserialization

string valueFromBody;
using (var streamReader = new StreamReader(bindingContext.HttpContext.Request.Body))
{
   valueFromBody = streamReader.ReadToEnd();
}

//Deserilaize body content to model instance  
var modelType = bindingContext.ModelMetadata.UnderlyingOrModelType;
var modelInstance = JsonConvert.DeserializeObject(valueFromBody, modelType)

Newtonsoft.Json Version: 12.0.3

Model Class

[Serializable]
public class GatewayTransaction : DynamicObject
{
        private CaseInSensitiveDictionary<string, string> customFields = new CaseInSensitiveDictionary<string, string>();

        public string PaymentMethodToken { get; set; }

        public PaymentMethodTypes PaymentMethodTypes { get; set; }

        public string InvoiceId { get; set; }

        public decimal TransactionAmount { get; set; }

        public string ChannelID { get; set; }

        public string TransactionType { get; set; }

        public string TransactionID { get; set; }

        public string MerchantAccountID { get; set; }

        public string InsuredName { get; set; }

        public string FirstName { get; set; }

        public string LastName { get; set; }

        public string SourceUserKey { get; set; }

        public string PaymentMethodTokenType { get; set; }

        public PaymentType PaymentType { get; set; }

        public List<TransactionInfo> PolicyInfo { get; set; }

        public string SourceTransactionId { get; set; }

        [JsonIgnore]
        public CaseInSensitiveDictionary<string, string> CustomFields
        {
            get { return this.customFields; }
        }

        public override bool TryGetMember(GetMemberBinder binder, out object value)
        {
            string stringValue;
            var isFound = this.customFields.TryGetValue(binder.Name, out stringValue);
            value = stringValue;
            return isFound;
        }

        public override bool TrySetMember(SetMemberBinder binder, object value)
        {
            if (value is string)
            {
                this.customFields[binder.Name] = (string)value;
                return true;
            }

            return false;
        }
}

Any help would be appreciated.

@Sergey Its just test data, You could ignore, But the problem is with the commented line. With that i am unable to deserialize — Chandra Mohan, May 04 '21 at 12:52
Correction the dev say that it does support comment both multiline /**/ and single line // and https://stackoverflow.com/a/10325432/6560478 — Drag and Drop, May 04 '21 at 12:53
@DragandDrop got a point, I tried reproducing this with a really simple example and as soon as you introduce comments to the json. Newtonsoft fails to deserialize, i suggest you remove comments from the json before deserializing — Jochem Van Hespen, May 04 '21 at 12:57
There should be unit test for that on the github. If not you may create an ssue and suggest one. — Drag and Drop, May 04 '21 at 12:59
@JochemVanHespen It works if we remove. All i wanted to see is ignoring the commented lines if present — Chandra Mohan, May 04 '21 at 13:01
I cannot reproduce this on .net 4.6.2 or .net 5 with 12.0.3. It works for me. — tymtam, May 04 '21 at 13:05
@ChandraMohan I dont think i quiet understand the issue then, if you'd like to ignore the property can't you just remove it from the model itself? — Jochem Van Hespen, May 04 '21 at 13:10
It was working earlier with the comments. But i have changed the model to derive from DynamicObject. Since then it is not accepting the comments. We have test scripts which are having commented lines. we donot wanted to remove all those comments, because those are helpful to understand the context. — Chandra Mohan, May 04 '21 at 13:12
Can't reproduce, see https://dotnetfiddle.net/Dc6CcQ. Are you sure you are not really using [tag:system.text.json]? If so, see [How can I parse JSON with comments using System.Text.Json?](https://stackoverflow.com/q/59447229/3744182), because `System.Text.Json` does not allow comments by default, support for them must be enabled. — dbc, May 04 '21 at 13:19
OK, it seems to be something related to inheriting from `DynamicObject`, see https://dotnetfiddle.net/GblHeT. This is why you should always provide a [mcve]. Do you really need to inherit from `DynamicObject`? — dbc, May 04 '21 at 13:22
@dbc nice one, so one solution could be to deserialize to an object not inheriting from `DynamicObject` and then mapping those values to one which does inherit from it? — Jochem Van Hespen, May 04 '21 at 13:28
@JochemVanHespen - that would work. Or preloading the JSON into a `JToken` and stripping the comments would work. — dbc, May 04 '21 at 13:36
@ChandraMohan - are you sure you need to inherit from `DynamicObject`? Json.NET doesn't support it very well, see [How can I make Json.NET serialize and deserialize declared properties of custom dynamic types that also implement IDictionary?](https://stackoverflow.com/q/55464757/3744182) and [JsonConvert.DeserializeObject w/ DynamicObject and TypeCreationConverter](https://stackoverflow.com/q/49916136/3744182). You're also not doing it correctly, see [Serialize instance of a class deriving from DynamicObject class](https://stackoverflow.com/q/49118571/3744182). — dbc, May 04 '21 at 13:38
@dbc Now this is becoming overcomplicated. without dynamic object how can i achieve the TryGetMember and TrySetMember functionality by inheriting with object. Sometimes i get new fields in request object. so I wanted to read them as custom fields as shown in the model object. — Chandra Mohan, May 04 '21 at 14:24
@ChandraMohan - you could use `[JsonExtensionData]`. See: [Deserialize json with known and unknown fields](https://stackoverflow.com/q/15253875/3744182) and [How to serialize a Dictionary as part of its parent object using Json.Net](https://stackoverflow.com/a/23786127/3744182). — dbc, May 04 '21 at 14:29
@dbc [JsonExtensionData] is the perfect solution what i was looking. thanks. — Chandra Mohan, May 04 '21 at 16:29

Sangeeth Nandakumar · Answer 1 · 2021-05-04T14:18:44.373

1

A different approach is preprocess your JSON string. Strip out all the comments before de-serializing. You could do something like

static string StripComments(string code)
{
    var re = @"(@(?:""[^""]*"")+|""(?:[^""\n\\]+|\\.)*""|'(?:[^'\n\\]+|\\.)*')|//.*|/\*(?s:.*?)\*/";
    return Regex.Replace(code, re, "$1");
}

Using the above function, You can strip out single and multiline comments from JSON. Then do de-serializing.

Attributed to: Remove all comment (single-/multi-line) & blank lines from source file

edited May 04 '21 at 14:18

answered May 04 '21 at 13:14

Sangeeth Nandakumar

1,362
1
12
23

please use either atribution, link or flag as dupe for that https://stackoverflow.com/questions/9113163/remove-all-comment-single-multi-line-blank-lines-from-source-file/9119463#9119463 and https://stackoverflow.com/a/9119583/6560478 – Drag and Drop May 04 '21 at 13:43
Sure, I missed it on hurry. Included the complete link – Sangeeth Nandakumar May 04 '21 at 14:20

Newtonsoft.Json.JsonSerializationException: 'Unexpected token when deserializing object: Comment with Dynamic Object

1 Answers1