Object List at third level not deserializing using Newtonsoft Json.Net (can't find the solution in the documentation)

Question

I have a project where I need to read in a response from a Http server. The response is in Json. The object graph from that json deserializes to works for the most part, however the array at the lowest level fails, leaving a null.

I've created code below that can be pasted into a blank test project and run. The sole test fails and I can't work out why. The sample Json is the const string at the top.

I found that the JavaScriptSerializer from System.Web.Extensions does work (when I use List instead of arrays). However, the Json.Net equivalent does not work. There are two tests in the sample below, the Newtonsoft one fails, but why? What item of Newtonsoft documentation am I missing?

using Microsoft.VisualStudio.TestTools.UnitTesting;
using Newtonsoft.Json;
using System.Collections.Generic;

/// <summary>
/// Unit Test project that also includes a reference to System.Web.Extensions.
/// Also includes Newtonsoft from NuGet.
/// The constant `_downloadRootObjectEg` holds the sample json.
/// </summary>
namespace Savaged
{
    [TestClass]
    public class DownloadDeserialisationTest
    {
        private const string _downloadRootObjectEg = "{ \"error\": \"\", \"success\": true, \"data\": [{ \"data\": [{ \"TextSearched\": \"New product\", \"TextFound\": \"New product\", \"data \": [{ \"x\": 0.585, \"y\": 0.21496437 }, { \"x\": 0.63666666, \"y\": 0.21496437 }, { \"x\": 0.6933333, \"y\": 0.23515439 } ], \"Page\": 16 }, { \"TextSearched\": \"Expiry\", \"TextFound\": \"Expiry\", \"data \": [{ \"x\": 0.6666667, \"y\": 0.16270784 }, { \"x\": 0.7133333, \"y\": 0.16270784 }, { \"x\": 0.7133333, \"y\": 0.18052256 }, { \"x\": 0.6666667, \"y\": 0.18052256 } ], \"Page\": 39 }, { \"TextSearched\": \"Expiry\", \"TextFound\": \"Expiry\", \"data \": [{ \"x\": 0.47833332, \"y\": 0.6686461 }, { \"x\": 0.52166665, \"y\": 0.6686461 }, { \"x\": 0.52166665, \"y\": 0.6864608 }, { \"x\": 0.47833332, \"y\": 0.6864608 } ], \"Page\": 43 } ], \"context\": { \"FileLocation\": \"Product-09-08-2007.pdf\", \"ID\": 1, \"Type\": \"product\" } }, { \"data\": [{ \"TextSearched\": \"New product\", \"TextFound\": \"New product\", \"data \": [{ \"x\": 0.585, \"y\": 0.21496437 }, { \"x\": 0.63666666, \"y\": 0.21496437 }, { \"x\": 0.6933333, \"y\": 0.23515439 }, { \"x\": 0.6433333, \"y\": 0.23515439 } ], \"Page\": 16 }, { \"TextSearched\": \"Expiry\", \"TextFound\": \"Expiry\", \"data \": [{ \"x\": 0.6666667, \"y\": 0.16270784 }, { \"x\": 0.7133333, \"y\": 0.16270784 }, { \"x\": 0.7133333, \"y\": 0.18052256 }, { \"x\": 0.6666667, \"y\": 0.18052256 } ], \"Page\": 39 } ], \"context\": { \"FileLocation\": \"Product-09-08-2007.pdf\", \"ID\": 1, \"Type\": \"product\" } } ], \"count\": 2 }";

        [TestMethod]
        public void DeserialiseTest()
        {
            var downloadRootObject =
                JsonConvert.DeserializeObject<DownloadRootObject>(_downloadRootObjectEg);

            Assert.IsNotNull(downloadRootObject.Data[0].Data[0].Data, "Why?");
        }

        [TestMethod]
        public void JavaScriptSerializerTest()
        {
            var downloadRootObject = new System.Web.Script.Serialization.
                JavaScriptSerializer().Deserialize<DownloadRootObject>(_downloadRootObjectEg);

            Assert.IsNotNull(downloadRootObject.Data[0].Data[0].Data, "Why?");
        }
    }

    #region Concrete implementation

    public abstract class RootObjectBase
    {
        public string Error { get; set; }

        public bool Success { get; set; }
    }

    public class DownloadRootObject : RootObjectBase
    {
        public DownloadRootObject()
        {
            Data = new List<WordSearch>();
        }

        [JsonConstructor]
        public DownloadRootObject(List<WordSearch> data)
        {
            Data = data;
        }

        public List<WordSearch> Data { get; set; }

        public int Count { get; set; }
    }

    public class WordSearch
    {
        public WordSearch()
        {
            Data = new List<Match>();
        }

        [JsonConstructor]
        public WordSearch(Context context, List<Match> data)
        {
            Context = context;
            Data = data;
        }

        public Context Context { get; set; }

        public List<Match> Data { get; set; }
    }

    public class Context
    {
        public string FileLocation { get; set; }

        public int ID { get; set; }

        public string Type { get; set; }
    }

    public class Match
    {
        public Match()
        {
            Data = new List<PointF>();
        }

        [JsonConstructor]
        public Match(List<PointF> data)
        {
            Data = data;
        }

        public int Page { get; set; }

        // TODO switch this to System.Drawing.PointF
        public List<PointF> Data { get; set; }

        public string TextSearched { get; set; }

        public string TextFound { get; set; }
    }

    public class PointF
    {
        public float X { get; set; }

        public float Y { get; set; }
    }

    #endregion
}

All help is much appreciated!

Answer 1

From what I can see, the mentioned list does not get deserialized, because the "data" property on the lowest level has a trailing whitespace in it.

 \"data \": [{ \"x\": 0.585, \"y\": 0.21496437 }

But it should actually be:

 \"data\": [{ \"x\": 0.585, \"y\": 0.21496437 }

Answer 2

@Redstone essentially has the correct answer (upvoted). Your innermost array keys in the JSON are called "data " (with a trailing space) instead of just "data". So what is happening is that the innermost list is actually not getting deserialized at all because the serializers are not able to match the key from the JSON to the Data property in your Match class.

As for why it "works" in the JavaScriptSerializer versus Json.Net-- it doesn't really. Your tests are not quite equivalent. The difference is that you are using different constructors in each case. JavaScriptSerializer does not honor the [JsonConstructor] attribute, so it always calls the default constructor, which creates an empty list in your code. Json.Net calls the other constructor you have marked, which does not create an empty list. Since the deserializer cannot find a match for the data parameter in that constructor, it passes a null value. In your tests you are only testing whether or the resulting list is null, not whether it actually retrieved any values successfully. If you extend your tests, you will see that you are getting an empty list with JavaScriptSerializer and not the actual data points from the JSON.

The best solution is to fix your JSON such that is has the correct data key without the trailing space. If you can't do that (e.g. because you don't own the JSON), then your next best option is to mark the Data property in your Match class with [JsonProperty("data ")]. Json.Net will still pass a null to the data parameter in the constructor (after all, a parameter name cannot contain a space), but it should then find and use the public property accessor to set the list correctly. Note that this solution will not work with JavaScriptSerializer because it doesn't honor the [JsonProperty] attribute either, so if you need to go with that serializer you would probably have to write a custom converter to work around the issue. Json.Net also supports custom converters, so that is another option if you need to ensure the alternate constructor is called with a non-null data parameter. See JSON.net: how to deserialize without using the default constructor? for more on that approach.