1

I created a simple class model (AnchorMetaData), shown below, in which there are two items. One is a list field (Vector3) that cannot be serialized due to how it's made so I created a property for class (SerializableVector3) that could serialize and deserialize. I hoped to use this property with Newtonsoft for saving/loading the model.

The class saves just fine however, when I try to deserialize the model from JSON it calls the AttachedTaskLocations property's getter rather than setter. That makes the field to be initialized to be empty.

I only noticed this by using logs message and making some breakpoints. It never calls the setter when deserializing. Which is very odd as it should function.

Another odd behaviour is that it does pause on setter of x, y, z of SerializableVector3 with the correct values from the file. This is super odd.

I'm working with Unity 2019.1.14 but this should work without it too, just change vector list to something you have.

When I load the shown JSON file, which was created by serializing AnchorMetaData it has zero items in attachedTaskLocations. Why is this happening? Why does the setter not get called?


Class I created to save/load Vector3 is called SerializableVector3. Class I wish to save/load:

[Serializable]
public class AnchorMetaData
{
    // Cannot serialize this.
    [JsonIgnore]
    public List<Vector3> attachedTaskLocations = new List<Vector3>();

    /// <summary>
    /// This property servers as an interface for JSON de-/serialization.
    /// It uses a class that can be serialized by Newtonsoft.
    /// Should not be used in code except for serialization purposes.
    /// </summary>
    [JsonProperty("AttachedTaskLocations")]
    public List<SerializableVector3> AttachedTaskLocations
    {
        get
        {
            Debug.Log("Writing serialized vector.");
            return attachedTaskLocations
                .Select(vector3 => new SerializableVector3(vector3))
                .ToList();
        }
        set
        {
            Debug.Log("Loading serialized vector.");
            attachedTaskLocations = value
                .Select(sVector3 => new Vector3(sVector3.x, sVector3.y, sVector3.z))
                .ToList();
        }
    }

}

Serialized JSON:

{
  "AttachedTaskLocations": [
      {
        "x": 1.0,
        "y": 1.0,
        "z": 1.0
      },
      {
        "x": 1E+12,
        "y": 2.0,
        "z": 3.0
      },
      {
        "x": 0.0,
        "y": 0.0,
        "z": 0.0
      }
    ]
  }

Stack when breakpoint hits the getter on deserialize. Getter call stack

user14492
  • 2,116
  • 3
  • 25
  • 44
  • Is it btw relevant/intended that only `y` is tagged `[JsonProperty]` but `x` and `z` are not? – derHugo Nov 06 '19 at 13:57
  • @derHugo Nice catch. No that was left over before refactoring, it used to be capital; X, Y, Z. It should have same behavior anyways. – user14492 Nov 06 '19 at 14:03
  • An alternative might be using a normal field `List` and use [`ISerializationCallbackReceiver`](https://docs.unity3d.com/ScriptReference/ISerializationCallbackReceiver.html) in order to copy the values between both lists on (de)serialization accordingly – derHugo Nov 06 '19 at 14:06
  • @derHugo I suppose that is a solution. I'm not using Unity's serializer though, I'm using Newtonsoft; which also has something similar. Might have to do this for time being but still would like to know what's happening here. – user14492 Nov 06 '19 at 14:15
  • Side comment: make `AttachedTaskLocations` private (json.net can serialize private members), then you don't need to write such a comment and if name in json match, use `[JsonProperty]` without parameters. In any case, try to create [mcve]. – Sinatr Nov 06 '19 at 14:18
  • @EdPlunkett The setter of `SerializableVector3.x, y, z` do get hit. But the setter of `AnchorMetaData.AttachedTaskLocations` doesn't get hit here. Which is bizzare. – user14492 Nov 06 '19 at 14:20
  • @Sinatr comments are the currency of coders. And I'm a generous man :D – user14492 Nov 06 '19 at 14:21
  • @user14492 I see, thank you. Re-reading your question, that was implied. I've clarified that in the question, hope you don't mind. – 15ee8f99-57ff-4f92-890c-b56153 Nov 06 '19 at 14:21
  • Does the setter get called in any other moment later? – derHugo Nov 06 '19 at 14:24
  • @derHugo No it does not. It never stops in debugger and never prints the log message either. – user14492 Nov 06 '19 at 14:26
  • 1
    I tried adding `private List _atl` and changing get to `{ Debug.Log(...); return _atl; }`. I added `_atl = value;` to the setter you have. Now get is called, and then set. I tried a few variations, and if I had anything meaningful in the getter other than `return _atl;`, the setter was never called. I think Newtonsoft is being a little too clever about serializing properties with backing fields. – 15ee8f99-57ff-4f92-890c-b56153 Nov 06 '19 at 14:31
  • 1
    I should say, "too clever about properties it deems to be calculated". If this is documented, maybe there's an attribute to modify the behavior. Otherwise, the private field solution is goofy but functional, with generous commenting. – 15ee8f99-57ff-4f92-890c-b56153 Nov 06 '19 at 14:33
  • 1
    @EdPlunkett Thanks for taking the time to investigate it. That kinda pointed me in the right direction. So if the getter returns a null when it's called it calls the setter otherwise it doesn't. Weird, I don't know why that does it that way but it does. So good to know. If you want to write up a answer I can solve this until a better solution comes along. <3 – user14492 Nov 06 '19 at 15:49
  • @user14492 Ahhh, now I get it. At least I wasn't 100% wrong! – 15ee8f99-57ff-4f92-890c-b56153 Nov 06 '19 at 15:52
  • 1
    By the way, you can use a custom contract resolver and custom serializer for Unity objects (such as Vector3). The classes to look for (and read the documentation about) are `DefaultContractResolver` and `JsonConverter`. I unfortunately don't have an example on GitHub, but its pretty straight forward. – Draco18s no longer trusts SE Nov 06 '19 at 16:41
  • Looks to be a duplicate of [Why are all the collections in my POCO are null when deserializing some valid json with the .NET Newtonsoft.Json component](https://stackoverflow.com/q/32491966/3744182). The solution of using a surrogate array instead of `List` property should work here also. Agree? – dbc Nov 06 '19 at 19:46
  • @dbc it’s very similar but no. I’ve tried that setting with replace option and it behaved the same way. So the chosen solution isn’t valid. – user14492 Nov 06 '19 at 21:52
  • @user14492 - I'd have to check that, but I still think the 2nd solution of using an array (or `ReadOnlyCollection` come to think of it) should work. – dbc Nov 06 '19 at 22:44
  • @derHugo I found that Vector3 was *de*serializable, but not serializable. The reason is because it contains some properties (such as `normalized`) that return a new `Vector3` instance. And so the serializer tries to serialize those and gets yet more instances... so it ends up throwing a "Self referencing loop detected" exception. You need a converter to work around that issue. – Brian Rogers Nov 07 '19 at 07:31

1 Answers1

4

The reason that your AttachedTaskLocations is empty after deserialization is twofold:

  1. By default, Json.Net will reuse existing object values during deserialization rather than create new ones. So, for properties such as your AttachedTaskLocations list, it will call the getter first, and, finding an existing value, it will then proceed to populate it from the JSON.
  2. The getter of your AttachedTaskLocations doesn't return the same instance each time; it always creates a new instance from the attachedTaskLocations backing field.

So what appears to be happening is this:

  1. The serializer calls the AttachedTaskLocations getter, which returns a new empty list.
  2. The serializer populates that list from the JSON.
  3. The populated list is thrown away (the serializer assumes that the AnchorMetaData instance already has a reference to the list so it never calls the setter).
  4. When you later access the AttachedTaskLocations getter, it returns a new empty list again.

You can change the behavior of the serializer by setting the ObjectCreationHandling setting to Replace. This change alone appears to solve the issue in my testing.

However, I think that you are jumping through a bunch of hoops here to get Vector3 to serialize/deserialize properly when there is a better solution: use a custom JsonConverter. Here is the code you would need for the converter. It is not that much:

public class Vector3Converter : JsonConverter
{
    public override bool CanConvert(Type objectType)
    {
        return objectType == typeof(Vector3);
    }

    public override object ReadJson(JsonReader reader, Type objectType, object existingValue, JsonSerializer serializer)
    {
        if (reader.TokenType == JsonToken.StartObject)
        {
            JObject obj = JObject.Load(reader);
            return new Vector3((float)obj["x"], (float)obj["y"], (float)obj["z"]);
        }
        if (reader.TokenType == JsonToken.Null)
        {
            return null;
        }
        throw new JsonException("Unexpected token type: " + reader.TokenType);
    }

    public override void WriteJson(JsonWriter writer, object value, JsonSerializer serializer)
    {
        if (value != null)
        {
            Vector3 vector = (Vector3)value;
            JObject obj = new JObject(
                new JProperty("x", vector.x),
                new JProperty("y", vector.y),
                new JProperty("z", vector.z)
            );
            obj.WriteTo(writer);
        }
        else
        {
            JValue.CreateNull().WriteTo(writer);
        }
    }
}

With this converter in place you can get rid of the SerializableVector3 class altogether and you can simplify your AnchorMetaData class down to this:

public class AnchorMetaData
{
    [JsonProperty("AttachedTaskLocations")]
    public List<Vector3> AttachedTaskLocations { get; set; } = new List<Vector3>();
}

To use the converter, you can either:

  • pass it to the JsonConvert.SerializeObject()/DeserializeObject() methods;
  • add it to the Converters collection on JsonSerializerSettings and pass the settings to JsonConvert.SerializeObject()/DeserializeObject(), or
  • add it to the Converters collection directly on a JsonSerializer instance.

For example:

var settings = new JsonSerializerSettings();
settings.Converters.Add(new Vector3Converter());
var metaData = JsonConvert.DeserializeObject<AnchorMetaData>(json, settings);

Round-trip demo: https://dotnetfiddle.net/jmYIq9


If you don't have access to the serializer (it's hard to tell from your question whether you are doing the serializing/deserializing in your own code, or whether some third-party component is handling that) then another way to use the converter is via attributes. For a list property like AttachedTaskLocations you can specify the ItemConverterType right in the [JsonProperty] attribute like this:

    [JsonProperty("AttachedTaskLocations", ItemConverterType = typeof(Vector3Converter))]
    public List<Vector3> AttachedTaskLocations { get; set; } = new List<Vector3>();

If you had a single instance property, then you would use a [JsonConverter] attribute instead like this:

    [JsonConverter(typeof(Vector3Converter))]
    public Vector3 SingleVector { get; set; }

Fiddle: https://dotnetfiddle.net/yxwqDL

Brian Rogers
  • 125,747
  • 31
  • 299
  • 300
  • `it always creates a new instance from the attachedTaskLocations backing field` ofcourse! That's it ^^ – derHugo Nov 07 '19 at 07:26
  • Thx for the great explanation. It does work when I don't discard the returned list when it calls getter so good to know how it works. I did try the replace option but it didn't help. It was still only calling the getter not setter like your eg. This was the code I used, anything obvious I'm doing wrong? `serializerSettings.ObjectCreationHandling = ObjectCreationHandling.Replace; string jsonContent = TextFileReaderWriter.ReadTextFromFile(filePath); luSerializedObject = JsonConvert.DeserializeObject(jsonContent, serializerSettings);` – user14492 Nov 07 '19 at 11:25
  • That looks correct to me. I'm at a loss to explain why `ObjectCreationHandling.Replace` wouldn't be working for you. Did you try the converter approach? – Brian Rogers Nov 07 '19 at 15:50
  • Yes. The converter approach works great. Thx for taking the time to answer this. – user14492 Nov 14 '19 at 11:06
  • No problem; glad I could help! – Brian Rogers Nov 14 '19 at 16:06