fastJSON Deserialization List

Question

So sort of a build on from a previous question of mine. I am trying to save a blueprint, which is just a heap of settings for a gameobject/Entity. I'm now storing the components (And their settings) as a List < IEntityComponent > (IEntityComponent is the interface for any component) wrapped in a class called ComponentTable. I only want to serialize the list, and all the private stuff is not serialized, and is just for faster look ups (At the price of memory). This serializes properly and even deserializes without any errors, but i noticed the componentTable isn't deserializing PROPERLY.

It creates an instance of ComponentTable, but never actually adds the values to it. So instead of a Component table containing CameraComponent, VelocityComponent and InputComponent, it just just an empty ComponentTable.

{
 "$types" : {
  "ECS.Framework.Collections.Blueprint, Assembly-CSharp, Version=0.0.0.0, Culture=neutral, PublicKeyToken=null" : "1",
  "ECS.Features.Core.CameraComponent, Assembly-CSharp, Version=0.0.0.0, Culture=neutral, PublicKeyToken=null" : "2",
  "ECS.Features.Core.VelocityComponent, Assembly-CSharp, Version=0.0.0.0, Culture=neutral, PublicKeyToken=null" : "3",
  "InputComponent, Assembly-CSharp, Version=0.0.0.0, Culture=neutral, PublicKeyToken=null" : "4"
 },
 "$type" : "1",
 "Components" : [
  {
     "$type" : "2",
     "Tag" : "MainCamera",
     "Test" : "0, 0, 0",
     "BackgroundColour" : "0, 0, 1, 1",
     "ViewportRect" : "10, 10 : 10, 10",
     "Orthographic" : false,
     "FieldOfView" : 60,
     "OrthoSize" : 5,
     "Depth" : 0,
     "OcclusionCulling" : true,
     "HDR" : false,
     "Enabled" : true
  },
  {
     "$type" : "3",
     "Enabled" : true,
     "CurrentVelocity" : "0, 0, 0"
  },
  {
     "$type" : "4",
     "TEST" : 0,
     "Enabled" : true
  }
 ],
 "Children" : [

 ],
 "Parent" : ""
}

This is how it saves, so it seems like it is saving correctly. I am only controlling the serialization/serialization of the Vectors, Rect and colors, as any unity value types cause errors.

I believe it is serializing properly, but for some reason it is not deserializing into the componentTable. Does anyone know if fastJSON has problems with this kind of inheritance (Making a class inherit from a List< customClass >?

Ideally i would have it inherit as a Dictionary< Type, IEntityComponent >, but fastJSON wont serialize the Type, just saves it as 'System.Mono' then causes an error when serializing.

Edit: Here are the blueprint and component Table classes

public sealed class Blueprint 
{
    public ComponentTable Components { get; private set; }

    public List<string> Children { get; set; }

    public string Parent { get; set; }

    public Blueprint()
    {
        Components = new ComponentTable();

        Children = new List<string>();
        Parent = "";
    }

    public Blueprint(Blueprint _blueprint)
    {
        Children = new List<string>(_blueprint.Children);

        Parent = _blueprint.Parent;
    }
}


public class ComponentTable : List<IEntityComponent>
{
    private Dictionary<Type, IEntityComponent> Components { get; set; }

    #region Constructors

    public ComponentTable()
    {
        Components = new Dictionary<Type, IEntityComponent>();
    }

    #endregion

    #region Base Function Overrides

    public void Add(Type _type)
    {
        if (Components.ContainsKey(_type))
            return;

        InternalAdd(_type, (IEntityComponent)Activator.CreateInstance(_type));
    }
    public new void Add(IEntityComponent _component)
    {
        InternalAdd(_component.GetType(), _component);
    }
    public void Add<T>() where T : IEntityComponent
    {
        Add(typeof(T));
    }
    private void InternalAdd(Type _type, IEntityComponent _component)
    {
        if (Components.ContainsKey(_type))
            throw new InvalidOperationException("Component already contained");

        Components.Add(_type, _component);
        base.Add(_component);
    }

    public bool Remove(Type _type)
    {
        if (Components.ContainsKey(_type))
            return InternalRemove(_type, Components[_type]);
        return false;
    }
    public new bool Remove(IEntityComponent _component)
    {
        return InternalRemove(_component.GetType(), _component);
    }
    public bool Remove<T>() where T : IEntityComponent
    {
        return Remove(typeof(T));
    }
    private bool InternalRemove(Type _type, IEntityComponent _component)
    {
        if (!Components.ContainsKey(_type))
            return false;

        Components.Remove(_type);
        return base.Remove(_component);
    }

    public IEntityComponent Get(Type _type)
    {
        if (Contains(_type))
            return Components[_type];
        return null;
    }
    public T Get<T>() where T : IEntityComponent
    {
        return (T)Get(typeof(T));
    }

    public bool TryGetValue(Type _type, out IEntityComponent _component)
    {
        return Components.TryGetValue(_type, out _component);
    }
    public bool TryGetValue<T>(out IEntityComponent _component) where T : IEntityComponent
    {
        return TryGetValue(typeof(T), out _component);
    }

    public bool Contains(Type _type)
    {
        return Components.ContainsKey(_type);
    }
    public new bool Contains(IEntityComponent _component)
    {
        return Contains(_component.GetType());
    }
    public bool Contains<T>() where T : IEntityComponent
    {
        return Contains(typeof(T));
    }

    #endregion

}

Answer 1

I tested this a bit on Microsoft .Net, and found the following issues:

1. fastJSON will not deserialize the Components property unless it has a public setter:

   public sealed class Blueprint 
   {
       public ComponentTable Components { get; set; }
   

   There doesn't appear to be any sort of configuration option to work around this. From Reflection.cs you can see the method to create the setter delegate returns null if the setter is not public:

   internal static GenericSetter CreateSetMethod(Type type, PropertyInfo propertyInfo)
   {
       MethodInfo setMethod = propertyInfo.GetSetMethod();
       if (setMethod == null)
           return null;
   

2. fastJSON does indeed appear to be unable to deserialize a subclass of List<T> -- or any other non-array collection class -- that is not generic. Inside the deserializer there is the following check:

   if (pi.IsGenericType && pi.IsValueType == false && v is List<object>)
       oset = CreateGenericList((List<object>)v, pi.pt, pi.bt, globaltypes);
   

   As you can see, it checks for the target type being generic, rather than the target type or one of its base types being generic.

   You could work around this by making your ComponentTable be generic:

   public class ComponentTable<TEntityComponent> : List<TEntityComponent> where TEntityComponent : IEntityComponent
   {
       private Dictionary<Type, TEntityComponent> Components { get; set; }
   
       #region Constructors
   
       public ComponentTable()
       {
           Components = new Dictionary<Type, TEntityComponent>();
       }
   
       #endregion
   
       #region Base Function Overrides
   
       public void Add(Type _type)
       {
           if (Components.ContainsKey(_type))
               return;
   
           InternalAdd(_type, (TEntityComponent)Activator.CreateInstance(_type));
       }
       public new void Add(TEntityComponent _component)
       {
           InternalAdd(_component.GetType(), _component);
       }
       public void Add<T>() where T : IEntityComponent
       {
           Add(typeof(T));
       }
       private void InternalAdd(Type _type, TEntityComponent _component)
       {
           if (Components.ContainsKey(_type))
               throw new InvalidOperationException("Component already contained");
   
           Components.Add(_type, _component);
           base.Add(_component);
       }
   
       public bool Remove(Type _type)
       {
           if (Components.ContainsKey(_type))
               return InternalRemove(_type, Components[_type]);
           return false;
       }
       public new bool Remove(TEntityComponent _component)
       {
           return InternalRemove(_component.GetType(), _component);
       }
       public bool Remove<T>() where T : IEntityComponent
       {
           return Remove(typeof(T));
       }
       private bool InternalRemove(Type _type, TEntityComponent _component)
       {
           if (!Components.ContainsKey(_type))
               return false;
   
           Components.Remove(_type);
           return base.Remove(_component);
       }
   
       public IEntityComponent Get(Type _type)
       {
           if (Contains(_type))
               return Components[_type];
           return null;
       }
       public T Get<T>() where T : IEntityComponent
       {
           return (T)Get(typeof(T));
       }
   
       public bool TryGetValue(Type _type, out TEntityComponent _component)
       {
           return Components.TryGetValue(_type, out _component);
       }
       public bool TryGetValue<T>(out TEntityComponent _component) where T : IEntityComponent
       {
           return TryGetValue(typeof(T), out _component);
       }
   
       public bool Contains(Type _type)
       {
           return Components.ContainsKey(_type);
       }
       public new bool Contains(TEntityComponent _component)
       {
           return Contains(_component.GetType());
       }
       public bool Contains<T>() where T : IEntityComponent
       {
           return Contains(typeof(T));
       }
   
       #endregion
   }
   

   Then change Blueprint to be:

   public sealed class Blueprint
   {
       public ComponentTable<IEntityComponent> Components { get; set; }
   

   And the list contents will be deserialized. However...

3. Your ComponentTable is inheriting from List<T> and needs to override Add(). But Add() is not virtual, and so you are using public new Add() instead. The problem here is that if anybody casts your class to a List<T> and calls Add(), your method won't get called. In particular, fastJSON will not call it during deserialization! So your type dictionary is never initialized during deserialization, defeating the purpose.

   What it appears you are doing is re-inventing KeyedByTypeCollection<TItem>. Rather than do that, you can just use it. Using this class, your ComponentTable becomes very simple:

   public class ComponentTable<TEntityComponent> : KeyedByTypeCollection<TEntityComponent> where TEntityComponent : IEntityComponent
   {
       public void Add(Type _type)
       {
           if (Contains(_type))
               return;
           Add((TEntityComponent)Activator.CreateInstance(_type));
       }
   
       public void Add<T>() where T : IEntityComponent, new()
       {
           Add(typeof(T));
       }
   }
   

   Now the collection can be serialized and deserialized with no data corruption on Microsoft .Net. However...

4. I'm not sure that KeyedByTypeCollection<TItem> exists in unity. If not, you may need to port it. See the reference source KeyedByTypeCollection.cs and the base class source keyedcollection.cs. One attempt to do it can be found here: Alternative to KeyedByTypeCollection in Mono .Net.

5. As an alternative, you might consider using Json.NET. Json.NET supports serializing polymorphic types via the TypeNameHandling setting. But, is it available on unity?

   Google suggests that there are some ports for unity. There are also a variety of questions here tagged with both Json.NET and unity3d, for instance JSON .Net Unity Deserialize dictionary of anonymous objects. So, you might investigate that option further.