Generic method of modifying JSON before being returned to client

Question

I'm after a generic method that allows me to modify the JSON of an object being returned to the client, specifically the removal of certain properties in returned objects. Similar to what is suggested here.

The modifications are non-deterministic in that they are determined per-request, based on rules associated with the user. So this not suited to a method that is cached.

I've reviewed several methods. The most obvious choice would be a JsonConverter, however there are problems with this, as listed here, here and here.

The main problem with this approach is that calling JToken.FromObject in WriteJson to get the JSON for the specific value, recursively calls the same JsonConverter, resulting in a loop.

I've tried a variant of the solution listed here which provides a method of temporarily disabling CanWrite to prevent the looping issue. However it doesn't seem to work for more than one concurrent request. A single instance of the JsonConverter is being shared between multiple threads that are changing and reading the state of the CanWrite property at different times, causing inconsistent results.

I've also tried using a different serializer in WriteJson (i.e. other than the one supplied to the method) however this doesn't support recursion (because that serializer doesn't use my JsonConverter) so any nested items aren't processed by my JsonConverter. Removing my JsonConverter from the default serializer's converters collection has the same problem.

Basically, if I want to be able to recursively process my model object, I'm going to get the self referencing loop issue.

Ideally, JToken.FromObject would have some way of selectivly NOT calling the JsonConverter on the object itself, but still applying it to any child objects during serialization. I got half way to fixing this by modifying CanConvert to set CanWrite to true, only if the object passed to CanConvert was a different type to the last object passed to WriteJson.

However for this to work I would need a per-request scoped JsonConverter (for the same threading reasons above), but I can't see how to get that.

Here is a sample of what I have:-

using Newtonsoft.Json;
using Newtonsoft.Json.Linq;
using Newtonsoft.Json.Serialization;
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;

namespace Test
{
    public class TestConverter : JsonConverter
    {
        bool CannotWrite { get; set; }

        public override bool CanWrite { get { return !CannotWrite; } }

        public override void WriteJson(JsonWriter writer, object value, JsonSerializer serializer)
        {
            JToken token;

            //----------------------------------------

            // this works; but because it's (i think) creating a new
            // serializer inside the FromObject method
            // which means any nested objects won't get processed

            //token = JToken.FromObject(value);

            //----------------------------------------

            // this creates loop because calling FromObject will cause this
            // same JsonConverter to get called on the same object again

            //token = JToken.FromObject(value, serializer);

            //----------------------------------------

            // this gets around the loop issue, but the JsonConverter will
            // not apply to any nested objects

            //serializer.Converters.Remove(this);
            //token = JToken.FromObject(value, serializer);

            //----------------------------------------

            // see https://stackoverflow.com/a/29720068/1196867
            //
            // this works as it allows us to use the same serializer, but
            // temporarily sets CanWrite to false so the invocation of
            // FromObject doesn't cause a loop
            //
            // this also means we can't process nested objects, however
            // see below in CanConvert for a potential workaround.

            using (new PushValue<bool>(true, () => CannotWrite, (cantWrite) => CannotWrite = cantWrite))
            {
                token = JToken.FromObject(value, serializer);
            }

            // store the type of this value so we can check it in CanConvert when called for any nested objects
            this.currentType = value.GetType();

            //----------------------------------------

            // in practice this would be obtained dynamically
            string[] omit = new string[] { "Name" };

            JObject jObject = token as JObject;

            foreach (JProperty property in jObject.Properties().Where(p => omit.Contains(p.Name, StringComparer.OrdinalIgnoreCase)).ToList())
            {
                property.Remove();
            }

            token.WriteTo(writer);
        }

        private Type currentType;

        public override bool CanConvert(Type objectType)
        {
            if (typeof(Inua.WebApi.Authentication.IUser).IsAssignableFrom(objectType))
            {
                // if objectType is different to the type which is currently being processed,
                // then set CanWrite to true, so this JsonConverter will apply to any nested
                // objects that we want to process
                if (this.currentType != null && this.currentType != objectType)
                {
                    this.CannotWrite = false;
                }

                return true;
            }

            return false;
        }

        public override bool CanRead { get { return false; } }

        public override object ReadJson(JsonReader reader, Type objectType, object existingValue, JsonSerializer serializer)
        {
            throw new NotImplementedException();
        }
    }
}

Options I've considered:-

1. Use a custom JsonConverter, but build the JSON manually instead of leveraging JToken.FromObject (adds a lot of complexity)
2. Using an ActionFilterAttribute and removing properties from the model prior to serialization (I'd need to use reflection for every request to modify the model object)
3. Using ShouldSerialzeX() methods in my models that perform lookups (not easily maintainable)
4. Using a custom ContractResolver (this suffers from the same caching issue, even if I use the now obsolete constructor in DefaultContractResolver that sets "shareCache" to false)

Can anyone suggest:-

• A way to make JsonConverters per-request
• Assuming it can't be made per-request, a way to fix the threading issue with JsonConverter
• An alternative to JsonConverter that allows me to globally inspect and modify JSON objects before they are returned to the client that doesn't rely on a lot of reflection overhead
• Something else?

Thanks in advance for taking the time to read this.

Answer 1

One possibility to fix the TestConverter for multi-threaded, multi-type scenarios would be to create a [ThreadStatic] stack of types being serialized. Then, in CanConvert, return false if the candidate type is of the same type as the type on top of the stack.

Note this only works when the converter is included in JsonSerializerSettings.Converters. If the converter is applied directly to a class or property with, say,

    [JsonConverter(typeof(TestConverter<Inua.WebApi.Authentication.IUser>))]

Then infinite recursion will still occur since CanConvert is not called for directly applied converters.

Thus:

public class TestConverter<TBaseType> : JsonConverter
{
    [ThreadStatic]
    static Stack<Type> typeStack;

    static Stack<Type> TypeStack { get { return typeStack = (typeStack ?? new Stack<Type>()); } }

    public override void WriteJson(JsonWriter writer, object value, JsonSerializer serializer)
    {
        JToken token;

        using (TypeStack.PushUsing(value.GetType()))
        {
            token = JToken.FromObject(value, serializer);
        }

        // in practice this would be obtained dynamically
        string[] omit = new string[] { "Name" };

        JObject jObject = token as JObject;

        foreach (JProperty property in jObject.Properties().Where(p => omit.Contains(p.Name, StringComparer.OrdinalIgnoreCase)).ToList())
        {
            property.Remove();
        }

        token.WriteTo(writer);
    }

    public override bool CanConvert(Type objectType)
    {
        if (typeof(TBaseType).IsAssignableFrom(objectType))
        {
            return TypeStack.PeekOrDefault() != objectType;
        }

        return false;
    }

    public override bool CanRead { get { return false; } }

    public override object ReadJson(JsonReader reader, Type objectType, object existingValue, JsonSerializer serializer)
    {
        throw new NotImplementedException();
    }
}

public static class StackExtensions
{
    public struct PushValue<T> : IDisposable
    {
        readonly Stack<T> stack;

        public PushValue(T value, Stack<T> stack)
        {
            this.stack = stack;
            stack.Push(value);
        }

        #region IDisposable Members

        // By using a disposable struct we avoid the overhead of allocating and freeing an instance of a finalizable class.
        public void Dispose()
        {
            if (stack != null)
                stack.Pop();
        }

        #endregion
    }

    public static T PeekOrDefault<T>(this Stack<T> stack)
    {
        if (stack == null)
            throw new ArgumentNullException();
        if (stack.Count == 0)
            return default(T);
        return stack.Peek();
    }

    public static PushValue<T> PushUsing<T>(this Stack<T> stack, T value)
    {
        if (stack == null)
            throw new ArgumentNullException();
        return new PushValue<T>(value, stack);
    }
}

In your case TBaseType would be Inua.WebApi.Authentication.IUser.

Prototype fiddle.

Answer 2

In typical fashion, the process of posing the question caused me to take a fresh take on the problem.

I have found one possible work-around: creating a custom MediaTypeFormatter.

With help from here and here, a potential solution:-

using Newtonsoft.Json;
using Newtonsoft.Json.Serialization;
using System;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using System.Net.Http.Formatting;
using System.Text;
using System.Threading.Tasks;

namespace Test
{
    public class TestFormatter : MediaTypeFormatter
    {
        public TestFormatter()
        {
            SupportedMediaTypes.Add(new System.Net.Http.Headers.MediaTypeHeaderValue("application/json"));
        }

        public override bool CanReadType(Type type)
        {
            return false;
        }

        public override bool CanWriteType(Type type)
        {
            return true;
        }

        public override Task WriteToStreamAsync(Type type, object value, System.IO.Stream writeStream, System.Net.Http.HttpContent content, System.Net.TransportContext transportContext)
        {
            JsonSerializer serializer = new JsonSerializer();

            serializer.ContractResolver = new CamelCasePropertyNamesContractResolver();
            serializer.Converters.Add(new TestConverter());

            return Task.Factory.StartNew(() =>
            {
                using (JsonTextWriter jsonTextWriter = new JsonTextWriter(new StreamWriter(writeStream, Encoding.ASCII)) { CloseOutput = false })
                {
                    serializer.Serialize(jsonTextWriter, value);
                    jsonTextWriter.Flush();
                }
            });
        }
    }
}

and then configure the app to use it:-

// insert at 0 so it runs before System.Net.Http.Formatting.JsonMediaTypeFormatter
config.Formatters.Insert(0, new TestFormatter());

This creates a new instance of my JsonConverter for each request, which in combination with the other fixes in the original post, seem to solve the issue.

This is probably not the best way of doing this, so I'll leave this open for some better suggestions, or until I realise why this isn't going to work.