-1

In a real-world situation I know in advance that a particular API endpoint among many will (perhaps temporarily) produce JSON with a list of known minor errors or variations and I want to pass run-time parameters to the JSON de-serializer to allow for this. I am happy to write a JsonConverter for each type to handle variations but am wondering on the best (also most performant) way to pass information to specify the corrections to the de-serialization/JsonConverter at run-time?

I do not want the variations to be detected dynamically inside the JsonConverter as I want to maintain a list of which endpoints have what variations and update this list as things change. We are talking here about a situation where there are dozens of endpoint providers (with support desks) and so I want to raise issues with them in parallel to working around issues in my code.

I do not want to create multiple target classes or JsonConverters for each error variant as the target type is usually complex and the errors are mostly small issues (e.g. Date-time format errors, use of string array instead of space-separated values) which may occur in any combination. I believe corrections are best handled by some kind of message-passing to JsonConverters which can handle the variations.

Here is a simplified example involving claims of a received JSON token. Depending on the provider I want to send a parameter to both the DateTimeOffsetUnixConverter and DateTimeOffsetUnixConverter converters instructing the desired deserialization behaviour. I've included the code for DateTimeOffsetUnixConverter where I've added a property MilliSecondsNotSeconds to allow for two de-serialisation options.

namespace MyNamespace
{
    public class ClaimsSubset
    {
        // Unix epoch integer
        // Sometimes seconds, other times milli-seconds
        [JsonConverter(typeof(DateTimeOffsetUnixConverter))]
        [JsonProperty("iat")]
        public DateTimeOffset Iat { get; set; }
        
        // Sometimes this is string with space-separated values, sometimes string array
        [JsonProperty("scope")]
        [JsonConverter(typeof(StringArrayConverter))]
        public string Scope { get; set; }
    }

    public class DateTimeOffsetUnixConverter : JsonConverter<DateTimeOffset>
    {
        public bool MilliSecondsNotSeconds { get; set; } = false;

        public override void WriteJson(JsonWriter writer, DateTimeOffset value, JsonSerializer serializer)
        {
            long seconds = value.ToUnixTimeSeconds();
            long timeValue = MilliSecondsNotSeconds ? (seconds * 1000) : seconds;
            JToken jt = JToken.FromObject(seconds);

            jt.WriteTo(writer);
        }

        public override DateTimeOffset ReadJson(
            JsonReader reader,
            Type objectType,
            DateTimeOffset existingValue,
            bool hasExistingValue,
            JsonSerializer serializer)
        {
            if (objectType == typeof(DateTimeOffset))
            {
                long timeValue = long.Parse(reader.Value.ToString());
                long seconds = MilliSecondsNotSeconds ? (timeValue / 1000) : timeValue;
                return DateTimeOffset.FromUnixTimeSeconds(seconds);
            }

            throw new NotSupportedException($"The type {objectType} is not supported.");
        }
    }

}

Following a discussion in the comments I tried to use code like the following to configure DateTimeOffsetUnixConverter at the de-serialisation call site (where endpoint providers are known). The code below doesn't work however as Json.Net does not seem to use the provided DateTimeOffsetUnixConverter when de-serialising the properties of ClaimsSubset.

    JsonSerializerSettings jsonSerializerSettings = new JsonSerializerSettings();
    jsonSerializerSettings
        .Converters
        .Add(
            new DateTimeOffsetUnixConverter
            {
                MilliSecondsNotSeconds = true
            });
    JsonConvert.DeserializeObject<ClaimsSubset>(json, jsonSerializerSettings)

Any advice much appreciated!

MFL
  • 69
  • 1
  • 8
  • 3
    without an example of `these minor variations` it's borderline impossible to answer your question – zaitsman Jul 27 '20 at 23:16
  • If the JSON is not **well-formed** then there is not much you can do. If the JSON is well-formed but invalid according to some schema then corrective actions are possible. But we need examples to actually help you. [How to handle both a single item and an array for the same property using JSON.net](https://stackoverflow.com/q/18994685/3744182) is one that comes up frequently. [Supporting multiple custom DateTime formats when deserializing with Json.Net](https://stackoverflow.com/q/51318713/3744182) may also apply. – dbc Jul 27 '20 at 23:52
  • if you have an API that does not produce valid JSON, then i'd suggest to contact whomever wrote the API and hammer them with support requests _until it is fixed_. producing valid JSON is as easy as using one of the hundreds of libraries built to do exactly that. if that's not an option, the next best way would be to write manual fixes automatically applied to the JSON-string, and hoping that doesn't screw it up even more. – Franz Gleichmann Jul 28 '20 at 05:22
  • Thanks all, I'll edit the question with an example. The JSON is well-formed but using a variant of the correct schema (there are sometimes interpretation issues so let's perhaps avoid the term "correct"). Thanks @dbc for your suggested solutions. The problem with these is that they involve detecting from the received JSON how to proceed whereas I do not want choices made dynamically inside the JsonConverter but rather specified and instructed from the call site where the de-serialisation is performed. – MFL Jul 31 '20 at 17:14
  • Thanks @FranzGleichmann for your comment. However there are so many APIs I cannot afford to wait for third-parties to fix things before having a working product. I need to both have a workaround for known issues and a process to get them fixed permanently. I want to fix things inside the JsonConverter based on options specified outside the JsonConverter (to maintain full control). – MFL Jul 31 '20 at 17:15
  • Are you calling `JsonConvert.DeserializeObject()` directly, or is some higher-level framework making the call? If you are calling directly, [Pass additional data to JsonConverter](https://stackoverflow.com/q/53193503/3744182) may be relevant to your needs. – dbc Jul 31 '20 at 17:41
  • Thanks for comment @dbc. I am calling ```JsonConvert.DeserializeObject()```. I had a look at your reference and there's a lot of Visual Basic code, I'll try and work out what's going on there if I can. – MFL Jul 31 '20 at 17:45
  • @dbc, a question. Do you suggest to add constructor parameters to JsonConverter then instance converter before each de-serialisation and add to settings as shown here: https://stackoverflow.com/a/51430812/10453911. Is it okay to have constructor parameters on a JsonConverter? And what if the converter is used more than once in heavily nested object with different parameters in different places? – MFL Jul 31 '20 at 17:55
  • BTW if my question is now clear, please upvote so it can get re-opened. – MFL Jul 31 '20 at 18:00
  • Well you could use the first alternative from https://stackoverflow.com/a/53196799/3744182, namely to pass additional data to your statically applied converters using `StreamingContext.Context`. In the second alternative don't use statically applied converters and instead pass converters in runtime. That might work for `DateTimeOffsetUnixConverter` but probably wouldn't work for `StringArrayConverter` since your data model might well have string properties to which the converter should not be applied. – dbc Jul 31 '20 at 18:00
  • For someone to answer the question, there needs to be enough code present to represent a [mcve] which the answerer can then modify to meet your requirements. You haven't quite done that yet as we can't see `StringArrayConverter` or `DateTimeOffsetUnixConverter`. Can you share one of the two converters, say `StringArrayConverter`, and indicate precisely what you need it to do? – dbc Jul 31 '20 at 18:02
  • Thanks @dbc. Thinking about the multiple use of converter problem, that will occur in both cases right as either constructor parameters or parameters in StreamingContext.Context will collide for the same JsonConverter. So I think multiple JsonConverters will be needed in this case. – MFL Jul 31 '20 at 18:06
  • Sure @dbc will try to generate suitable update to question. – MFL Aug 01 '20 at 13:56
  • @dbc I've improved my question as requested and also shown what I tried based on our discussion. I'm not sure StreamingContext.Context is the best solution as it's a single dynamic object similar to a global namespace whereas I want to pass different parameters to different converters. – MFL Aug 01 '20 at 14:54

1 Answers1

1

Okay, here's what I've done to solve the problem.... shown by example for run-time configuration of DateTimeOffsetUnixConverter. Any comments/suggestions for improvement welcome.

I created a new abstract class JsonConverterWithOptions and an options Enum DateTimeOffsetUnixConverterOptions to support DateTimeOffsetUnixConverter as shown below.

    public abstract class JsonConverterWithOptions<TClass, TOptionsEnum> : JsonConverter<TClass>
        where TOptionsEnum : struct, Enum
    {
        protected JsonConverterWithOptions(TOptionsEnum activeOptions = default)
        {
            ActiveOptions = activeOptions;
        }

        protected TOptionsEnum ActiveOptions { get; }

        protected TOptionsEnum getOptions(JsonSerializer serializer)
        {
            List<string> contextOptions = serializer.Context.Context as List<string>;
            if (contextOptions is null || ActiveOptions.Equals(default(TOptionsEnum)))
            {
                return default;
            }

            int options = 0;
            Array values = Enum.GetValues(typeof(TOptionsEnum));
            foreach (TOptionsEnum item in values)
            {
                bool optionActive = ActiveOptions.HasFlag(item);
                bool optionSelected = contextOptions.Contains($"{typeof(TOptionsEnum).Name}:{item.ToString()}");
                if (optionActive && optionSelected)
                {
                    options |= (int) (object) item;
                }
            }

            return (TOptionsEnum) (object) options;
        }
    }

    [Flags]
    public enum DateTimeOffsetUnixConverterOptions
    {
        None = 0,
        MilliSecondsNotSeconds = 1
    }

    public class
        DateTimeOffsetUnixConverter : JsonConverterWithOptions<DateTimeOffset, DateTimeOffsetUnixConverterOptions>
    {
        public DateTimeOffsetUnixConverter() { }

        public DateTimeOffsetUnixConverter(DateTimeOffsetUnixConverterOptions activeOptions) : base(activeOptions) { }


        public override void WriteJson(JsonWriter writer, DateTimeOffset value, JsonSerializer serializer)
        {
            DateTimeOffsetUnixConverterOptions options = getOptions(serializer);
            long seconds = value.ToUnixTimeSeconds();
            long timeValue = options.HasFlag(DateTimeOffsetUnixConverterOptions.MilliSecondsNotSeconds)
                ? seconds * 1000
                : seconds;
            JToken jt = JToken.FromObject(seconds);

            jt.WriteTo(writer);
        }

        public override DateTimeOffset ReadJson(
            JsonReader reader,
            Type objectType,
            DateTimeOffset existingValue,
            bool hasExistingValue,
            JsonSerializer serializer)
        {
            if (objectType == typeof(DateTimeOffset))
            {
                DateTimeOffsetUnixConverterOptions options = getOptions(serializer);
                long timeValue = long.Parse(reader.Value.ToString());
                long seconds = options.HasFlag(DateTimeOffsetUnixConverterOptions.MilliSecondsNotSeconds)
                    ? timeValue / 1000
                    : timeValue;
                return DateTimeOffset.FromUnixTimeSeconds(seconds);
            }

            throw new NotSupportedException($"The type {objectType} is not supported.");
        }
    }

Then in the target class ClaimsSubset I can limit what options are available to use at run-time ("Active") for any property as shown below. This is important to ensure options are ignored by default e.g. for another property using the same converter.

public class ClaimsSubset
    {
        // Unix epoch integer
        // Sometimes seconds, other times milli-seconds
        [JsonConverter(
            converterType: typeof(DateTimeOffsetUnixConverter),
            DateTimeOffsetUnixConverterOptions.MilliSecondsNotSeconds)]
        [JsonProperty("iat")]
        public DateTimeOffset Iat { get; set; }
        
        // Sometimes this is string with space-separated values, sometimes string array
        [JsonProperty("scope")]
        [JsonConverter(typeof(StringArrayConverter))]
        public string Scope { get; set; }
    }

Then at run-time I can select any active options according to endpoint provider....

    JsonSerializerSettings jsonSerializerSettings = new JsonSerializerSettings();
    jsonSerializerSettings.Context = new StreamingContext(
        state: StreamingContextStates.All,
        additional: new List<string>
        {
            $"{nameof(DateTimeOffsetUnixConverterOptions)}:{nameof(DateTimeOffsetUnixConverterOptions.MilliSecondsNotSeconds)}"
        });
    JsonConvert.DeserializeObject<ClaimsSubset>(json, jsonSerializerSettings)

This probably needs a bit more work but is the best I came up with after much trial and error. As stated earlier, all comments welcome and I still very much welcome any better solution! Also hoping this will be compatible with future migration to System.Text.Json....

MFL
  • 69
  • 1
  • 8