JSON deserialization for multiple messages design pattern C#?

Question

I have a use case in which a .net app is running in the background and listening for Kafka messages. I will receive various payloads/messages from Kafka. Each payload has different structure and contains different items in it. For each payload, we must deserialize it into its own entity(not always be one to one mapping, there can be some mapping logic for payload to the entity in some cases), each of which has its own table in the database. So, after deserialization, these entities must be saved in a database and also sent to another Kafka topic.

Broadly I have divided this whole flow into 3 parts maintaining SRP. One will be deserialization, second will be database save and third will be Kafka.

I am currently implementing deserialization as shown below:

First payload examples:

{
   "type":"fuel",
   "data":{
      "fueltype":"petrol",
      "mileage":23.76,
      "tankcapacity":37
   }
}

{
   "type":"engine",
   "data":{
      "enginetype":"K series",
      "maxpower":88.50,
      "displacement":1197
   }
}

So these messages are differentiated using the type

For code I thought of using individual parsers for each type

public interface IJsonParser
    {
        Payload Parse(dynamic data);
    }

 public class FuelParser : IJsonParser
    {
        public Payload Parse(dynamic payload)
        {
            Fuel f = new Fuel();
            f.Mileage = (float)payload.data.mileage;
            return f;
        }
    }

    public class EngineParser : IJsonParser
    {
        public Payload Parse(dynamic data)
        {
            Engine e = new Engine();
            return e;
        }
    }



public class Payload
    {
        public int Id { get; set; }
    }

 public class Fuel : Payload
    {
        public string FuelType { get; set; }
        public float Mileage { get; set; }
        public int TankCapacity { get; set; }
    }

 public class Engine : Payload
    {
        public string EngineType { get; set; }
        public float MaxPower { get; set; }
        public int Displacement { get; set; }
    }
 


 public static class JsonParserFactory
    {
        public static IJsonParser GetJsonParser(string type)
        {
            switch (type)
            {
                case "fuel":
                    return new FuelParser();
                case "engine":
                    return new EngineParser();
                default:
                    return null;
            }
        }
    }

string message = "{\r\n\t\"type\": \"fuel\",\r\n\t\"data\":\r\n\t\t{\r\n\t\t\t\"fueltype\": \"petrol\",\r\n\t\t\t\"mileage\": 23.76,\r\n\t\t\t\"tankcapacity\": 37\r\n\t\t}\r\n}";
            dynamic data = JsonConvert.DeserializeObject<object>(message);

            IJsonParser parser = JsonParserFactory.GetJsonParser(data.type.ToString());
            var model = parser.Parse(data);// This model will then be saved in DB as well as sent to another Kafka topic which is the 2nd and 3rd part of the flow.

So based on the type I have create a Factory which is creating the individual parsers.

I just wanted the suggestion if this is a good design. My only concern is in the future there will be multiple types of payloads coming which will then increase the number of parsers as we go along.

Any suggestions?

I'm thinking in defining data as dynamic and then use Automapper to map data to each Entity based on the type — J.Salas, Jan 13 '22 at 12:10
there are some cases where i need to write some logic before mapping payload data to entity data. can we define custom mapping logic in automapper? I created these type based parsers to write those custom logics only for each parser type. — Ricko K, Jan 13 '22 at 12:13
yes, you can lot of logic in the map definition https://codewithmukesh.com/blog/automapper-in-aspnet-core/#Mapping_Complex_Objects — J.Salas, Jan 13 '22 at 12:16
but using automapper what will be better design because on the basis of type i have to create the mapping. So simply just using a condition on type create the respective mapping. because that will create so many conditions and automapper code in one class. Since my actual concern is not mapping but how to structure the code in better way that will do the deserialization for any message type that will be recieved from Kafka — Ricko K, Jan 13 '22 at 12:20
It's a minor detail, but I'd prefer parsing the sting to a `JObject` - `JObject.Parse(message);` rather than using `dynamic`. `dynamic` is not always evil, but once you introduce it, it has a way of spreading. People see it and start doing evil things with it. Actually they do the same thing with `JObject`. — Scott Hannen, Jan 13 '22 at 14:33
as mentioned earlier my question is not abt parsing but how to handle these multiple types effectively — Ricko K, Jan 14 '22 at 01:53
@ScottHannen : Actually I needed an idea how can we create a generic message handler that will handle all the message types of json coming? — Ricko K, Jan 17 '22 at 10:36

vhr · Answer 1 · 2022-01-13T12:29:05.417

It is also possible to use DeserializeObject method which accepts a type. That way you don't need to write any type-specific parser, you just have to find out a type and provide it to that method. Something like this:

public Payload GetPayload(string type, string message)
    {
        switch (type)
        {
            case "fuel":
                return JsonConvert.DeserializeObject(message, typeof(Fuel));
            case "engine":
                return JsonConvert.DeserializeObject(message, typeof(Engine));
            default:
                return null;
        }
    }

You can then decorate your entity classes with JsonProperties attributes (mapping basically) as indicated here (plus use that coverter):

[JsonConverter(typeof(JsonPathConverter))]
class Fuel : Payload
{
   [JsonProperty("data.fueltype")]
   public string FuelType{ get; set; }

   [JsonProperty("data.mileage")]
   public float Mileage { get; set; }
}



string message = "{\r\n\t\"type\": \"fuel\",\r\n\t\"data\":\r\n\t\t{\r\n\t\t\t\"fueltype\": \"petrol\",\r\n\t\t\t\"mileage\": 23.76,\r\n\t\t\t\"tankcapacity\": 37\r\n\t\t}\r\n}";
var type = GetMessageType(); // get type from message i.e. with JObject
var payload = GetPayload(type, message);

there are some cases where i need to write some logic before mapping payload data to entity data. thats why I created these type based parsers to write those custom logics for each parser type. — Ricko K, Jan 13 '22 at 12:11
Nope because not always payload and entity has the same structure so u cant straight forward deseriialize. But my actual concern is not deserialization but the structure of the system which will enable to process any message type create the entity and save the entity into the db. — Ricko K, Jan 13 '22 at 12:33

JSON deserialization for multiple messages design pattern C#?

1 Answers1