Moshi with duplicate fields

Question

I've got a json response that looks like this for some cases:

{
      "id" : 12345,
      "events": [
         {
            "desc": "Bla bla"
             ...
         }, 
         {
            "desc": "Yada yada",
            ...
         },
       ]

    }

Whilst for some other scenarios it looks like this:

{
  "id" : 12345,
  "events": {
     "desc": "Bla bla"
     ...
  },
  "events" : {
    "desc": "Yada yada"
    ...
  },
}

That is, sometimes events will be an array, sometimes events is duplicated with multiple values. This throws the following exception using moshi + retrofit:

    2019-12-30 13:58:20.458 21419-21419/com.myapp.android.debug E/DetailFragment$loadUp: Multiple values for 'events' at $[0].events
2019-12-30 13:58:20.458 21419-21419/com.myapp.android.debug E/DetailFragment$loadUp: com.squareup.moshi.JsonDataException: Multiple values for 'events' at $[0].events
2019-12-30 13:58:20.458 21419-21419/com.myapp.android.debug E/DetailFragment$loadUp:     at com.squareup.moshi.kotlin.reflect.KotlinJsonAdapter.fromJson(KotlinJsonAdapter.kt:80)
2019-12-30 13:58:20.458 21419-21419/com.myapp.android.debug E/DetailFragment$loadUp:     at com.squareup.moshi.internal.NullSafeJsonAdapter.fromJson(NullSafeJsonAdapter.java:40)
2019-12-30 13:58:20.458 21419-21419/com.myapp.android.debug E/DetailFragment$loadUp:     at com.squareup.moshi.CollectionJsonAdapter.fromJson(CollectionJsonAdapter.java:76)
2019-12-30 13:58:20.458 21419-21419/com.myapp.android.debug E/DetailFragment$loadUp:     at com.squareup.moshi.CollectionJsonAdapter$2.fromJson(CollectionJsonAdapter.java:53)
2019-12-30 13:58:20.458 21419-21419/com.myapp.android.debug E/DetailFragment$loadUp:     at com.squareup.moshi.internal.NullSafeJsonAdapter.fromJson(NullSafeJsonAdapter.java:40)
2019-12-30 13:58:20.458 21419-21419/com.myapp.android.debug E/DetailFragment$loadUp:     at retrofit2.converter.moshi.MoshiResponseBodyConverter.convert(MoshiResponseBodyConverter.java:45)
2019-12-30 13:58:20.458 21419-21419/com.myapp.android.debug E/DetailFragment$loadUp:     at retrofit2.converter.moshi.MoshiResponseBodyConverter.convert(MoshiResponseBodyConverter.java:27)
2019-12-30 13:58:20.458 21419-21419/com.myapp.android.debug E/DetailFragment$loadUp:     at retrofit2.OkHttpCall.parseResponse(OkHttpCall.java:225)
2019-12-30 13:58:20.458 21419-21419/com.myapp.android.debug E/DetailFragment$loadUp:     at retrofit2.OkHttpCall.execute(OkHttpCall.java:188)
2019-12-30 13:58:20.459 21419-21419/com.myapp.android.debug E/DetailFragment$loadUp:     at retrofit2.adapter.rxjava2.CallExecuteObservable.subscribeActual(CallExecuteObservable.java:45)
2019-12-30 13:58:20.459 21419-21419/com.myapp.android.debug E/DetailFragment$loadUp:     at io.reactivex.Observable.subscribe(Observable.java:11194)
2019-12-30 13:58:20.459 21419-21419/com.myapp.android.debug E/DetailFragment$loadUp:     at retrofit2.adapter.rxjava2.BodyObservable.subscribeActual(BodyObservable.java:34)
2019-12-30 13:58:20.459 21419-21419/com.myapp.android.debug E/DetailFragment$loadUp:     at io.reactivex.Observable.subscribe(Observable.java:11194)
2019-12-30 13:58:20.459 21419-21419/com.myapp.android.debug E/DetailFragment$loadUp:     at io.reactivex.internal.operators.observable.ObservableSingleSingle.subscribeActual(ObservableSingleSingle.java:35)
2019-12-30 13:58:20.459 21419-21419/com.myapp.android.debug E/DetailFragment$loadUp:     at io.reactivex.Single.subscribe(Single.java:3096)
2019-12-30 13:58:20.459 21419-21419/com.myapp.android.debug E/DetailFragment$loadUp:     at io.reactivex.internal.operators.single.SingleFlatMap$SingleFlatMapCallback.onSuccess(SingleFlatMap.java:84)
2019-12-30 13:58:20.459 21419-21419/com.myapp.android.debug E/DetailFragment$loadUp:     at io.reactivex.internal.operators.single.SingleDoOnSuccess$DoOnSuccess.onSuccess(SingleDoOnSuccess.java:59)
2019-12-30 13:58:20.459 21419-21419/com.myapp.android.debug E/DetailFragment$loadUp:     at io.reactivex.internal.operators.single.SingleFromCallable.subscribeActual(SingleFromCallable.java:56)
2019-12-30 13:58:20.459 21419-21419/com.myapp.android.debug E/DetailFragment$loadUp:     at io.reactivex.Single.subscribe(Single.java:3096)
2019-12-30 13:58:20.459 21419-21419/com.myapp.android.debug E/DetailFragment$loadUp:     at io.reactivex.internal.operators.single.SingleDoOnSuccess.subscribeActual(SingleDoOnSuccess.java:35)
2019-12-30 13:58:20.459 21419-21419/com.myapp.android.debug E/DetailFragment$loadUp:     at io.reactivex.Single.subscribe(Single.java:3096)
2019-12-30 13:58:20.459 21419-21419/com.myapp.android.debug E/DetailFragment$loadUp:     at io.reactivex.internal.operators.single.SingleFlatMap.subscribeActual(SingleFlatMap.java:36)
2019-12-30 13:58:20.459 21419-21419/com.myapp.android.debug E/DetailFragment$loadUp:     at io.reactivex.Single.subscribe(Single.java:3096)
2019-12-30 13:58:20.459 21419-21419/com.myapp.android.debug E/DetailFragment$loadUp:     at io.reactivex.internal.operators.single.SingleMap.subscribeActual(SingleMap.java:34)
2019-12-30 13:58:20.460 21419-21419/com.myapp.android.debug E/DetailFragment$loadUp:     at io.reactivex.Single.subscribe(Single.java:3096)
2019-12-30 13:58:20.460 21419-21419/com.myapp.android.debug E/DetailFragment$loadUp:     at io.reactivex.internal.operators.single.SingleFlatMap.subscribeActual(SingleFlatMap.java:36)
2019-12-30 13:58:20.460 21419-21419/com.myapp.android.debug E/DetailFragment$loadUp:     at io.reactivex.Single.subscribe(Single.java:3096)
2019-12-30 13:58:20.460 21419-21419/com.myapp.android.debug E/DetailFragment$loadUp:     at io.reactivex.internal.operators.single.SingleDoOnSuccess.subscribeActual(SingleDoOnSuccess.java:35)
2019-12-30 13:58:20.460 21419-21419/com.myapp.android.debug E/DetailFragment$loadUp:     at io.reactivex.Single.subscribe(Single.java:3096)
2019-12-30 13:58:20.460 21419-21419/com.myapp.android.debug E/DetailFragment$loadUp:     at io.reactivex.internal.operators.single.SingleToFlowable.subscribeActual(SingleToFlowable.java:37)
2019-12-30 13:58:20.460 21419-21419/com.myapp.android.debug E/DetailFragment$loadUp:     at io.reactivex.Flowable.subscribe(Flowable.java:13234)
2019-12-30 13:58:20.460 21419-21419/com.myapp.android.debug E/DetailFragment$loadUp:     at io.reactivex.Flowable.subscribe(Flowable.java:13180)
2019-12-30 13:58:20.460 21419-21419/com.myapp.android.debug E/DetailFragment$loadUp:     at io.reactivex.internal.operators.flowable.FlowableZip$ZipCoordinator.subscribe(FlowableZip.java:127)
2019-12-30 13:58:20.460 21419-21419/com.myapp.android.debug E/DetailFragment$loadUp:     at io.reactivex.internal.operators.flowable.FlowableZip.subscribeActual(FlowableZip.java:79)
2019-12-30 13:58:20.460 21419-21419/com.myapp.android.debug E/DetailFragment$loadUp:     at io.reactivex.Flowable.subscribe(Flowable.java:13234)
2019-12-30 13:58:20.460 21419-21419/com.myapp.android.debug E/DetailFragment$loadUp:     at io.reactivex.internal.operators.flowable.FlowableMap.subscribeActual(FlowableMap.java:38)
2019-12-30 13:58:20.460 21419-21419/com.myapp.android.debug E/DetailFragment$loadUp:     at io.reactivex.Flowable.subscribe(Flowable.java:13234)
2019-12-30 13:58:20.460 21419-21419/com.myapp.android.debug E/DetailFragment$loadUp:     at io.reactivex.internal.operators.flowable.FlowableOnErrorReturn.subscribeActual(FlowableOnErrorReturn.java:33)
2019-12-30 13:58:20.460 21419-21419/com.myapp.android.debug E/DetailFragment$loadUp:     at io.reactivex.Flowable.subscribe(Flowable.java:13234)
2019-12-30 13:58:20.460 21419-21419/com.myapp.android.debug E/DetailFragment$loadUp:     at io.reactivex.Flowable.subscribe(Flowable.java:13180)
2019-12-30 13:58:20.460 21419-21419/com.myapp.android.debug E/DetailFragment$loadUp:     at io.reactivex.internal.operators.flowable.FlowableSubscribeOn$SubscribeOnSubscriber.run(FlowableSubscribeOn.java:82)
2019-12-30 13:58:20.460 21419-21419/com.myapp.android.debug E/DetailFragment$loadUp:     at io.reactivex.internal.schedulers.ScheduledRunnable.run(ScheduledRunnable.java:66)
2019-12-30 13:58:20.460 21419-21419/com.myapp.android.debug E/DetailFragment$loadUp:     at io.reactivex.internal.schedulers.ScheduledRunnable.call(ScheduledRunnable.java:57)
2019-12-30 13:58:20.460 21419-21419/com.myapp.android.debug E/DetailFragment$loadUp:     at java.util.concurrent.FutureTask.run(FutureTask.java:266)
2019-12-30 13:58:20.460 21419-21419/com.myapp.android.debug E/DetailFragment$loadUp:     at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:301)
2019-12-30 13:58:20.460 21419-21419/com.myapp.android.debug E/DetailFragment$loadUp:     at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1167)
2019-12-30 13:58:20.460 21419-21419/com.myapp.android.debug E/DetailFragment$loadUp:     at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)
2019-12-30 13:58:20.461 21419-21419/com.myapp.android.debug E/DetailFragment$loadUp:     at java.lang.Thread.run(Thread.java:764)

I'd like to standardize the output for both scenarios, that is, I would like to turn this into

data class Parcel(val events: List<Event>)

I know the second kind of response is malformed but I have no control over the backend (its an external service that im consuming), is there a way to fix this?

I tried fiddling with custom adapters but i cant make heads or tails of how to do it :(

EDIT: My best attempt at a Custom Adapter:

class CorreosApiParcelAdapter(private val eventAdapter: JsonAdapter<CorreosApiEvent>,
                              private val errorAdapter: JsonAdapter<Error>) : JsonAdapter<CorreosApiParcel>() {
    override fun fromJson(reader: JsonReader): CorreosApiParcel? = with(reader) {
        val events = mutableListOf<CorreosApiEvent>()
        val parcel = CorreosApiParcel.allNull()
        beginObject()
        while (hasNext()) {
            val nextName = nextName()
            if (nextName != "eventos" && nextName != "error") {
                val value = reader.nextString()
                when (nextName) {
                    "codEnvio" -> parcel.codEnvio = value
                    "refCliente" -> parcel.refCliente = value
                    "codProducto" -> parcel.codProducto = value
                    "fecha_calculada" -> parcel.fechaCalculada = value
                    "largo" -> parcel.largo = value
                    "ancho" -> parcel.ancho = value
                    "alto" -> parcel.alto = value
                    "peso" -> parcel.peso = value
                }
                continue
            }
            if (nextName == "error") {
                val error = errorAdapter.fromJson(reader)
                if (error != null) {
                    parcel.error = error
                }
                continue
            }

            if (peek() == JsonReader.Token.BEGIN_OBJECT) {
                val fromJson = eventAdapter.fromJson(reader)
                if (fromJson != null) {
                    events += fromJson
                }
                continue
            }
            beginArray()
            while (hasNext()) {
                val fromJson = eventAdapter.fromJson(reader)
                if (fromJson != null) {
                    events += fromJson
                }
            }
            endArray()

        }
        endObject()
        return parcel
    }


    override fun toJson(writer: JsonWriter, value: CorreosApiParcel?) {
    }

    companion object {
        val FACTORY: JsonAdapter.Factory = Factory { type, _, moshi ->


            if (Types.getRawType(type) != CorreosApiParcel::class.java)
                return@Factory null

            val eventAdapter = moshi.adapter<CorreosApiEvent>(CorreosApiEvent::class.java)
            val errorAdapter = moshi.adapter<Error>(Error::class.java)
            return@Factory CorreosApiParcelAdapter(eventAdapter, errorAdapter)
        }
    }
}

You can use `Multimap` .. refer https://stackoverflow.com/questions/1062960/map-implementation-with-duplicate-keys — Rajasekaran M, Dec 30 '19 at 14:23

score 2 · Accepted Answer · answered Jan 04 '20 at 11:02

2

Multiple fields are not supported with Moshi, so you'd have to implement a custom JsonAdapter for it. With fromJson() you have access to the JsonReader which you can use to figure out if events is an object or array.

override fun fromJson(reader: JsonReader) = with(reader) {
    val events = mutableList<Event>()
    beginObject()
    while (hasNext()) {
        if (nextName() != "events") { 
            skipValue()
            continue
        }
        if (peek() == BEGIN_OBJECT) {
            events += eventAdapter.fromJson(reader)
            continue
        }
        beginArray()
        while(hasNext()) {
            events += eventAdapter.fromJson(reader)
        }
        endArray()
    }
    endObject()
    Parcel(events)
}

The implementation reads all properties named events as object or array, depending on which is provided. All other properties are skip as they're not relevant for the Parcel. The eventAdapter is a dependency you have to provide. It's responsible for reading the Event objects from json.

answered Jan 04 '20 at 11:02

tynn

38,113
8
108
143

I actually made the Parcel object contain only events for simplicity, but how can I resort to the "default" adapter for the rest of the fields of my object (namely Strings and another object)? – M Rajoy Jan 04 '20 at 16:58
Easiest would be to implement it yourself. Just set the values instead of `skipValue()`. Have a look at the generated adapter for guidance. It's fairly simple. Otherwise you could create an entity without the `events` properties and use `peekJson()` to read the data before the custom implementation. – tynn Jan 04 '20 at 21:11
thanks for your help, been trying to create such custom adapter and I'm struggling to get it working. I edited the original question to reflect my latest attempt, for which I get an error because my JsonAdapter.Factory is never asked whether it can support CorreosApiParcel, only List. Also I had to put every field explicitly which is terrible if I add more fields in the future :( – M Rajoy Jan 05 '20 at 13:44
Really hard to do custom adapters with moshi, I dont see any good documentation on it either – M Rajoy Jan 05 '20 at 13:47
The `CollectionJsonAdapter` should handle the list and request your adapter. It might be possible that another adapter already handles your type and thus your adapter is never asked. Also you might want to add the events to the parcel as well. – tynn Jan 05 '20 at 19:01
I got it working thanks to your help. Ill have to keep investigating on how to rely on moshis standard deserializer for the rest of the fields (if thats even possible), but at least its working! – M Rajoy Jan 06 '20 at 08:11
I got it working thanks to your help. Ill have to keep investigating on how to rely on moshis standard deserializer for the rest of the fields (if thats even possible), but at least its working! – M Rajoy Jan 06 '20 at 08:12

score 0 · Answer 2 · answered Dec 30 '19 at 13:29

0

As far as I'm concerned, it is not natively supported by Moshi and Retrofit, so the answer is no.

Is this something that can't be coordinated with the backend team? It will be a lot complex if you try and manage to handle it in the Android side. The only workaround I can see right now is to have subsequent method call that will validate, and parse the JSON string into a meaningful data type using your own implementation. Eg var events: List<Event>.

There were discussions whether this type of response is even allowed as part of the standardisation for the JSON text format.

You can read more about it here:

Does JSON syntax allow duplicate keys in an object?

answered Dec 30 '19 at 13:29

Joshua de Guzman

2,063
9
24

Its an external service that im consuming so i have absolutely no control over the responses. Sucks but i have to deal with this myself. – M Rajoy Dec 30 '19 at 13:33
Yes. You really have to deal with it on your own. If you find the time, and if authors of moshi found it helpful, you may want to submit a PR or feature request in their repo. For the meantime, you can manually deal with parsing the JSON string. – Joshua de Guzman Dec 31 '19 at 05:05

Moshi with duplicate fields

2 Answers2

Linked