Custom KotlinX Serializer for List class

Question

I want to make a custom List serializer that will parse invalid json arrays safely. Example: list of Int [1, "invalid_int", 2] should be parsed as [1, 2]. I've made a serializer and added it to Json provider, but serialization keeps failing after first element and cannot continue, so I'm getting list of 1 element [1], how can I handle invalid element correctly so decoder will keep parsing other elements?


class SafeListSerializerStack<E>(val elementSerializer: KSerializer<E>) : KSerializer<List<E>> {

    override val descriptor: SerialDescriptor = ListSerializer(elementSerializer).descriptor

    override fun serialize(encoder: Encoder, value: List<E>) {
        val size = value.size
        val composite = encoder.beginCollection(descriptor, size)
        val iterator = value.iterator()
        for (index in 0 until size) {
            composite.encodeSerializableElement(descriptor, index, elementSerializer, iterator.next())
        }
        composite.endStructure(descriptor)
    }

    override fun deserialize(decoder: Decoder): List<E> {
        val arrayList = arrayListOf<E>()
        try {
            val startIndex = arrayList.size
            val messageBuilder = StringBuilder()
            val compositeDecoder = decoder.beginStructure(descriptor)
            while (true) {
                val index = compositeDecoder.decodeElementIndex(descriptor) // fails here on number 2
                if (index == CompositeDecoder.DECODE_DONE) {
                    break
                }
                try {
                    arrayList.add(index, compositeDecoder.decodeSerializableElement(descriptor, startIndex + index, elementSerializer))
                } catch (exception: Exception) {
                    exception.printStackTrace() // falls here when "invalid_int" is parsed, it's ok
                }
            }
            compositeDecoder.endStructure(descriptor)
            if (messageBuilder.isNotBlank()) {
                println(messageBuilder.toString())
            }
        } catch (exception: Exception) {
            exception.printStackTrace() // falls here on number 2
        }
        return arrayList
    }
}

Error happens after invalid element is parsed and exception is thrown at compositeDecoder.decodeElementIndex(descriptor) line with:

kotlinx.serialization.json.internal.JsonDecodingException: Unexpected JSON token at offset 4: Expected end of the array or comma
JSON input: [1, "invalid_int", 2]

I had a feeling that it should "swallow" invalid element and just keep moving, but instead it's stuck and cannot continue parsing, which doesn't make sense to me.

Михаил Нафталь · Accepted Answer · 2021-06-25T09:38:52.950

This could be done without custom serializer. Just parse everything as a String (specify isLenient = true to allow unquoted strings) and then convert to Int all valid integers:

fun main() {
    val input = "[1, \"invalid_int\", 2]"
    val result: List<Int> = Json { isLenient = true }
        .decodeFromString<List<String>>(input)
        .mapNotNull { it.toIntOrNull() }
    println(result) // [1, 2]
}

In a more generic case (when the list is a field and/or its elements are not simple Ints), you'll need a custom serializer:

class SafeListSerializerStack<E>(private val elementSerializer: KSerializer<E>) : KSerializer<List<E>> {
    private val listSerializer = ListSerializer(elementSerializer)
    override val descriptor: SerialDescriptor = listSerializer.descriptor

    override fun serialize(encoder: Encoder, value: List<E>) {
        listSerializer.serialize(encoder, value)
    }

    override fun deserialize(decoder: Decoder): List<E> = with(decoder as JsonDecoder) {
        decodeJsonElement().jsonArray.mapNotNull {
            try {
                json.decodeFromJsonElement(elementSerializer, it)
            } catch (e: SerializationException) {
                e.printStackTrace()
                null
            }
        }
    }
}

Note that this solution works only with deserialization from the Json format and requires kotlinx.serialization 1.2.0+

May work for primitives, but what about data structures? – Wackaloon Jun 25 '21 at 08:04 — Wackaloon, Jun 25 '21 at 08:04

score 1 · Answer 2 · answered Jun 25 '21 at 13:06

Found a way, we can extract json array from decoder given we are using Json to parse it

    override fun deserialize(decoder: Decoder): List<E> {
        val jsonInput = decoder as? JsonDecoder
            ?: error("Can be deserialized only by JSON")
        val rawJson = jsonInput.decodeJsonElement()
        if (rawJson !is JsonArray) {
            return arrayListOf()
        }

        val jsonArray = rawJson.jsonArray
        val jsonParser = jsonInput.json
        val arrayList = ArrayList<E>(jsonArray.size)

        jsonArray.forEach { jsonElement ->
            val result = readElement(jsonParser, jsonElement)
            when {
                result.isSuccess -> arrayList.add(result.getOrThrow())
                result.isFailure -> Log.d("ERROR", "error parsing array")
            }
        }
        arrayList.trimToSize()
        return arrayList
    }

    private fun readElement(json: Json, jsonElement: JsonElement): Result<E> {
        return try {
            Result.success(json.decodeFromJsonElement(elementSerializer, jsonElement))
        } catch (exception: Exception) {
            Result.failure(exception)
        }
    }

Custom KotlinX Serializer for List class

2 Answers2