Debugging json4s read deserialization errors

Question

I am attempting to consume an API that I do not have control over which is somewhat poorly documentented and somewhat inconsistent. This means that sometimes, the API returns a different type than what is documented or what you would normally see. For this example, we'll look at a case when an array was returned in a place where I would normally see a string. That makes a crappy API, but my real problem is: How can I more easily track those things down? Right now, the errors look something like this:

No usable value for identifier
Do not know how to convert JArray(List(JString(3c8723eceb1a), JString(cba8849e7a2f))) into class java.lang.String

After deciphering the problem (why JValue::toString doesn't emit a JSON string is utterly perplexing to me), I can figure out the API returned an array when I made my case class only able to deal with Strings. Great. My issue is that finding this discrepancy between my object model and the contents of the JSON seems significantly more difficult than it should be.

Currently, this is my workflow for hunting down decoding errors:

1. Hope bad data has some sort of identifying marker. If this is not true, then it is way more guesswork and you will have to repeat the following steps for each entry that looks like the bad bits.
2. Go through the troubles of converting the JArray(List(JString(...), ...)) from the error message into valid JSON, hoping that I encode JSON the same way at the API endpoint I got the data from does. If this is not true, then I use a JSON formatter (jq) to format all data consistently.
3. Locate the place in the source data where the decoding error originates from.
4. Backtrack through arrays and objects to discover how I need to change my object model to more accurately represent what data is coming back to me from the API.

Some background: I'm coming from C++, where I rolled my own JSON deserialization framework for this purpose. The equivalent error when using the library I built is:

Error decoding value at result.taskInstances[914].subtasks[5].identifier: expected std::string but found array value (["3c8723eceb1a","cba8849e7a2f"]) at 1:4084564

This is my process when using my hand-rolled library:

1. Look at the expected type (std::string) compared with the data that was actually found (["3c8723eceb1a","cba8849e7a2f"]) and alter my data model for the path for the data in the source (result.taskInstances[914].subtasks[5].identifier)

As you can see, I get to jump immediately to the problem that I actually have.

My question is: Is there a way to more quickly debug inconsistencies between my data model and the results I'm getting back from the API?

I'm using json4s-native_2.10 version 3.2.8.

---

A simplified example:

{ "property": ["3c8723eceb1a", "cba8849e7a2f"] }

Does not mesh with Scala class:

case class Thing(property: String)

Answer 1

The best solution would be to use Try http://www.scala-lang.org/api/current/#scala.util.Try in Scala, but unfortunately json4s API cannot.

So, I think you should use Scala Option type http://www.scala-lang.org/api/current/#scala.Option .

In Scala, and more generally in functional languages, Options are used to represent an object that can be there or not (like à nil value).

For handle parsing failures, you can use parse(str).toOption, which is a function that return an Option[JValue], and you can doing a pattern matching on the resulting value.

For handling extraction of data extraction into case classes, you can use extractOpt function, to do pattern matching on the value.

You can read this answer : https://stackoverflow.com/a/15944506/2330361