Retrieving null values from json using circe-optics

Question

The json I have looks like this:

{
    "cards": [
        {
            "card_id":"1234567890",
            "card_status":"active",
            "card_expiration":{
                "formatted":"01/20"
            },
            "debit":{
                "masked_debit_card_number":"1111 **** **** 1111",
            }
        },
        {
            "card_id":"1234567891",
            "card_status":"active",
            "card_expiration":null,
            "debit":{
                "masked_debit_card_number":"2222 **** **** 2222",
            }
        }
    ]
}

I'm trying to retrieve all card_expiration fields values using this function:

def getExpirations(json: Json) =
    root
        .cards
        .each
        .filter(root.card_status.string.exist(_ == "active"))
        .card_expiration
        .selectDynamic("formatted")
        .string
        .getAll(json)

The thing is, the above expression only returns 1 result - for the first card, but I really need to get something like List(Some("01/20"), None)! What can I do in this situation?

So, you want `nulls` but you are using a functional json library... interesting choice. Or are you taking about `JsonNull` ? — sarveshseri, Oct 23 '19 at 10:25
What you want to do is to just remove `.selectDynamic` and `.string`. You can just get a `List[Json]`... then you map those to extract the field from `JObjects` and just get None for `JNulls`. — sarveshseri, Oct 23 '19 at 10:41
Any idea on how to do that in code? :) Using `.getAll(json)` right after `card_expiration` doesn't compile as it expects an int (index?). And operating directly on what the `card_expiration` field has returned is very hard as it's something internal to the library circe-optics is using - the monocle. — Marek M., Oct 23 '19 at 10:53
Ok, I think I managed to do that, but it involves a bit more manual work than I would like. Isn't there an easier way? — Marek M., Oct 23 '19 at 11:05

score 3 · Accepted Answer · answered Oct 23 '19 at 12:22

The issue is that by the time you've done the formatted step you're no longer matching the null expiration. You could do something like this:

import io.circe.Json, io.circe.optics.JsonPath.root

def getExpirations(json: Json) =
  root
    .cards
    .each
    .filter(root.card_status.string.exist(_ == "active"))
    .card_expiration
    .as[Option[Map[String, String]]]
    .getAll(json)

Or:

import io.circe.Json, io.circe.generic.auto._, io.circe.optics.JsonPath.root

case class Expiration(formatted: String)

def getExpirations(json: Json) =
  root
    .cards
    .each
    .filter(root.card_status.string.exist(_ == "active"))
    .card_expiration
    .as[Option[Expiration]]
    .getAll(json)

And then:

scala> getExpirations(io.circe.jawn.parse(doc).right.get)
res0: List[Option[Expiration]] = List(Some(Expiration(01/20)), None)

Without more context, it's not clear in my view that this is a good use case for circe-optics. You're probably better off decoding into case classes, or maybe using cursors. If you can provide more information it'd be easier to tell.

I have a ton of APIs returning all sorts of data and the business wants me to develop a program able to consume all of them with the use of configuration files - so very little to none coding should be included. The reason is that if a new API is developed I shouldn't write any API specific code, but instead use some kind of a DSL to extract what data they need. My first idea was to use JsonPath as described on Baeldung, but then I thought of circe-optics, as it's more native to scala. Anyway, I did something similar to the above and improved the code following your post. Thanks :) — Marek M., Oct 23 '19 at 14:33
Btw. those configuration files will be a set of classes encapsulating expression logic similar to the above (this was just a boiled down form of one, simple usecase) - one per api used. No one said that the config files can't be scala code ;) — Marek M., Oct 23 '19 at 14:34

Retrieving null values from json using circe-optics

1 Answers1