Shapeless code to convert Map[String, Any] to case class cannot handle optional substructures

Question

I'm trying to use this https://stackoverflow.com/a/31641779/1586965 (How to use shapeless to convert generic Map[String, Any] to case class inside generic function?) to process

case class Address(street: String, zip: Int)
case class PersonOptionalAddress(name: String, address: Option[Address])

I have a test that fails:

"Convert Map to PersonOptionalAddress Some" in {
  CaseClassFromMap[PersonOptionalAddress](Map(
    "name" -> "Tom",
    "address" -> Some(Map("street" -> "Jefferson st", "zip" -> 10000))
  )) must_=== PersonOptionalAddress("Tom", Some(Address("Jefferson st", 10000)))
}

with

java.util.NoSuchElementException: None.get

If the substructure is not nested, or it is None, then the tests work fine.

I've also tried this, but it doesn't work either

"Convert Map to PersonOptionalAddress Some" in {
  CaseClassFromMap[PersonOptionalAddress](Map(
    "name" -> "Tom",
    "address" -> Map("x" -> Map("street" -> "Jefferson st", "zip" -> 10000))
  )) must_=== PersonOptionalAddress("Tom", Some(Address("Jefferson st", 10000)))
}

Dmytro Mitin · Answer 1 · 2020-11-09T12:36:59.413

If you want the code to work with PersonOptionalAddress you should add one more instance of the type class so that it will work also with Map( "name" -> "Tom", "address" -> Some(Map ...) )

implicit def hconsFromMap0opt[K <: Symbol, V, R <: HList, T <: HList](implicit
                                                                          witness: Witness.Aux[K],
                                                                          gen: LabelledGeneric.Aux[V, R],
                                                                          fromMapH: FromMap[R],
                                                                          fromMapT: FromMap[T]
                                                                         ): FromMap[FieldType[K, Option[V]] :: T] =
      new FromMap[FieldType[K, Option[V]] :: T] {
        def apply(m: Map[String, Any]): Option[FieldType[K, Option[V]] :: T] = (for {
          v <- m.get(witness.value.name)
          r <- Typeable[Map[String, Any]].cast(v)
          h <- fromMapH(r)
          t <- fromMapT(m)
        } yield field[K](Some(gen.from(h))) :: t).orElse(for {
          v <- m.get(witness.value.name)
          r1 <- Typeable[Option[Map[String, Any]]].cast(v)
          opt = for {
            r <- r1
            h <- fromMapH(r)
          } yield gen.from(h)
          t <- fromMapT(m)
        } yield field[K](opt) :: t)
      }

The whole code

import shapeless._
import labelled.{FieldType, field}

object App {

  trait FromMap[L <: HList] {
    def apply(m: Map[String, Any]): Option[L]
  }

  trait LowPriorityFromMap {
    implicit def hconsFromMap1[K <: Symbol, V, T <: HList](implicit
                                                           witness: Witness.Aux[K],
                                                           typeable: Typeable[V],
                                                           fromMapT: Lazy[FromMap[T]]
                                                          ): FromMap[FieldType[K, V] :: T] = new FromMap[FieldType[K, V] :: T] {
      def apply(m: Map[String, Any]): Option[FieldType[K, V] :: T] = for {
        v <- m.get(witness.value.name)
        h <- typeable.cast(v)
        t <- fromMapT.value(m)
      } yield field[K](h) :: t
    }
  }

  object FromMap extends LowPriorityFromMap {
    implicit val hnilFromMap: FromMap[HNil] = new FromMap[HNil] {
      def apply(m: Map[String, Any]): Option[HNil] = Some(HNil)
    }

    implicit def hconsFromMap0[K <: Symbol, V, R <: HList, T <: HList](implicit
                                                                       witness: Witness.Aux[K],
                                                                       gen: LabelledGeneric.Aux[V, R],
                                                                       fromMapH: FromMap[R],
                                                                       fromMapT: FromMap[T]
                                                                      ): FromMap[FieldType[K, V] :: T] =
      new FromMap[FieldType[K, V] :: T] {
        def apply(m: Map[String, Any]): Option[FieldType[K, V] :: T] = for {
          v <- m.get(witness.value.name)
          r <- Typeable[Map[String, Any]].cast(v)
          h <- fromMapH(r)
          t <- fromMapT(m)
        } yield field[K](gen.from(h)) :: t
      }

    implicit def hconsFromMap0opt[K <: Symbol, V, R <: HList, T <: HList](implicit
                                                                          witness: Witness.Aux[K],
                                                                          gen: LabelledGeneric.Aux[V, R],
                                                                          fromMapH: FromMap[R],
                                                                          fromMapT: FromMap[T]
                                                                         ): FromMap[FieldType[K, Option[V]] :: T] =
      new FromMap[FieldType[K, Option[V]] :: T] {
        def apply(m: Map[String, Any]): Option[FieldType[K, Option[V]] :: T] = (for {
          v <- m.get(witness.value.name)
          r <- Typeable[Map[String, Any]].cast(v)
          h <- fromMapH(r)
          t <- fromMapT(m)
        } yield field[K](Some(gen.from(h))) :: t).orElse(for {
          v <- m.get(witness.value.name)
          r1 <- Typeable[Option[Map[String, Any]]].cast(v)
          opt = for {
            r <- r1
            h <- fromMapH(r)
          } yield gen.from(h)
          t <- fromMapT(m)
        } yield field[K](opt) :: t)
      }
  }

  trait CaseClassFromMap[P <: Product] {
    def apply(m: Map[String, Any]): Option[P]
  }

  object CaseClassFromMap {
    implicit def mk[P <: Product, R <: HList](implicit gen: LabelledGeneric.Aux[P, R],
                                              fromMap: FromMap[R]): CaseClassFromMap[P] = new CaseClassFromMap[P] {
      def apply(m: Map[String, Any]): Option[P] = fromMap(m).map(gen.from)
    }

    def apply[P <: Product](map: Map[String, Any])(implicit fromMap: CaseClassFromMap[P]): P = fromMap(map).get
  }

  case class Address(street: String, zip: Int)
  case class PersonOptionalAddress(name: String, address: Option[Address])
  case class PersonAddress(name: String, address: Address)

  def main(args: Array[String]): Unit = {
    println(
      CaseClassFromMap[PersonAddress](Map(
        "name" -> "Tom",
        "address" -> Map("street" -> "Jefferson st", "zip" -> 10000)
      ))
    )//PersonAddress(Tom,Address(Jefferson st,10000))

    println(
      CaseClassFromMap[PersonOptionalAddress](Map(
        "name" -> "Tom",
        "address" -> Map("street" -> "Jefferson st", "zip" -> 10000)
      ))
    )//PersonOptionalAddress(Tom,Some(Address(Jefferson st,10000)))

    println(
      CaseClassFromMap[PersonOptionalAddress](Map(
        "name" -> "Tom",
        "address" -> Some(Map("street" -> "Jefferson st", "zip" -> 10000))
      ))
    )//PersonOptionalAddress(Tom,Some(Address(Jefferson st,10000)))

    println(
      CaseClassFromMap[PersonOptionalAddress](Map(
        "name" -> "Tom",
        "address" -> None
      ))
    )//PersonOptionalAddress(Tom,None)

  }
}

hey thanks @Dmytro Mitin! I think we are nearly there, but it doesn't seem to work with `None`, i.e. `Map("name" -> "Tom", "address" -> None)` — samthebest, Mar 08 '19 at 15:46
Thanks again @Dmytro Mitin, think I found another case where it doesn't seem to work. When we have a struct inside a list inside an option, e.g. `case class Foo(foo: String); case class NestedFoo(foos: Option[List[Foo]])` then `Map[String, Any]("foos" -> Some(List(Map("foo" -> "value"))))` doesn't work — samthebest, Mar 11 '19 at 16:58
Surery it doesn't work for list. Why should it? You should define an instance for list if you want it to work for list. — Dmytro Mitin, Mar 11 '19 at 21:37
By the way, you incorrectly edited my answer. Now it states that " to work with `PersonOptionalAddress` you should add one more instance ... so that it will work also with `Map( ... "address" -> Some(Map ...)`". Actually making work with `PersonOptionalAddress` vs `PersonAddress` and with `Map( ... "address" -> Some(Map ...)` vs `Map( ... "address" -> Map ...` are different. — Dmytro Mitin, Mar 11 '19 at 22:01
This worked for me, required adding a `}` right before `trait CaseClassFromMap[P <: Product] {` however (not enough characters to qualify for SO's 6 character minimum for editing haha) — RyanQuey, Nov 08 '20 at 04:40
I'm having trouble getting this to work when one of the field's types is an `java.time.Instant` type. I just get `java.util.NoSuchElementException: None.get`. Any guesses why that would be the case? — RyanQuey, Nov 09 '20 at 06:33
@RyanQuey It's hard to say without details. I added a field to the class: `case class PersonAddress(name: String, address: Address, time: java.time.Instant)`. And `CaseClassFromMap[PersonAddress](Map("name" -> "Tom", "address" -> Map("street" -> "Jefferson st", "zip" -> 10000), "time" -> java.time.Instant.now() ))` seems to work: `PersonAddress(Tom,Address(Jefferson st,10000),2020-11-09T12:38:50.372Z)`. You should start a new question. — Dmytro Mitin, Nov 09 '20 at 12:41

Shapeless code to convert Map[String, Any] to case class cannot handle optional substructures

1 Answers1

Linked