How to use shapeless to convert generic Map[String, Any] to case class inside generic function?

Question

I'm trying to follow the answer here https://stackoverflow.com/a/31641779/1586965

That is, I want to be able to convert (potentially nested) Map[String, Any] to a case class.

scalaVersion := "2.11.8"
val shapelessV = "2.3.3"

If I try to wrap the code in the above answer in another generic function, I can't seem to get it to compile

import shapeless._, labelled._
import FromMap._

def usesGenerics[P](map: Map[String, Any]): P = {
  to[P].from(mp).get
}

I get the following compile error

could not find implicit value for parameter gen: shapeless.LabelledGeneric.Aux[P,R]

UDPATE

This approach is much easier & complete (handles more edge cases): https://stackoverflow.com/a/55355685/1586965

FULL ANSWER

This incorporates both Travis's initial idea and Dmytros fix, and finally some simplification

import shapeless._, labelled.{FieldType, field}

trait FromMap[L <: HList] {
  def apply(m: Map[String, Any]): Option[L]
}

trait LowPriorityFromMap {
  implicit def hconsFromMap1[K <: Symbol, V, T <: HList](implicit
                                                         witness: Witness.Aux[K],
                                                         typeable: Typeable[V],
                                                         fromMapT: Lazy[FromMap[T]]
                                                        ): FromMap[FieldType[K, V] :: T] = new FromMap[FieldType[K, V] :: T] {
    def apply(m: Map[String, Any]): Option[FieldType[K, V] :: T] = for {
      v <- m.get(witness.value.name)
      h <- typeable.cast(v)
      t <- fromMapT.value(m)
    } yield field[K](h) :: t
  }
}

object FromMap extends LowPriorityFromMap {
  implicit val hnilFromMap: FromMap[HNil] = new FromMap[HNil] {
    def apply(m: Map[String, Any]): Option[HNil] = Some(HNil)
  }

  implicit def hconsFromMap0[K <: Symbol, V, R <: HList, T <: HList](implicit
                                                                     witness: Witness.Aux[K],
                                                                     gen: LabelledGeneric.Aux[V, R],
                                                                     fromMapH: FromMap[R],
                                                                     fromMapT: FromMap[T]
                                                                    ): FromMap[FieldType[K, V] :: T] =
    new FromMap[FieldType[K, V] :: T] {
      def apply(m: Map[String, Any]): Option[FieldType[K, V] :: T] = for {
        v <- m.get(witness.value.name)
        r <- Typeable[Map[String, Any]].cast(v)
        h <- fromMapH(r)
        t <- fromMapT(m)
      } yield field[K](gen.from(h)) :: t
    }
}

trait CaseClassFromMap[P <: Product] {
  def apply(m: Map[String, Any]): Option[P]
}

object CaseClassFromMap {
  implicit def mk[P <: Product, R <: HList](implicit gen: LabelledGeneric.Aux[P, R],
                                            fromMap: FromMap[R]): CaseClassFromMap[P] = new CaseClassFromMap[P] {
    def apply(m: Map[String, Any]): Option[P] = fromMap(m).map(gen.from)
  }

  def apply[P <: Product](map: Map[String, Any])(implicit fromMap: CaseClassFromMap[P]): P = fromMap(map).get
}

I have no idea whether this is the issue, but can you update to 2.11.12? 2.11.8 is three years old—that's like ancient history. — Travis Brown, Mar 04 '19 at 17:15
(As far as I know any version of Spark that works on 2.11.8 should work on 2.11.12.) — Travis Brown, Mar 04 '19 at 17:16
@TravisBrown Spark also has a 2.12 port out, perhaps trying that out 2.12 as well might be a good idea. — Yuval Itzchakov, Mar 04 '19 at 17:21
You must have something weird going on, because this works just fine for me with your versions, and `Symbol#name` returning `NameType` doesn't make sense. You'll need to provide more context about your imports, etc. — Travis Brown, Mar 04 '19 at 17:22
Thanks @TravisBrown yup, my imports messing things up, I had `import scala.reflect.runtime.universe._` — samthebest, Mar 04 '19 at 17:27
@TravisBrown I want to call `to` inside another generic function, but it can't find the implicit it needs for `from` - particularly because I don't know what `R` is. I updated the question with the code I've tried, please could you take a look? — samthebest, Mar 04 '19 at 18:26
@samthebest Could you include a usage example for this? I'm trying to use your code like this: `CaseClassFromMap[P](params)`, where `params` is `Map[String, Any]`, but I'm getting an error: `Could not find implicit value for parameter fromMap: ...CaseClassFromMap[P]`. Am I missing an import somewhere? — Alan Thomas, Jan 03 '20 at 01:06
@AlanThomas Try this instead, it's much easier to follow: https://stackoverflow.com/a/55355685/1586965 — samthebest, Jan 04 '20 at 09:09
@samthebest hmph, that's a neat solution! I'll try it out. My only concern would be whether the shapes are provable at compile time. Thanks! — Alan Thomas, Jan 04 '20 at 21:11

score 2 · Accepted Answer · edited Mar 05 '19 at 09:44

2

Does this work for you?

def usesGenerics[P, R <: HList](map: Map[String, Any])(implicit gen: LabelledGeneric.Aux[P, R], fromMap: FromMap[R]): P = {
  to[P].from[R](map).get
}

You need one more type class

  trait CaseClassFromMap[P <: Product] {
    def apply(m: Map[String, Any]): Option[P]
  }
  object CaseClassFromMap {
    implicit def mk[P <: Product, R <: HList](implicit
      gen: LabelledGeneric.Aux[P, R],
      fromMap: FromMap[R]
      ): CaseClassFromMap[P] = new CaseClassFromMap[P] {
      def apply(m: Map[String, Any]): Option[P] = to[P].from[R](m)
    }
  }

  def usesGenerics[P <: Product](map: Map[String, Any])(implicit fromMap: CaseClassFromMap[P]): P = {
    fromMap(map).get
  }

  usesGenerics[Person](mp)

edited Mar 05 '19 at 09:44

samthebest

30,803
25
102
142

answered Mar 04 '19 at 18:43

Dmytro Mitin

48,194
3
28
66

How do I call that since I now need two type params. The first I know to be `Person`, but what to put for the second `val converted2 = usesGenerics[Person, ???](mp)` – samthebest Mar 05 '19 at 09:09
OK, I see. Then you need one more type class. I updated my answer. – Dmytro Mitin Mar 05 '19 at 09:21

How to use shapeless to convert generic Map[String, Any] to case class inside generic function?

UDPATE

FULL ANSWER

1 Answers1

Linked