Converting nested case classes to nested Maps using Shapeless

Question

I am trying to solve [this][1] question using Shapeless, in summary it's about converting a nested case class to Map[String,Any], here is the example:

case class Person(name:String, address:Address)
case class Address(street:String, zip:Int)

val p = Person("Tom", Address("Jefferson st", 10000))

The goal is to convert p to following:

Map("name" -> "Tom", "address" -> Map("street" -> "Jefferson st", "zip" -> 10000))

I am trying to do it using Shapeless LabelledGeneric, here is what I have so far:

import shapeless._
import record._, syntax.singleton._
import ops.record._
import shapeless.ops.record._

def writer[T,A<:HList,H<:HList](t:T)
(implicit lGeneric:LabelledGeneric.Aux[T,A],
 kys:Keys.Aux[A,H],
 vls:Values[A]) = {
    val tGen = lGeneric.to(t)
    val keys = Keys[lGeneric.Repr].apply
    val values = Values[lGeneric.Repr].apply(tGen)
    println(keys)
    println(values)
  }

I am trying to have a recursive writer check each value and try to make Map for each element in value. The above code works fine but when I want to iterate over values with a sample Poly, using the following code I got these errors.

values.map(identity)
//or
tGen.map(identity)

Error:(75, 19) could not find implicit value for parameter mapper: shapeless.ops.hlist.FlatMapper[shapeless.poly.identity.type,vls.Out]
    values.flatMap(identity)
                  ^
Error:(75, 19) not enough arguments for method flatMap: (implicit mapper: shapeless.ops.hlist.FlatMapper[shapeless.poly.identity.type,vls.Out])mapper.Out.
Unspecified value parameter mapper.
    values.flatMap(identity)
                  ^

I don't know why I'm getting that error. I also would be happy to know if there is an easier way to do the whole thing using Shapeless. [1]: Scala macros for nested case classes to Map and other way around

score 26 · Accepted Answer · answered Jul 26 '15 at 14:53

26

Any time you want to perform an operation like flatMap on an HList whose type isn't statically known, you'll need to provide evidence (in the form of an implicit parameter) that the operation is actually available for that type. This is why the compiler is complaining about missing FlatMapper instances—it doesn't know how to flatMap(identity) over an arbitrary HList without them.

A cleaner way to accomplish this kind of thing would be to define a custom type class. Shapeless already provides a ToMap type class for records, and we can take it as a starting point, although it doesn't provide exactly what you're looking for (it doesn't work recursively on nested case classes).

We can write something like the following:

import shapeless._, labelled.FieldType, record._

trait ToMapRec[L <: HList] { def apply(l: L): Map[String, Any] }

Now we need to provide instances for three cases. The first case is the base case—the empty record—and it's handled by hnilToMapRec below.

The second case is the case where we know how to convert the tail of the record, and we know that the head is something that we can also recursively convert (hconsToMapRec0 here).

The final case is similar, but for heads that don't have ToMapRec instances (hconsToMapRec1). Note that we need to use a LowPriority trait to make sure that this instance is prioritized properly with respect to hconsToMapRec0—if we didn't, the two would have the same priority and we'd get errors about ambiguous instances.

trait LowPriorityToMapRec {
  implicit def hconsToMapRec1[K <: Symbol, V, T <: HList](implicit
    wit: Witness.Aux[K],
    tmrT: ToMapRec[T]
  ): ToMapRec[FieldType[K, V] :: T] = new ToMapRec[FieldType[K, V] :: T] {
    def apply(l: FieldType[K, V] :: T): Map[String, Any] =
      tmrT(l.tail) + (wit.value.name -> l.head)
  }
}

object ToMapRec extends LowPriorityToMapRec {
  implicit val hnilToMapRec: ToMapRec[HNil] = new ToMapRec[HNil] {
    def apply(l: HNil): Map[String, Any] = Map.empty
  }

  implicit def hconsToMapRec0[K <: Symbol, V, R <: HList, T <: HList](implicit
    wit: Witness.Aux[K],
    gen: LabelledGeneric.Aux[V, R],
    tmrH: ToMapRec[R],
    tmrT: ToMapRec[T]
  ): ToMapRec[FieldType[K, V] :: T] = new ToMapRec[FieldType[K, V] :: T] {
    def apply(l: FieldType[K, V] :: T): Map[String, Any] =
      tmrT(l.tail) + (wit.value.name -> tmrH(gen.to(l.head)))
  }
}

Lastly we provide some syntax for convenience:

implicit class ToMapRecOps[A](val a: A) extends AnyVal {
  def toMapRec[L <: HList](implicit
    gen: LabelledGeneric.Aux[A, L],
    tmr: ToMapRec[L]
  ): Map[String, Any] = tmr(gen.to(a))
}

And then we can demonstrate that it works:

scala> p.toMapRec
res0: Map[String,Any] = Map(address -> Map(zip -> 10000, street -> Jefferson st), name -> Tom)

Note that this won't work for types where the nested case classes are in a list, tuple, etc., but you could extend it to those cases pretty straightforwardly.

answered Jul 26 '15 at 14:53

Travis Brown

138,631
12
375
680

Thanks for awesome magics! One question, how would be the other way, from Map[String,Any] to case class, is there a good way to do it because we have lost all the types? – Omid Jul 26 '15 at 17:54
@Omid It's possible if you know the target type and are willing to receive the result in an `Option` (or some other type that can represent failure). – Travis Brown Jul 26 '15 at 18:00
yes, I know the target type and a failure mechanism would be great. – Omid Jul 26 '15 at 18:02
1

@Omid Great! It's probably worth a new question, though. :) – Travis Brown Jul 26 '15 at 18:10
Thanks—I'll give it a shot as soon as I can if nobody beats me to it. – Travis Brown Jul 26 '15 at 18:42
So for all container like types for case class like Array, Map and ... should I provide something like HNill with a something like `A[T],T<:HList ...` ? – Omid Jul 26 '15 at 21:13
When I use it like what you did: `p.toMapRec` it works perfectly but when I want to wrap it another method like: `def convert[A](c: A)= { val params = cc.toMapRec }` I get:`Error:(36, 21) could not find implicit value for parameter gen: shapeless.LabelledGeneric.Aux[A,L] val params = cc.toMapRec ^` – Omid Jul 26 '15 at 22:39
@Omid You'll have to include an implicit `ToMapRec[A]` parameter on your generic method (for the reason given in my first paragraph above). – Travis Brown Jul 26 '15 at 22:41
1

@Omid, oh no, sorry—you'll need a `ToMapRec` for the generic representation of `A`, not `A` itself. Probably worth a new question to avoid drawing out the comments further. – Travis Brown Jul 26 '15 at 22:54
thanks, here is the questions: http://stackoverflow.com/questions/31642832/compiler-cannot-find-right-implicits-for-shapeless-labelledgeneric – Omid Jul 26 '15 at 23:02
I am trying to add Seq,set,Map,Option,Tuples, can you give me a hint on how should I use priority traits for those ? – Omid Jul 30 '15 at 22:24
1

@Omid Instead of returning `ToMapRec[FieldType[K, V] :: T]` you'd have a method returning `ToMapRec[FieldType[K, List[V]] :: T]`, etc. Probably worth a new question. – Travis Brown Jul 30 '15 at 23:42
Here is the question: http://stackoverflow.com/questions/31736369/adding-container-types-like-seq-list-option-and-map-to-shapeless-generic-conv – Omid Aug 02 '15 at 23:23
It's not fully recursive. If you use it with A, defined as ```case class A (b:B); case class B(c:C); case class C(n:String)``` it compiles but don't fully expends the structure. – jwinandy Aug 05 '15 at 13:40

score 3 · Answer 2 · edited May 23 '17 at 11:46

3

I have a problem with an approach provided by Travis Brown.
Some of nesting case classes are not converted to Map https://scalafiddle.io/sf/cia2jTa/0.

The answer was found here.
To correct the solution just wrap ToMapRec[T] in implicit parameters to Lazy[ToMapRec[T]]. Corrected fiddle https://scalafiddle.io/sf/cia2jTa/1

edited May 23 '17 at 11:46

Community

1
1

answered Oct 04 '16 at 13:06

Vladimir Korablev

53
7

Converting nested case classes to nested Maps using Shapeless

2 Answers2

Linked