3

I'm using Shapeless to accumulate materialized values in Akka as an HList and convert that to a case class.

(You don't have to know Akka much for this question, but the default approach accumulates materialized values as recursively nested 2-tuples, which isn't much fun, so Shapeless HLists seemed a more sensible approach -- and works pretty well. But I don't know how to properly re-use that approach. Here, I'll simplify the kinds of values Akka produces.)

For example, let's say we've got two materialized types, "A" and "B":

case class Result(b: B, a: A)

createA
  .mapMaterialized((a: A) => a :: HNil)
  .viaMat(flowCreatingB)((list1, b: B) => b :: list1)
  .mapMaterialized(list2 => Generic[Result].from(list2))
   
// list1 = A :: HNil
// list2 = B :: A :: HNil

... and that produces Result just fine. But it requires that your case class be written backwards -- first value last, etc -- which is kind of dorky and hard to follow.

So the sensible thing is to reverse the list before converting to the case class, like this:

case class Result(a: A, b: B)
// ...
  .mapMaterialized(list2 => Generic[Result].from(list2.reverse))

Now we can think about Result properties in the same order they were built. Yay.

But how to simplify and reuse this line of code?

The problem is that implicits don't work on multiple type parameters. For example:

def toCaseClass[A, R <: HList](implicit g: Generic.Aux[A, R], r: Reverse.Aux[L, R]): R => A =
  l => g.from(l.reverse) 

I'd need to specify both A (Result, above) and the HList being built:

  .mapMaterialized(toCaseClass[Result, B :: A :: HNil])

Obviously, that invocation is going to be absurd with long lists (and Akka tends to build up really ugly-looking materialized types, not merely "A" and "B"). It'd be nicer to write something like:

  .mapMaterialized(toCaseClass[Result])

I've tried to solve this using implicits, like this:

  implicit class GraphOps[Mat <: HList](g: RunnableGraph[Mat]) {

    implicit def createConverter[A, RL <: HList](implicit
        r: Reverse.Aux[Mat, RL],
        gen: Generic.Aux[A, RL]): Lazy[Mat => A] =
      Lazy { l =>
        val x: RL = l.reverse
        val y: A = gen.from(x)
        gen.from(l.reverse)
      }

    def toCaseClass[A](implicit convert: Lazy[Mat => A]): RunnableGraph[A] = {
      g.mapMaterializedValue(convert.value)
    }

But the compiler complains "No implicit view available".

The deeper problem is that I don't quite understand how to properly infer...

// R = Reversed order (e.g. B :: A :: NHNil)
// T = Type to create (e.g. Result(a, b))
// H = HList of T (e.g. A :: B :: HNil)
gen: Generic.Aux[T, H] // Generic[T] { type Repr = H }
rev: Reverse.Aux[R, H] // Reverse[R] { type Out = H }

This is sort of backwards from how Shapeless likes to infer things; I can't quite chain the abstract type members properly.

Profound thanks if you have insight here.


My bad: the example above, of course, requires Akka to compile. A simpler way of putting it is this (with thanks to Dymtro):

  import shapeless._
  import shapeless.ops.hlist.Reverse

  case class Result(one: String, two: Int)

  val results = 2 :: "one" :: HNil
  println(Generic[Result].from(results.reverse)) 
  // this works: prints "Result(one,2)"

  case class Converter[A, B](value: A => B)

  implicit class Ops[L <: HList](list: L) {

    implicit def createConverter[A, RL <: HList](implicit
        r: Reverse.Aux[L, RL],
        gen: Generic.Aux[A, RL]): Converter[L, A] =
      Converter(l => gen.from(l.reverse))

    def toClass[A](implicit converter: Converter[L, A]): A =
      converter.value(list)
  }

  println(results.toClass[Result]) 
  // error: could not find implicit value for parameter converter:
  // Converter[Int :: String :: shapeless.HNil,Result]

Dymtro's final example, below...

implicit class GraphOps[Mat <: HList, R <: HList](g: RunnableGraph[Mat]) {
  def toCaseClass[A](implicit
    r: Reverse.Aux[Mat, R],
    gen: Generic.Aux[A, R]
  ): RunnableGraph[A] = g.mapMaterializedValue(l => gen.from(l.reverse)) 
}

... does seem to do what I'd been hoping for. Thank you very much Dmytro!

(Note: I had been somewhat misled in analyzing it earlier: it seems IntelliJ's presentation compiler incorrectly insists it won't compile (missing implicits). Moral: Don't trust IJ's presentation compiler.)

Tim
  • 1,615
  • 14
  • 17

1 Answers1

2

If I understood correctly you wish that in

def toCaseClass[A, R <: HList, L <: HList](implicit 
  g: Generic.Aux[A, R], 
  r: Reverse.Aux[L, R]
): L => A = l => g.from(l.reverse)

you could specify only A and then R, L be inferred.

You can do this with PartiallyApplied pattern

import shapeless.ops.hlist.Reverse
import shapeless.{Generic, HList, HNil}

def toCaseClass[A] = new {
  def apply[R <: HList, L <: HList]()(implicit 
    g: Generic.Aux[A, R], 
    r0: Reverse.Aux[R, L], 
    r: Reverse.Aux[L, R]
  ): L => A = l => g.from(l.reverse)
}

class A
class B
val a = new A
val b = new B
case class Result(a: A, b: B)

toCaseClass[Result]().apply(b :: a :: HNil)

(without implicit r0 type parameter L can't be inferred upon call of .apply() because L becomes known only upon call .apply().apply(...))

or better

def toCaseClass[A] = new {
  def apply[R <: HList, L <: HList](l: L)(implicit 
    g: Generic.Aux[A, R], 
    r: Reverse.Aux[L, R]
  ): A = g.from(l.reverse)
}

toCaseClass[Result](b :: a :: HNil)

(here we don't need r0 because L becomes known already upon call .apply(...)).

If you want you can replace anonymous class with named one

def toCaseClass[A] = new PartiallyApplied[A]

class PartiallyApplied[A] {
  def apply...
}

Alternatively you can define a type class (although this is a little more wordy)

trait ToCaseClass[A] {
  type L
  def toCaseClass(l: L): A
}
object ToCaseClass {
  type Aux[A, L0] = ToCaseClass[A] { type L = L0 }
  def instance[A, L0](f: L0 => A): Aux[A, L0] = new ToCaseClass[A] {
    type L = L0
    override def toCaseClass(l: L0): A = f(l)
  }
  implicit def mkToCaseClass[A, R <: HList, L <: HList](implicit
    g: Generic.Aux[A, R],
    r0: Reverse.Aux[R, L],
    r: Reverse.Aux[L, R]
  ): Aux[A, L] = instance(l => g.from(l.reverse))
}

def toCaseClass[A](implicit tcc: ToCaseClass[A]): tcc.L => A = tcc.toCaseClass

toCaseClass[Result].apply(b :: a :: HNil)

Hiding several implicits with a type class: How to wrap a method having implicits with another method in Scala?

You could find an answer to your question in Type Astronaut:

https://books.underscore.io/shapeless-guide/shapeless-guide.html#sec:ops:migration (6.3 Case study: case class migrations)

Notice that IceCreamV1("Sundae", 1, true).migrateTo[IceCreamV2a] takes a single type parameter.

Your code with GraphOps doesn't work for several reasons.

Firstly, shapeless.Lazy is not just a wrapper. It's a macro-based type class to handle "diverging implicit expansion" (in Scala 2.13 there are by-name => implicits for that, although they are not equivalent to Lazy). You should use Lazy when you understand why you need it.

Secondly, you seem to define some implicit conversion (implicit view, Mat => A) but resolution of implicit conversions is trickier than resolution of other implicits (1 2 3 4 5).

Thirdly, you seem to assume that when you define

implicit def foo: Foo = ???

def useImplicitFoo(implicit foo1: Foo) = ???

foo1 is foo. But generally this is not true. foo is defined in current scope and foo1 will be resolved in the scope of useImplicitFoo call site:

Setting abstract type based on typeclass

When doing implicit resolution with type parameters, why does val placement matter? (difference between implicit x: X and implicitly[X])

So implicit createConverter is just not in scope when you call toCaseClass.

Fixed version of your code is

trait RunnableGraph[Mat]{
  def mapMaterializedValue[A](a: Mat => A): RunnableGraph[A]
}

case class Wrapper[A, B](value: A => B)

implicit class GraphOps[Mat <: HList](g: RunnableGraph[Mat]) {
  val ops = this

  implicit def createConverter[A, RL <: HList](implicit
    r: Reverse.Aux[Mat, RL],
    gen: Generic.Aux[A, RL],
  ): Wrapper[Mat, A] =
    Wrapper { l =>
      val x: RL = l.reverse
      val y: A = gen.from(x)
      gen.from(l.reverse)
    }

  def toCaseClass[A](implicit convert: Wrapper[Mat, A]): RunnableGraph[A] = {
    g.mapMaterializedValue(convert.value)
  }
}

val g: RunnableGraph[B :: A :: HNil] = ???
val ops = g.ops
import ops._
g.toCaseClass[Result]

Try

import akka.stream.scaladsl.RunnableGraph
import shapeless.{::, Generic, HList, HNil}
import shapeless.ops.hlist.Reverse

implicit class GraphOps[Mat <: HList, R <: HList](g: RunnableGraph[Mat]) {
  def toCaseClass[A](implicit
    r: Reverse.Aux[Mat, R],
    gen: Generic.Aux[A, R]
  ): RunnableGraph[A] = g.mapMaterializedValue(l => gen.from(l.reverse)) 
}

case class Result(one: String, two: Int)

val g: RunnableGraph[Int :: String :: HNil] = ???
g.toCaseClass[Result]
Dmytro Mitin
  • 48,194
  • 3
  • 28
  • 66
  • Thanks so much Dmtryo! Sadly, that doesn't quite do it - at least not when the real RunnableGraph" from Akka is used; I see "could not find implicit value for parameter convert". I think I could have done a bit better job at simplifying the example to remove that dependency; I'm going to edit my example above to do so. Thank you SO MUCH for your help, though, regardless. – Tim Oct 19 '20 at 22:44
  • I'm going to try messing around with your "PartiallyApplied" solution now; that looks promising. – Tim Oct 19 '20 at 22:53
  • @Tim My fixed version of `GraphOps` code seems to work with actual `akka.stream.scaladsl.RunnableGraph` as well https://scastie.scala-lang.org/DmytroMitin/qTsbP0VeQVSh8VHDNSFjwg/5 (you can see `NotImplementedError`, this is a runtime exception, so the code compiles). If you add proper import then your new version of code with my fix seems to work also https://scastie.scala-lang.org/DmytroMitin/P3pNKYYVTeecilJM7bQMZw – Dmytro Mitin Oct 19 '20 at 23:03
  • 1
    "(without implicit r0 type parameter L can't be inferred upon call of .apply() because L becomes known only upon call .apply().apply(...))" - Yes, that's exactly the problem I was hitting my head against. You put that nicely. I see your additional comment above; noted; will check. – Tim Oct 19 '20 at 23:05
  • I see; the crucial difference is that you're instantiating & importing the implicit in order to import "ops._". That does indeed fix the compile error (and run correctly). The problem is, unless I misunderstand, this means that they'd have to stop constructing the graph, assign it (as constructed so far) to a named value, import an "graph.op.", and then continue, e.g.: "graph.toCaseClass[X].withAttributes(...).run()". That may be the only possible option, but I was hoping to have something where you could just use the ".mapMat()/toCaseClass()" as usual, in the middle of the chain. – Tim Oct 19 '20 at 23:20
  • @Tim Yeah, without the import your implicit is not visible in the call site. – Dmytro Mitin Oct 19 '20 at 23:30
  • @Tim Maybe you don't need `Wrapper`. Try `PartiallyApplied` approach. – Dmytro Mitin Oct 19 '20 at 23:32
  • @Tim Thanks for accepting and upvoting my answer but I don't understand why my last version of `GraphOps` doesn't suit you and you write that this is "not what you were hoping for" and you need macros. – Dmytro Mitin Oct 20 '20 at 00:49
  • 1
    @Dmtryo It does work. It seems IntelliJ's presentation compiler was misleading me, saying it didn't compile due to missing implicits. (Who knows, I might have been hit that permutation earlier, myself, and missed it due to that.). I've updated the answer above to correct my error. Thank you VERY much, Dmytro, and best to you! – Tim Oct 20 '20 at 15:57