How to write generic function with Scala Quill.io library

Question

I am trying to implement generic method in Scala operating on database using Quill.io library. Type T will be only case classes what works with Quill.io.

def insertOrUpdate[T](inserting: T, equality: (T,T) => Boolean)(implicit ctx: Db.Context): Unit = {
  import ctx._

  val existingQuery = quote {
    query[T].filter { dbElement: T =>
      equality(dbElement, inserting)
    }
  }
  val updateQuery = quote {
    query[T].filter { dbElement =>
      equality(dbElement, lift(inserting))
    }.update(lift(inserting))
  }
  val insertQuery = quote { query[T].insert(lift(inserting)) }

  val existing = ctx.run(existingQuery)
  existing.size match {
    case 1 => ctx.run(updateQuery)
    case _ => ctx.run(insertQuery)

  }
}

But I am getting two types of compile error

Error:(119, 12) Can't find an implicit `SchemaMeta` for type `T`
  query[T].filter { dbElement: T =>

Error:(125, 33) Can't find Encoder for type 'T'
    equality(dbElement, lift(inserting))

How can I modify my code to let it work?

score 3 · Accepted Answer · answered Jun 28 '17 at 08:20

As I said in the issue that @VojtechLetal mentioned in his answer you have to use macros.

I added code implementing generic insert or update in my example Quill project.

It defines trait Queries that's mixed into context:

trait Queries {
  this: JdbcContext[_, _] =>
  def insertOrUpdate[T](entity: T, filter: (T) => Boolean): Unit = macro InsertOrUpdateMacro.insertOrUpdate[T]
}

This trait uses macro that's implementing your code with minor changes:

import scala.reflect.macros.whitebox.{Context => MacroContext}

class InsertOrUpdateMacro(val c: MacroContext) {

  import c.universe._

  def insertOrUpdate[T](entity: Tree, filter: Tree)(implicit t: WeakTypeTag[T]): Tree =
    q"""
      import ${c.prefix}._
      val updateQuery = ${c.prefix}.quote {
        ${c.prefix}.query[$t].filter($filter).update(lift($entity))
      }
      val insertQuery = quote {
        query[$t].insert(lift($entity))
      }
      run(${c.prefix}.query[$t].filter($filter)).size match {
          case 1 => run(updateQuery)
          case _ => run(insertQuery)
      }
      ()
    """
}

Usage examples:

import io.getquill.{PostgresJdbcContext, SnakeCase}

package object genericInsertOrUpdate {
  val ctx = new PostgresJdbcContext[SnakeCase]("jdbc.postgres") with Queries

  def example1(): Unit = {
    val inserting = Person(1, "")
    ctx.insertOrUpdate(inserting, (p: Person) => p.name == "")
  }

  def example2(): Unit = {
    import ctx._
    val inserting = Person(1, "")
    ctx.insertOrUpdate(inserting, (p: Person) => p.name == lift(inserting.name))
  }
}

P.S. Because update() returns number of updated records your code can be simplified to:

class InsertOrUpdateMacro(val c: MacroContext) {

  import c.universe._

  def insertOrUpdate[T](entity: Tree, filter: Tree)(implicit t: WeakTypeTag[T]): Tree =
    q"""
      import ${c.prefix}._
      if (run(${c.prefix}.quote {
        ${c.prefix}.query[$t].filter($filter).update(lift($entity))
      }) == 0) {
          run(quote {
            query[$t].insert(lift($entity))
          })
      }
      ()
    """
}

Your example needed only one change in `InsertOrUpdateMacro`. I was getting error from ambiguous import if `ctx._`. So I removes this import from your macro and added explicit calls where needed: `def insertOrUpdate[T](entity: Tree, filter: Tree)(implicit t: WeakTypeTag[T]): Tree = q""" if (${c.prefix}.run(${c.prefix}.quote { ${c.prefix}.query[$t].filter($filter).update(${c.prefix}.lift($entity)) }) == 0) { ${c.prefix}.run(quote { query[$t].insert(${c.prefix}.lift($entity)) }) } () """` — snilard, Jun 28 '17 at 11:40
I got this error: Error:scalac: Error: symbol_= inapplicable for java.lang.UnsupportedOperationException: symbol_= inapplicable for at scala.reflect.internal.Trees$Tree.symbol_$eq(Trees.scala:54) at scala.reflect.internal.Trees$Tree.setSymbol(Trees.scala:55) ... The version from @snilard did not compile. Maybe post a gist or fork the project? — Mike Slinn, Jun 28 '17 at 23:32
@mslinn Did you get this error when compiling my example project? — mixel, Jun 29 '17 at 06:45

score 0 · Answer 2 · answered Jun 27 '17 at 16:58

As one of the quill contributors said in this issue:

If you want to make your solution generic then you have to use macros because Quill generates queries at compile time and T type has to be resolved at that time.

TL;DR The following did not work either, just playing

Anyway... just out of curiosity I tried to fix the issue by following the error which you mentioned. I changed the definition of the function as:

def insertOrUpdate[T: ctx.Encoder : ctx.SchemaMeta](...)

which yielded the following log

[info] PopulateAnomalyResultsTable.scala:71: Dynamic query
[info]       case _ => ctx.run(insertQuery)
[info]  
[error] PopulateAnomalyResultsTable.scala:68: exception during macro expansion: 
[error] scala.reflect.macros.TypecheckException: Found the embedded 'T', but it is not a case class
[error]         at scala.reflect.macros.contexts.Typers$$anonfun$typecheck$2$$anonfun$apply$1.apply(Typers.scala:34)
[error]         at scala.reflect.macros.contexts.Typers$$anonfun$typecheck$2$$anonfun$apply$1.apply(Typers.scala:28)

It starts promising, since quill apparently gave up on static compilation and made the query dynamic. I checked the source code of the failing macro and it seems that quill is trying to get a constructor for T which is not known in the current context.

Andrzej Jozwik · Answer 3 · 2019-09-04T18:40:19.280

For more details see my answer Generic macro with quill or implementation CrudMacro:

Complete project you will find on quill-generic

   package pl.jozwik.quillgeneric.quillmacro

    import scala.reflect.macros.whitebox.{ Context => MacroContext }

    class CrudMacro(val c: MacroContext) extends AbstractCrudMacro {

      import c.universe._

  def callFilterOnIdTree[K: c.WeakTypeTag](id: Tree)(dSchema: c.Expr[_]): Tree =
    callFilterOnId[K](c.Expr[K](q"$id"))(dSchema)

  protected def callFilterOnId[K: c.WeakTypeTag](id: c.Expr[K])(dSchema: c.Expr[_]): Tree = {
    val t = weakTypeOf[K]

    t.baseClasses.find(c => compositeSet.contains(c.asClass.fullName)) match {
      case None =>
        q"$dSchema.filter(_.id == lift($id))"
      case Some(base) =>
        val query = q"$dSchema.filter(_.id.fk1 == lift($id.fk1)).filter(_.id.fk2 == lift($id.fk2))"
        base.fullName match {
          case `compositeKey4Name` =>
            q"$query.filter(_.id.fk3 == lift($id.fk3)).filter(_.id.fk4 == lift($id.fk4))"
          case `compositeKey3Name` =>
            q"$query.filter(_.id.fk3 == lift($id.fk3))"
          case `compositeKey2Name` =>
            query
          case x =>
            c.abort(NoPosition, s"$x not supported")

        }
    }
  }

      def createAndGenerateIdOrUpdate[K: c.WeakTypeTag, T: c.WeakTypeTag](entity: Tree)(dSchema: c.Expr[_]): Tree = {
        val filter = callFilter[K, T](entity)(dSchema)
        q"""
          import ${c.prefix}._
          val id = $entity.id
          val q = $filter
          val result = run(
            q.updateValue($entity)
          )
          if (result == 0) {
            run($dSchema.insertValue($entity).returningGenerated(_.id))
          } else {
            id
          }
        """
      }

      def createWithGenerateIdOrUpdateAndRead[K: c.WeakTypeTag, T: c.WeakTypeTag](entity: Tree)(dSchema: c.Expr[_]): Tree = {
        val filter = callFilter[K, T](entity)(dSchema)
        q"""
          import ${c.prefix}._
          val id = $entity.id
          val q = $filter
          val result = run(
            q.updateValue($entity)
          )
          val newId =
            if (result == 0) {
              run($dSchema.insertValue($entity).returningGenerated(_.id))
            } else {
              id
            }
          run($dSchema.filter(_.id == lift(newId)))
          .headOption
          .getOrElse(throw new NoSuchElementException(s"$$newId"))
        """
      }
    }

How to write generic function with Scala Quill.io library

3 Answers3