0

I am writing a small Scala Program which should:

  1. Read a file (line by line) from a local FS
  2. Parse from each line three double values
  3. Make instances of a case class based on those three values
  4. Pass those instances to a Binary Heap

To be able to parse Strings to both Doubles and CoordinatePoints I've came up with this trait:

trait Parseable[T] {
  def parse(input: String): Either[String, T]
}

and I have a number of type object implementations for the latter:

object Parseable {
  implicit val parseDouble: Parseable[Double] = new Parseable[Double] {
    override def parse(input: String): Either[String, Double] = {
      val simplifiedInput = input.replaceAll("[ \\n]", "").toLowerCase
      try Right(simplifiedInput.toDouble) catch {
        case _: NumberFormatException =>
          Left(input)
      }
    }
  }

  implicit val parseInt: Parseable[Int] = new Parseable[Int] {
    override def parse(input: String): Either[String, Int] = {
      val simplifiedInput = input.replaceAll("[ \\n]", "").toLowerCase
      try Right(simplifiedInput.toInt) catch {
        case _: NumberFormatException =>
          Left(input)
      }
    }
  }

  implicit val parseCoordinatePoint: Parseable[CoordinatePoint] = new Parseable[CoordinatePoint] {
    override def parse(input: String): Either[String, CoordinatePoint] = {
      val simplifiedInput = input.replaceAll("[ \\n]", "").toLowerCase
      val unparsedPoints: List[String] = simplifiedInput.split(",").toList
      val eithers: List[Either[String, Double]] = unparsedPoints.map(parseDouble.parse)
      val sequence: Either[String, List[Double]] = eithers.sequence
      sequence match {
        case Left(value) => Left(value)
        case Right(doublePoints) => Right(CoordinatePoint(doublePoints.head, doublePoints(1), doublePoints(2)))
      }
    }
  }
}

I have a common object that delegates the call to a corresponding implicit Parseable (in the same file):

object InputParser {
  def parse[T](input: String)(implicit p: Parseable[T]): Either[String, T] = p.parse(input)
}

and just for reference - this is the CoordinatePoint case class:

case class CoordinatePoint(x: Double, y: Double, z: Double)

In my main program (after having validated that the file is there, and is not empty, etc..) I want to transform each line into an instance of CoordinatePoint as follows:

  import Parseable._
  import CoordinatePoint._

  ...
  private val bufferedReader = new BufferedReader(new FileReader(fileName))

  private val streamOfMaybeCoordinatePoints: Stream[Either[String, CoordinatePoint]] = Stream
    .continually(bufferedReader.readLine())
    .takeWhile(_ != null)
    .map(InputParser.parse(_))

and the error I get is this:

[error] /home/vgorcinschi/data/eclipseProjects/Algorithms/Chapter 2 Sorting/algorithms2_1/src/main/scala/ca/vgorcinschi/algorithms2_4/selectionfilter/SelectionFilter.scala:42:27: ambiguous implicit values:
[error]  both value parseDouble in object Parseable of type => ca.vgorcinschi.algorithms2_4.selectionfilter.Parseable[Double]
[error]  and value parseInt in object Parseable of type => ca.vgorcinschi.algorithms2_4.selectionfilter.Parseable[Int]
[error]  match expected type ca.vgorcinschi.algorithms2_4.selectionfilter.Parseable[T]
[error]     .map(InputParser.parse(_))
[error]                           ^
[error] one error found
[error] (Compile / compileIncremental) Compilation failed
[error] Total time: 1 s, completed Sep 1, 2020 10:38:18 PM

I don't understand nor know where to look for why is the compiler finding Parseable[Int] and Parseable[Double] but not the only right one - Parseable[CoordinatePoint].

So I thought, ok let me give the compiler a hand by specifying the transformation function from beforehand:

  private val bufferedReader = new BufferedReader(new FileReader(fileName))

  val stringTransformer: String => Either[String, CoordinatePoint] = s => InputParser.parse(s)

  private val streamOfMaybeCoordinatePoints: Stream[Either[String, CoordinatePoint]] = Stream
    .continually(bufferedReader.readLine())
    .takeWhile(_ != null)
    .map(stringTransformer)

Alas this yields the same error just a bit up the code - in the function declaration.

I would love to learn what is that that causes such behavior. Both to rectify the code and for personal knowledge. At this point I am very curious.

vasigorc
  • 882
  • 11
  • 22
  • Underscore imports and implicits is almost a sure way to have something like this. Could you try replacing `import Parseable._` with `ImportParser.parseCoordinatePoint`? If it works out, it means you're stretching the type inference a bit farther than it could go. – J0HN Sep 02 '20 at 02:54
  • 1
    So it would be `Parseable.parseCoordinatePoint.parse` I tried it and I get this ```[error] there was one unchecked warning; re-run with -unchecked for details [error] there were three feature warnings; re-run with -feature for details [error] two errors found [error] (Compile / compileIncremental) Compilation failed [error] Total time: 3 s, completed Sep 1, 2020 11:00:26 PM ``` Probably unrelated. To what you said - isn't it the advantage of implicits so that the compiler picks the right value? – vasigorc Sep 02 '20 at 03:02
  • This happens on `sbt compile`... – vasigorc Sep 02 '20 at 03:03
  • So yes the `errors` go away once I remove the `"-Xfatal-warnings"` from `scalac` options in `build.sbt`...Back to your point I marked `Parseable` as implicit parameter to `InputParser.parse` . If this is not the right way to do it - I would love to know what is. What you're suggesting is not relying on implicits at all. – vasigorc Sep 02 '20 at 03:06
  • It is, but sometime s it needs some help doing that. I don't feel confident to talk about when and why it happens, but quite a few cases of such issues I've faced myself was due to generics type erasure found in JVM languages (Scala is not except). I.e. in your case it might be because the parse returns `Either[String, CoordinatePoint]`, so after the type erasure the compiler sees it as jut `Either[_]`. -and then all the other parse implicits look the same. You might want to re-run it with `-unchecked` to see the error, it might be insightful. – J0HN Sep 02 '20 at 03:07
  • 3
    Try `map(InputParser.parse[CoordinatePoint])` you have to tell the compiler which type do you want and it will search the implicit for that type. - BTW, I would recommend you to use `scala.util.Using` & `scala.io.Source` for reading the file. – Luis Miguel Mejía Suárez Sep 02 '20 at 03:08
  • No, I'm not suggesting that. I'm suggesting to try to only import one implicit (not replace it with the actual call) and see if it works like that. If it is, it likely happens due to type erasure, so you might want to help the compiler a bit and specify the types on the map call (i.e. `map[Either[String, Coordinate]]`). If it isn't - there's some other error in the code that might be affecting type inference – J0HN Sep 02 '20 at 03:09
  • 1
    @JOHN erasure do not affect the compiler in any way. Erasure is a consequence of one of the runtimes, not a property of the language. – Luis Miguel Mejía Suárez Sep 02 '20 at 03:10
  • @LuisMiguelMejíaSuárez I'm afraid I don't think you're right :) Type erasure is part of the compilation: https://typelevel.org/scala/docs/phases.html – J0HN Sep 02 '20 at 03:12
  • 1
    @JOHN Of course it will be considered by the compiler that writes JVM byte code. By that argument then the JVM byte code is part of the language, or the boxing and unboxing are part of the language, or the minification done by the scalajs compiler is also part of the language. - Again, erasure is not a language concept, it is not considered when the type checker and the implicit resolution is run. But much latter. While erasure is an important concept for Scala programmers, since most of our code runs in the JVM, we need to separate the language from the _(default)_ runtime. – Luis Miguel Mejía Suárez Sep 02 '20 at 03:22
  • @LuisMiguelMejíaSuárez I see your point, and I agree that from theoretical perspective that language and runtime are different. From practical perspective though, there is pretty concrete code that fails _compilation_, not at runtime. As I've said, Im not confident enough to talk about if and when erasure causes issues (it is indeed run much later than typing), so I might be wrong in my reasoning. Could you explain (to OP and to me as well) _why_ specifying an explicit type works? (genuine question - I'm interested to now). I.e. beyond the "give a hint to compiler" :) – J0HN Sep 02 '20 at 03:33
  • 1
    @JOHN so actually I should correct my previous comment since what I wanted to say was that erasure do not affect implicits _(I wanted to generalise and said compilation, but you proven me wrong in that regard)_ - Now, why specifying the type solves the problem, is somewhat complex to explain, I will try to make my best. Type inference in **Scala 2** is local and not global, so when solving the implicit for `parse` the compiler refuses to use the return type of all the expression as a hint, instead it tries to see if it can find only one valid implicit and use that to infer the type of the map – Luis Miguel Mejía Suárez Sep 02 '20 at 03:49
  • AFAIK, **Scala 3** will improve a lot in this regard and probably OPs code should compile in dotty _(making the corresponding changes obviously)_ – Luis Miguel Mejía Suárez Sep 02 '20 at 03:50
  • 1
    @LuisMiguelMejíaSuárez I see, thanks, makes sense! I've made a [simplified version](https://scastie.scala-lang.org/iGz1jm5JT3WEL2t2ZCJJcg) of the problem (without Either) and it indeed still fails as is, but works with specifying a type on InputParser.parse – J0HN Sep 02 '20 at 04:05
  • 1
    @J0HN Type erasure can't be relevant to type inference and implicit resolution. `erasure` is the 14th phase while type inference and implicit resolution occur at `typer` phase (the 4th phase). – Dmytro Mitin Sep 02 '20 at 10:37
  • 2
    @LuisMiguelMejíaSuárez You're right, in dotty this compiles https://scastie.scala-lang.org/LFznJGiPRKyGhfwvfxyD5Q (tested in 0.28.0-bin-20200901-0d22c74-NIGHTLY too). – Dmytro Mitin Sep 02 '20 at 11:09

2 Answers2

3

One fix is to specify type prameter explicitly

InputParser.parse[CoordinatePoint](_)

Another is to prioritize implicits. For example

trait LowPriorityParseable1 {
  implicit val parseInt: Parseable[Int] = ...
}

trait LowPriorityParseable extends LowPriorityParseable1 {
  implicit val parseDouble: Parseable[Double] = ...
}

object Parseable extends LowPriorityParseable {
  implicit val parseCoordinatePoint: Parseable[CoordinatePoint] = ...
}

By the way, since you put implicits into the companion object it doesn't make much sense now to import them.

In the call site of

object InputParser {
  def parse[T](input: String)(implicit p: Parseable[T]): Either[String, T] = p.parse(input)
}

type parameter T is inferred (if not specified explicitly) not before the implicit is resolved (type inference and implicit resolution make impact on each other). Otherwise the following code wouldn't compile

trait TC[A]
object TC {
  implicit val theOnlyImplicit: TC[Int] = null
}    
def materializeTC[A]()(implicit tc: TC[A]): TC[A] = tc
  
materializeTC() // compiles, A is inferred as Int

So during implicit resolution compiler tries to infer types not too early (otherwise in the example with TC type A would be inferred as Nothing and implicit wouldn't be found). By the way, an exception is implicit conversions where compiler tries to infer types eagerly (sometimes this can make troubles too)

// try to infer implicit parameters immediately in order to:
//   1) guide type inference for implicit views
//   2) discard ineligible views right away instead of risking spurious ambiguous implicits

https://github.com/scala/scala/blob/2.13.x/src/compiler/scala/tools/nsc/typechecker/Implicits.scala#L842-L854

Dmytro Mitin
  • 48,194
  • 3
  • 28
  • 66
1

The problem that the compiler does not inference and fix type parameter T in .map(InputParser.parse(_)) before trying to find the implicit in the second parameter list.

In the compiler, there is a concrete algorithm that infers types with its own logic, constraints, and tradeoffs. In that concrete compiler version that you use it first goes to the parameter lists and infer and checks types list by list, and only at the end, it infers type parameter by returning type (I do not imply that in other versions it differs, I only point out that it is implementation behavior not a fundamental constraint).

More precisely what is going on is that type parameter T is not being inferred or specified somehow at the step of typechecking of the second parameter list. T (at that point) is existential and it can be any/every type and there is 3 different implicit object that suitable for such type.

It is just how the compiler and its type inference works for now.

Artem Sokolov
  • 810
  • 4
  • 8