I'm haveing trouble, I presume with serialization, in a Spark repl.
I have the following code snippet:
class Foobar (val a: Int, val b: Int) extends Serializable {
override def equals (other: Any): Boolean =
other match {
case that: Foobar =>
println("Comparison of similar objects")
this.a == that.a && this.b == that.b
case _ =>
println("Comparison of disparate objects")
false
}
override def toString = s"[$a:$b]"
}
If I create two instances, one (foo) on a worker, and one (bar) on the driver:
val foo = sc.parallelize(Seq(1)).map(n => new Foobar(n, n)).collect.apply(0)
val bar = new Foobar(1, 1)
then foo != bar
(and spouts "Comparison of disparate objects") - yet
foo.getClass == bar.getClass
foo.a == bar.a
, andfoo.b == bar.b
Can anyone explain why this happens?