How provide encoders for custom spark object?

Question

I have a custom object that I want to use in spark.

var myTuple : Seq[(String, MyCustomObject)] = Seq()


class MyCustomObject(var x: String) extends Serializable {

  override def toString = s"$x"

  @throws[IOException]
  private def writeObject(out: ObjectOutputStream): Unit = {
    out.writeObject(x)
  }

  @throws[IOException]
  @throws[ClassNotFoundException]
  private def readObject(in: ObjectInputStream): Unit = {
    var myString = ""
    in.readObject(myString)
    x = myString;
  }
}

On the above, I defined a custom object class that extends the serializable, but I still get the following error

java.lang.UnsupportedOperationException: No Encoder found for MyCustomObject

Can someone point me a direction?

Edit:

I've look at the duplicate but still have trouble getting the following without error

var myTuple : Seq[Seq((String, MyCustomObject))] = Seq()

I've basically added

implicit def single[A](implicit c: ClassTag[A]): Encoder[A] = Encoders.kryo[A](c)

implicit def tuple2[A1, A2](
  implicit e1: Encoder[A1],
           e2: Encoder[A2]
): Encoder[(A1,A2)] = Encoders.tuple[A1,A2](e1, e2)

See https://stackoverflow.com/questions/36648128/how-to-store-custom-objects-in-dataset — maogautam, Sep 11 '19 at 21:28
Possible duplicate of [How to store custom objects in Dataset?](https://stackoverflow.com/questions/36648128/how-to-store-custom-objects-in-dataset) — Luis Miguel Mejía Suárez, Sep 11 '19 at 21:33

How provide encoders for custom spark object?

0 Answers0