How does toDS() get injected into the Seq object

Question

Normally in Scala if you invoke toDS() on a sequence, you get:

error: value toDS is not a member of Seq[Int]

However, in a SparkSession, you can invoke the same method on a Seq and it creates a DataSet.

How is this MetaProgramming actually happening?

`:imports` in scala shell vs spark-shell will show you the difference — philantrovert, Apr 05 '18 at 10:03

score 2 · Accepted Answer · answered Apr 05 '18 at 10:14

2

import org.apache.spark.sql.SparkSession

val spark = SparkSession.builder.getOrCreate()
import spark.implicits._

answered Apr 05 '18 at 10:14

user9601637

1

Ok but I'm confused. How can an import of another class effect an existing class? – More Than Five Apr 05 '18 at 10:18
@MoreThanFive Can you please share your findings if any? – Naveen Santhanavel Nov 29 '20 at 09:03

1 Answers1