Imagine we have some
val hiveTableDataFrame: DataFrame
And I want to stream rows of this data frame into my driver program. The thing is the hiveTableDataFrame
is too large and I cannot use hiveTableDataFrame.collect()
. Is there a way to read the data from the dataframe by chunks?