1974,1974-06-22 1966,1966-07-20 1954,1954-06-19 1994,1994-06-27 1954,1954-06-26 2006,2006-07-04 2010,2010-07-07 1990,1990-06-30 ...
It is type RDD[String].
What is wrong in the function iteself?
Asked
Active
Viewed 35 times
1 Answers
0
Try
def f(v: Iterable[Array[String]]): Int = {
val parsedDates = v.flatten.map(e => LocalDate.parse(e, formatter))
parsedDates.max.getDayOfYear - parsedDates.min.getDayOfYear
}
which outputs
val arrays = Iterable(Array("2014-10-10","2014-12-10"))
f(arrays) // res0: Int = 61
Mario Galic
- 47,285
- 6
- 56
- 98
-
1Error: org.apache.spark.SparkException: Task not serializable – Lazar Gugleta Jun 20 '19 at 09:05
-
1Try https://stackoverflow.com/questions/22592811/task-not-serializable-java-io-notserializableexception-when-calling-function-ou – Mario Galic Jun 20 '19 at 09:06