1
scala> val dates = filtering1.map(x => (format.parse(x._1),format.parse(x._2)))
dates: org.apache.spark.rdd.RDD[(java.util.Date, java.util.Date)] = MapPartitionsRDD[7] at map at <console>:34

stores below values 

scala> dates.collect
res0: Array[(java.util.Date, java.util.Date)] = Array((Sat Jun 30 23:42:00 IST 2018,Thu Jul 04 15:10:00 IST 2019), (Sat Jun 30 23:37:00 IST 2018,Sun Jul 01 14:44:00 IST 2018), (Sat Jun 30 23:13:00 IST 2018,Sun Feb 28 23:34:00 IST 219), (Sat Jun 30 22:58:00 IST 2018,Mon Jul 01 18:22:00 IST 2019), (Sat Jun 30 22:36:00 IST 2018,Mon Jul 01 16:01:00 IST 2019), (Sat Jun 30 21:53:00 IST 2018,Tue Jul 02 10:36:00 IST 2019), (Sat Jun 30 21:42:00 IST 2018,Sun Jun 30 23:25:00 IST 2019), (Sat Jun 30 21:36:00 IST 2018,Mon Jul 01 16:47:00 IST 2019), (Sat Jun 30 21:16:00 IST 2018,Mon Jul 01 18:18:00 IST 2019), (Sat Jun 30 21:10:00 IST 2018,Thu Jul 04 12:25:00 IST 2019), (Sat Jun 30 21:02:00 IST 2018,Sat Dec 01 17:29:00 IST 2018), (Sat Jun 30 20:54:00 IST 2018,Mon Jul 01 15:51:00 IST 2019), (Sat Jun 30 ...

But how to perform operation so the difference in dates, is grouped together, gives value in minutes.

I have command , it does not give me desired output, what changes should be made?

val time_diff = dates.map(x => (x._2.getTime()-x._1.getTime())/(60*1000)%60)

what is (60*1000)%60) values represent?

Mahesh Chand
  • 3,158
  • 19
  • 37
Prateek Kumar
  • 13
  • 8
  • Possible duplicate of [Calculate date/time difference in java](https://stackoverflow.com/questions/5351483/calculate-date-time-difference-in-java) – Ole V.V. Nov 14 '17 at 15:38

2 Answers2

1

getTime gives milliseconds, so dividing by 1000.0 gives seconds and dividing by 1000.0*60 gives minutes. Be aware that dividing a Long by an Int gives you another Long, so you are truncating the resulting minutes to the next lowest integer. Adding modulus 60, % 60, simply wraps the minutes to 0-59, so if you had a 90 minutes difference, that would be 1 hour 30 minutes, and the result of your calculation would just be 30.

val t = System.currentTimeMillis
val x = new java.util.Date(t)
val y = new java.util.Date(t + 10000) // ten seconds later
(y.getTime - x.getTime) / (1000.0 * 60) // 0.167
(y.getTime - x.getTime) / (1000   * 60) // 0 !
0__
  • 66,707
  • 21
  • 171
  • 266
1

First problem when you subtract two times, time might go negative. Second, getTime returns value in miliseconds. 

1000ms = 1 second

So, first there is need to divide it by 1000 to get time in seconds. To get in minutes,divide it again by 60. Since you require result in minutes. 

val time_diff = dates.map(x => (x._2.getTime()-x._1.getTime())/(60*1000))
Mahesh Chand
  • 3,158
  • 19
  • 37