comparing datetime in pig

Question

in pig 11, is there a support for comparing datetime types? for example: date1:datetime

and filter has condition: date1 >= ToDate('1999-01-01')

does this comparison returns correct result?

Answer 1

Date comparison can be considered as a numerical comparison. E.g:

cat date1.txt
1999-01-01
2011-03-19
2011-02-24
2011-02-25
2011-05-23
1978-12-13

A = load 'date1.txt' as (in:chararray);
B = foreach A generate ToDate(in, 'yyyy-MM-dd') as (dt:datetime);
--filter dates that are equal or greater than 2011-02-25: 
C = filter B by DaysBetween(dt, 
      (datetime)ToDate('2011-02-25', 'yyyy-MM-dd')) >=(long)0;

dump C;
(2011-03-19T00:00:00.000+01:00)
(2011-02-25T00:00:00.000+01:00)
(2011-05-23T00:00:00.000+02:00)

The custom format pattern passed to ToDate follows the Java's SimpleDateFormat convention.
Watch out for the uppercase and lowercase letters, for example D means Day in year but d refers to Day in month . This can lead to an inappropriate date conversion from chararray to datetime.

Alternatively, if your chararray dates are in ISO format, you may use the Piggybank's UDFs as well.