How to Specify ToDate during Schema Definition?

Question

I load my files and generate dates for them in two steps:

A = LOAD 'foo.txt' USING PigStorage('\\u001') AS (
    bar:chararray
    ,baz:int
);
B = FOREACH A GENERATE
    ToDate(bar, 'yyyy-MM-dd HH:mm:ss') AS bar
    ,baz
;

How can I do it in one step?

The following throws an error (mismatched input '(' expecting RIGHT_PARAN:

A = LOAD 'foo.txt' USING PigStorage('\\u001') AS (
    ToDate(bar:chararray, 'yyyy-MM-dd HH:mm:ss') AS bar
    ,baz:int
);

or

A = LOAD 'foo.txt' USING PigStorage('\\u001') AS (
    ToDate($0, 'yyyy-MM-dd HH:mm:ss') AS bar
    ,baz:int
);

score 1 · Accepted Answer · answered Apr 15 '14 at 10:14

1

UDFs can't be applied in the schema definition. You may write your own loader instead.

answered Apr 15 '14 at 10:14

Lorand Bendig

10,630
1
38
45

score 0 · Answer 2 · edited May 23 '17 at 11:57

0

Try using ^A instead of \\u001. For more information, use this link.

edited May 23 '17 at 11:57

Community

1
1

answered Apr 15 '14 at 10:07

Cast_A_Way

472
5
19

The question was not about delimiting but applying ToDate – Lorand Bendig Apr 15 '14 at 10:15

How to Specify ToDate during Schema Definition?

2 Answers2