0

Hi I am loading the files in the folder into HDFS using Pig for processing. The folder names are in the format MM-dd-yyyy. For instance

/Test/02-10-2016/file1
/Test/02-11-2016/file2
/Test/20-12-2016/file3

Likewise /Test folder includes date from 01-01-2015 to 20-12-2016. while loading this folder I want to exclude some date files such as 02-10-2016,02-09-2016.

I know we can use regex in the load statement like

ex = load '/TEST/*';

Like this how to use regex to exclude some files in load statement?

wazza
  • 770
  • 5
  • 17
  • 42
  • 1
    A few examples of valid and invalid dates will be beneficial. –  Feb 12 '16 at 04:46
  • 1
    That's considered a glob, not a regex – OneCricketeer Feb 12 '16 at 04:48
  • Possible duplicate of [Pig Latin: Load multiple files from a date range (part of the directory structure)](http://stackoverflow.com/questions/3515481/pig-latin-load-multiple-files-from-a-date-range-part-of-the-directory-structur) – OneCricketeer Feb 12 '16 at 04:50

0 Answers0