I am facing a basic issue of importing a python script into PIG. i'm just trying a simple script like this:
PIG script
REGISTER 'test.py' using jython as testPyUDF;
load_data = LOAD '$input_path' USING PigStorage(',') AS (row1);
resp = FOREACH load_data generate row1, testPyUDF.testFunc() as (test:CHARARRAY);
DUMP resp;
test.py (Python UDF to import):
@outputSchema("test:chararray")
def testFunc():
return "test"
But this results in an error: ExecException: ERROR 1070: Could not resolve testPyUDF.testFunc using imports: [, org.apache.pig.builtin., org.apache.pig.impl.builtin., com.yahoo.yst.sds.ULT., org.apache.pig.piggybank.evaluation., org.apache.pig.piggybank.evaluation.datetime., org.apache.pig.piggybank.evaluation.decode., org.apache.pig.piggybank.evaluation.math., org.apache.pig.piggybank.evaluation.stats., org.apache.pig.piggybank.evaluation.string., org.apache.pig.piggybank.evaluation.util., org.apache.pig.piggybank.evaluation.util.apachelogparser., string., util., math., datetime., sequence., util., java.lang., org.apache.pig.builtin., org.apache.pig.impl.builtin.]
I even tried changing the python script path in REGISTER to HDFS path:
REGISTER 'hdfs:///user/xxx/......./test.py' using jython as testPyUDF;
and also absolute path on local..
REGISTER '/homes/user/..../test.py' using jython as testPyUDF;
No luck either way. What I am missing here?
