Apache Pig UDF Resolution Issue

Question

I've got the following Java code (returns fixed value for testing):

Static.java

package com.company;

import java.io.IOException;
import org.apache.pig.EvalFunc;
import org.apache.pig.data.Tuple;
import org.apache.pig.impl.util.WrappedIOException;

public class Static extends EvalFunc<String>
{
        public String exec(Tuple input) throws IOException
        {
                if (input == null || input.size() == 0)
                    return null;
                try
                {
                        String str = (String)input.get(0);
                        return "5876L";
                }
                catch (Exception e)
                {
                        throw WrappedIOException.wrap("Caught exception processing input row ", e);
                }
        }
}

Built using

javac -Xbootclasspath:/usr/lib/jvm/j2sdk1.6-oracle/jre/lib/rt.jar -source 1.6 -cp `hadoop classpath`:/opt/cloudera/parcels/CDH-4.3.0-1.cdh4.3.0.p0.22/lib/pig/pig-0.11.0-cdh4.3.0.jar Static.java

jar -cf Static.jar Static.class

Apache Pig job

REGISTER /path/to/Static.jar;

loaded = LOAD 'data/' USING com.twitter.elephantbird.pig.store.LzoPigStorage() AS (line:chararray);

loaded = SAMPLE loaded 0.00001;

sized = FOREACH loaded GENERATE com.company.Static(line);

DUMP sized;

Is ran with

# pig -f static.pig -x local

Which errors

[main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 1070: Could not resolve com.company.Static using imports: [, org.apache.pig.builtin., org.apache.pig.impl.builtin.]

Changing the line

sized = FOREACH loaded GENERATE com.company.Static(line);

To

sized = FOREACH loaded GENERATE Static(request);

It errors with

[main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 2998: Unhandled internal error. Static (wrong name: com/company/Static)

Yea, sorry. It's consistently com.company across the board. I missed that package name when I was sanitizing the code. Could you remove your reference as well? — Carl Sagan, Oct 01 '13 at 07:55
Can you try `DEFINE Static com.company.Static();` and then use `Static` to call the UDF? Does that work? — Hari Menon, Oct 01 '13 at 08:22
[main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 1070: Could not resolve com.company.Static using imports: [, org.apache.pig.builtin., org.apache.pig.impl.builtin.] — Carl Sagan, Oct 01 '13 at 19:06

score 3 · Accepted Answer · edited May 23 '17 at 12:29

3

You seem to be missing the required directory structure in the jar. com.company.Static (i.e., the Static.class file) should be located under the com/company directory in the jar. See this other SO question for more details.

For more details, see my response to this other question you posted.

edited May 23 '17 at 12:29

Community

1
1

answered Oct 01 '13 at 19:35

cabad

4,555
1
20
33