writing a udf in pig kind of like tutorial

Question

I am new to pig.. and am trying to write a udf function.

So basically here is the problem statement.

I have a dummy data like this..

 user_id, movie_id, date_time_stamp

So what I am trying to do is this. if the transaction is between

    9 am and 11 am --> breakfast
    and so on

So here is my pig script

     REGISTER path/myudfs.jar
      in = LOAD 'path/input' USING  
          PigStorage('\\u001') AS (user:long,movie:long, time:chararray);

     result = foreach in GENERATE  myudfs.time(time);
     STORE result INTO 'path/output/time' using PigStorage(',');

Now myudf.jar java code is like this

      public class time extends EvalFunc<String>{

public String exec(Tuple input) throws IOException {

    if ((input == null) || (input.size() == 0))
        return null;
    try{
        String time = (String) input.get(0) ;
        DateFormat df = new SimpleDateFormat("hh:mm:ss.000");
        Date date = df.parse(time);
        String timeOfDay = getTimeOfDay(date);
        return timeOfDay;
    } catch (ParseException e) {
        //how will I handle when df.parse(time) fails and throws ParseException?
        //maybe:
        return null;
    }


}

So it takes in the tuple and returns a string... (I am new to java also..)

After this i try to run this script as

 pig -f time.pig

It returns an error

   2012-11-12 08:33:08,214 [main] INFO    
    org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - Connecting to  
  hadoop file system at: maprfs:///
  2012-11-12 08:33:08,353 [main] INFO  
      org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - Connecting to  
                         map-reduce job tracker at: maprfs:///
  2012-11-12 08:33:08,767 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 1069:  
     Problem resolving class version numbers for class myudfs.time

Some one posted on pig mailing list is that my PIG_CLASSPATH is not set and that i should point it to /path/hadoop/conf

I did that.. so now $echo PIG_CLASSPATH --> /path/hadoop/conf

But i get the same error

Please advise. Thanks

Edit 1: On looking into the log, the error trace is:

     Caused by: java.lang.UnsupportedClassVersionError: myudfs/time : Unsupported major.minor version 51.0
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClassCond(ClassLoader.java:631)
at java.lang.ClassLoader.defineClass(ClassLoader.java:615)
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:141)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:283)
at java.net.URLClassLoader.access$000(URLClassLoader.java:58)
at java.net.URLClassLoader$1.run(URLClassLoader.java:197)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:247)
at org.apache.pig.impl.PigContext.resolveClassName(PigContext.java:427)
... 27 more

is this like a java issue?

'Unsupported version' usually suggests you have a conflict of versions for jar and jdk. This has been asked many times on SO, including here http://stackoverflow.com/questions/7237536/exception-in-thread-main-java-lang-unsupportedclassversionerror-a-unsupporte — Dave Richardson, Nov 12 '12 at 16:18
@DaveRlz: Thanks for giving me the clue.. I am new to java also.. hence didnt knew this.. How do I find out the jar and jdk versions...? and how do i make them compatible? I am using standard eclipse on windows.. — frazman, Nov 12 '12 at 16:21

score 5 · Accepted Answer · answered Nov 12 '12 at 16:49

To find the jar version, open the jar using winzip (or similar) and look for manifest.mf. There should be a line in there that says 'Created-By' and this will give the version of java that was used to build the jar.

This needs to be older or equal to the version of java you are using to build your app. If you are doing this at the command line type:

java -version

or in eclipse go to

project(menu) > properties (menu item) > java build path (in list) > libraries (tab)

and take a look at the version that you are using for the JDK/JRE (you may be able to tell this from the directory, if not then go to that directory and do java -version).

Chances are you'll need to update the version of java you have in eclipse.

writing a udf in pig kind of like tutorial

1 Answers1

Linked