1

I have file in my local HDFS and it it is delimited by ':::'.

However when i am using the following command

A = load '/user/vishal/WordCount/hw3data/c0001' using PigStorage(':::') as (a, b, c);

it gives me the following error----

ERROR 1200: could not instantiate 'PigStorage' with arguments '[:::]'

What exactly could be the issue?

Thanks

Uselesssss
  • 2,127
  • 6
  • 28
  • 37
  • 1
    I think Pig will not allow multiple characters as delimiter,you have to change the delimiter of the file manually to single character with unix tr command,later try to load – Balaswamy Vaddeman Feb 28 '13 at 07:09

1 Answers1

3

PigStorage supports single-character delimiter only.
A solution would be to either follow Donald's answer or have a look at MyRegExLoader if you don't want to create a custom loader. In your case it looks something like this:

REGISTER '/my_pig_home/contrib/piggybank/java/piggybank.jar'
A = LOAD '/user/vishal/WordCount/hw3data/c0001' 
  USING org.apache.pig.piggybank.storage.MyRegExLoader(
    '([^\\:]+):::([^\\:]+):::([^\\:]+)') 
      as (a:chararray, b:chararray, c:chararray);
Community
  • 1
  • 1
Lorand Bendig
  • 10,630
  • 1
  • 38
  • 45
  • Hi Lonard, While trying for the above thing I got the following error ERROR 2998: Unhandled internal error. Implementing class.What could be the reason – Uselesssss Mar 02 '13 at 07:01
  • Which Pig version do you use? Could you provide some more information about the issue (e.g stack trace) ? – Lorand Bendig Mar 05 '13 at 14:16
  • I am using 0.10.1(pig version) Following is the error--- 2013-03-07 14:35:58,235 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 2998: Unhandled internal error. Implementing class Details at logfile: /home/vishal/pig_1362638150546.log ERROR 2998: Unhandled internal error. Implementing class – Uselesssss Mar 07 '13 at 09:06
  • following is stack trace java.lang.IncompatibleClassChangeError: Implementing class at java.lang.ClassLoader.defineClass1(Native Method) at java.lang.ClassLoader.defineClassCond(ClassLoader.java:631) at java.lang.ClassLoader.defineClass(ClassLoader.java:615) at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:141) at java.net.URLClassLoader.defineClass(URLClassLoader.java:283) at j – Uselesssss Mar 07 '13 at 09:08
  • at java.net.URLClassLoader.access$000(URLClassLoader.java:58) at java.net.URLClassLoader$1.run(URLClassLoader.java:197) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:190) at java.lang.ClassLoader.loadClass(ClassLoader.java:306) at java.lang.ClassLoader.loadClass(ClassLoader.java:247) at java.lang.ClassLoader.defineClass1(Native Method) at java.lang.ClassLoader.defineClassCond(ClassLoader.java:631) at java.lang.ClassLoader.defineClass(ClassLoader.java:615) – Uselesssss Mar 07 '13 at 09:10
  • at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:141) at java.net.URLClassLoader.defineClass(URLClassLoader.java:283) at java.net.URLClassLoader.access$000(URLClassLoader.java:58) at java.net.URLClassLoader$1.run(URLClassLoader.java:197) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:190) at java.lang.ClassLoader.loadClass(ClassLoader.java:306) at java.lang.ClassLoader.loadClass(ClassLoader.java:247) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:247) – Uselesssss Mar 07 '13 at 09:10
  • Isn't the end of the first part of the stack trace is missing? I don't see the failing class. Not quite sure, but I'd rebuild piggybank and would also check the classpath if it contains the right Pig version. Some explanation about the IncompatibleClassChangeError: http://stackoverflow.com/a/1980474/1050422 – Lorand Bendig Mar 07 '13 at 17:31
  • I had this class compatibility issue. In my case, it is an issue of using different pig version than the piggybank came with. If we use pig that comes out of the trunk, it works – kich Apr 19 '13 at 16:04