0

The question refers to a local pig 0.11.1 run, but with a usage prospect eventually on EMR's pig. I'm trying to write a jython UDF with an import of the 're' module, and I get this after I register 'xyz.py' with jython as xyz:

2013-11-06 13:59:47,259 [main] WARN  org.apache.pig.scripting.jython.JythonScriptEngine -   module file does not exist: re, /home/amit/Servers/Pig/pig-0.11.1/lib/jython-standalone-2.5.3.jar/Lib/re.py

The udf is just a simple:

import re

I have PIG_HOME setup in my .bashrc and the question is basically what's wrong ? It's worthwhile noting that the below example works

import sys
print(sys.path)

Thanks, Amit

Amit
  • 1
  • 1
  • I encountered the same issue and solved it as described here: http://stackoverflow.com/a/24542649/1128392 – erwaman Jul 02 '14 at 23:29

1 Answers1

0

It appears the installation is either missing the stdlib or can not find it. import sys should work since it's built into the runtime itself unlike the re module which lives outside of the runtime in a separate python module. Try and import another module that lives in the standard library like unittest. I suspect you'll have similar result.

@Amit I downloaded the same version to confirm, and I was able to import the re module inside a UDF. It sounds like it's a problem wiht your local installation. I'd suggest trying the following:

  1. Delete and redownload the Pig tgz file.
  2. If the problem is still present, confirm that you don't have an alternative jython jar sitting in your classpath. Pig comes bundled with a Jython jar in the /Lib directory. Inside of this jar is the python stdlib (including the re module). Jython also comes bundled as a 'bare' jar which does not include the stdlib. If this is on your classpath it could cause your problem.
Mark Roddy
  • 27,122
  • 19
  • 67
  • 71
  • The same errors have indeed risen when trying to import `unittest`. I really didn't install Pig besides unpacking its binary package, setting `PIG_HOME` and adding it to the `PATH` env variables. What else should be setup so that it coult import the aforementioned jython modules ? For those interested, I'm currently using `from java.util.regex import Pattern` and its likes to do regex and other stuff related to pattern matching - a hack, but works. – Amit Nov 11 '13 at 13:39
  • What version of Pig are you using? Sounds like it *might* be a bug in the package. – Mark Roddy Nov 13 '13 at 03:14
  • Apache Pig version 0.11.1 (r1459641) compiled Mar 22 2013, 02:13:53 – Amit Nov 13 '13 at 08:17