Why does this Jython loop fail after a single run?

Question

I've got the following code:

public static String getVersion() 
{
    PythonInterpreter interpreter = new PythonInterpreter();

    try 
    {
        interpreter.exec(IOUtils.toString(new FileReader("./Application Documents/Scripts/Version.py")));
        PyObject get_version = interpreter.get("get_latest_version");
        PyObject result = get_version.__call__(interpreter.get("url"));
        String latestVersion = (String) result.__tojava__(String.class);
        interpreter.close();
        return latestVersion;
    } catch (IOException ex) {
        ex.printStackTrace();
        interpreter.close();
        return Version.getLatestVersionOnSystem();
    }

For the sake of completeness, I'm adding the Python code:

import urllib2 as urllib
import warnings

url = 'arcticlights.ca/api/paint&requests?=version'

def get_latest_version(link=url):
    request = urllib.Request(link)
    handler = urllib.urllopen(request)
    if handler.code is not 200:
        warnings.warn('Invalid Status Code', RuntimeWarning)
    return handler.read()

version = get_latest_version()

It works flawlessly, but only 10% of the time. If I run it with a main like follows:

public static void main(String[] args)
{
    for (int i = 0; i < 10; i++) {
        System.out.println(getVersion());
    }   
}

It works the first time. It gives me the output that I want, which is the data from the http request that is written in my Versions.py file, which the java code above calls. After the second time, it throws this massive error (which is 950 lines long, but of course, I won't torture you guys). Here's the gist of it:

Aug 26, 2015 10:41:21 PM org.python.netty.util.concurrent.DefaultPromise execute
SEVERE: Failed to submit a listener notification task. Event loop shut down?
java.util.concurrent.RejectedExecutionException: event executor terminated

My Python traceback that is supplied at the end of the 950 line Java stack trace is mostly this:

File "<string>", line 18, in get_latest_version 
urllib2.URLError: <urlopen error [Errno -1] Unmapped exception: java.util.concurrent.RejectedExecutionException: event executor terminated>

If anyone is curious, the seemingly offending line in my get_latest_version is just:

handler = urllib2.urlopen(request)

Since the server that the code is calling is being run (by cherrypy) on the localhost on my network, I can see how it is interacting with my server. It actually sends two requests (and throws the exception right after the second).

127.0.0.1 - - [26/Aug/2015:22:41:21] "GET / HTTP/1.1" 200 3 "" "Python-urllib/2.7"
127.0.0.1 - - [26/Aug/2015:22:41:21] "GET / HTTP/1.1" 200 3 "" "Python-urllib/2.7"

While I'm never going to run this code in a loop likely, I'm quite curious as to two things:

• Is the offending code my Python or Java code? Or could it just be an issue with Jython altogether?
• What does the exception mean (it looks like a java exception)? Why is it being thrown when it is? Is there a way to make a loop like this work? Could this be written better?

Answer 1

The python library urllib2, which you use, uses Netty.

Netty has a problem, which is widely known:

• Hopper: java.util.concurrent.RejectedExecutionException: event executor terminated
• Error recurrent : DefaultPromise Failed to notify a listener. Event loop shut down?
• Calling HttpClient.shutdown() causes problem to later created clients
• Shutting down netty 4 application throws RejectedExecutionException

According to all of these links Netty HttpClient fails from time to time after closing. It looks like Netty recovers after some time and some applications work just normally with this problem. Anyway it looks unstable.

---

Q: Is the offending code my Python or Java code? Or could it just be an issue with Jython altogether?

A: The problem is caused by Jython library urllib2, which uses Netty.

---

Q: What does the exception mean (it looks like a java exception)? Why is it being thrown when it is?

A: urllib2 uses internally Netty. Netty is written in Java and throws this Java exception. Netty uses its own Thread Executor, which is shut down and unusable for some time after closing a request. You hit exactly this time.

---

Q: Is there a way to make a loop like this work? Could this be written better?

A: I would try to use Requests library.

Answer 2

Try giving the interpreter a freshly initialized system state each time you create it:

PySystemState.initialize();
PythonInterpreter interpreter = new PythonInterpreter(null, new PySystemState());