sand boxing threads without separate processes

Question

In the interest of ease of programming (local function calls instead of IPC) and performance (e.g. avoiding copies of large buffers), I'd like to have a Java VM call native code using JNI instead of through interprocess communication. There would be lots of worker threads, each doing computer vision on some image and sending back a list of detected features.

I've found a few other posts about this topic:

1. How to implement a native code sandbox?

2. Linux: Is it possible to sandbox shared library code

but in all cases, the agreed upon solution is to use multiple processes.

But I would like to explore the feasibility of partly sand boxing threads. Clearly, this goes against common sense, but I think if your client processes aren't malicious and if you can recover from faults, and in the worst case, are willing to tolerate a whole system crash once in a blue moon, it might work.

There are some hints that this is possible such as from jmajnert in #2. You would have to capture segfaults and other crashes, and terminate and restart the crashed thread. But I also want to reset the heap of the thread. That means each thread should have a private heap, but I don't know of any common malloc implementation that lets you create multiple heaps (AIX seems to).

Then I would want to close all files opened by the thread when it gets restarted.

Also, if Java objects get compromised by the native code, would it be practical to provide some fault tolerance like recreating them?

Answer 1

Because if complexity of hopping models between some native code, and the JVM -- The idea itself is really not even feasible.

To be feasible, you'd need to be within a single machine/threading model.

Lets assume you're in posix/ansi c.

You'd need to write a custom allocator that allocated from pools. Each time you launched a thread you'd allocate a new pool and set that pool as a thread local variable that all your custom_malloc() functions would allocate from. This way, when your thread died you could crush all of it's memory along with it.

Next, you'll need to set up some niftyness with setjmp/longjmp and signal to catch all those segfaults etc. exit the thread, crush it's memory and restart.

If you have objects from the "parent process" that you don't want to get corrupted, you'd have to create some custom mutexes that would have rollback functions that could be triggered when a threads signal handler was triggered to destroy the thread.