1

I'm writting a Windows Java application that needs to call unsafe native code, and I need to prevent this code from having access to Java objects and JVM data structures, otherwise it might crash the JVM or hack into sensitive data. Before you ask, this native code is previously verified - it can only call a few APIs and cannot have certain instructions, so we know it won't VirtualProtect itself or other memory regions to gain more access and mess around.

Anyway, my first attempt was to wrap this code into a separate process (sandbox) and use IPC to talk with Java. There's a JNI DLL that does IPC stuff on Java side. Basically, every time we need to run unsafe native code, our Java app calls a JNI function that wakes up the sandbox using an auto-reset Windows Event, and then awaits completion. The sandbox runs unsafe native code and wakes up the JVM using another auto-reset Windows Event, and life continues. It would be perfect if it weren't so slow.

The problem is that unsafe native code can contain some functions that perform very quick calculations and can be called millions of times from Java, hence the call overhead should be minimum. But this overhead is huge because JVM wakes-up sandbox with a Windows Event, and vice-versa when the sandbox returns. This process is 8x the time of an in-process (non-IPC) solution, where unsafe native code is wrapped in JNI DLL (and hence the call happens in the same thread, in the same time slice).

My first guess is that when JVM wakes-up the sandbox, Windows only puts the sandbox thread on the ready set, so it runs only after some milliseconds. And the same happens when the sandbox returns. Not to count for two (possibly expensive) context switches.

Microsoft documentation here says the following:

If a higher-priority thread becomes available to run, the system ceases to execute the lower-priority thread (without allowing it to finish using its time slice), and assigns a full time slice to the higher-priority thread.

To test this theory, I assigned THREAD_PRIORITY_TIME_CRITICAL to the sandbox thread. There was some gains. Performance went from 8x to 5x the time of the in-process (non-IPC) solution. But I need more, otherwise this Java app might not get a change to go into production!

You can help me in two ways:

  • Tell me if there's a faster method to wake-up another process, such as forcing a context switch or performing an inter-process procedure call.

  • Tell me how can I protect JVM while running unsafe native code in-process. I heard that Google Native Client does this, but I only found this documentation. If you know more, please provide links to more detailed information about how this is implemented.

fernacolo
  • 7,012
  • 5
  • 40
  • 61
  • Have you tracked down if the performance problem is the windows event or your IPC? Typically I'd expect the IPC to be the limiting factor. – Joel Lucsy Jul 05 '12 at 15:57
  • Yes, I have isolated. The IPC uses shared memory to tell the sandbox which native function to execute and its arguments. The JNA DLL writes to this memory, and the sandbox just reads, calls unsafe code and writes the result. Unless arguments are big buffers, these memory read/write operations causes less than 10% of overhead in the in-process solution. So it's not the IPC. The slowness is definitely related either to windows event or context switching. – fernacolo Jul 05 '12 at 16:58
  • I'm also curious as to what threats are you trying to protect against? Obviously running in a second process is the best protection, but it might help to know if there are specific things you want to target. For example, are you worried about it writing into other areas of memory, catching exceptions, or just a general protection of anything unforeseen? – Joel Lucsy Jul 05 '12 at 18:22
  • The native code must not access JVM memory (for reading or writing). Ideally, it should not perform an infinite loop or infinite wait. Using another process, we can kill it based on answer timeout, but that's not a strong requirement. But access to JVM memory must be forbidden. – fernacolo Jul 05 '12 at 18:30
  • Also, the native code must not call dangerous APIs such as CreateThread, CreateProcess, LoadLibrary, VirtualProtect, GetProcAddress, etc. But we can easely prevent it with code verification prior invocation. – fernacolo Jul 05 '12 at 18:31
  • Do you need to do the calls one after the other, or could you batch them to reduce overhead? – Joel Lucsy Jul 05 '12 at 18:34
  • There's no way we can batch. The calls are issued by Java classes in an _ad hoc_ manner. – fernacolo Jul 05 '12 at 18:36

1 Answers1

0

I solved the problem by performing JVM-sandbox interactions using spinlock over a shared memory variable accessed from JVM through file mapping. This question explains how to implement in a C++ environment. Porting to Java is easy with MappedByteBuffer.

Community
  • 1
  • 1
fernacolo
  • 7,012
  • 5
  • 40
  • 61