1

The circumstances changed a lot for JDK9 so I was wondering a possible way it is to unmap a memory mapped file under Java 9 (JDK9).

One of the possible ways for <JDK9 is here

Karussell
  • 17,085
  • 16
  • 97
  • 197
  • Please explain your downvotes. It is recommended to post answers to your own questions if you think they are worth the effort. – Karussell Jun 29 '17 at 19:30
  • 2
    I did not downvote but, "bestness" is not a scientific/objective measure, a better question would be "how to unmap an mmapped file in Java?", if such a question already exists, you could post your answer there, with a comment saying it's for Java 9. – Jorn Vernee Jun 29 '17 at 22:49
  • Ok, edited to just 'possible' and added the link – Karussell Jun 30 '17 at 06:58

1 Answers1

3

Currently no special compiler parameter is necessary if you use something like:

public static void cleanMappedByteBuffer(final ByteBuffer buffer) {
 try {
   AccessController.doPrivileged(new PrivilegedExceptionAction<Object>() {
      @Override
      public Object run() throws Exception {
           final Class<?> unsafeClass = Class.forName("sun.misc.Unsafe");
           // we do not need to check for a specific class, we can call the Unsafe method with any buffer class
           MethodHandle unmapper = MethodHandles.lookup().findVirtual(unsafeClass, "invokeCleaner",
               MethodType.methodType(void.class, ByteBuffer.class));
           // fetch the unsafe instance and bind it to the virtual MethodHandle
           final Field f = unsafeClass.getDeclaredField("theUnsafe");
           f.setAccessible(true);
           final Object theUnsafe = f.get(null);
           try {
               unmapper.bindTo(theUnsafe).invokeExact(buffer);
               return null;
           } catch (Throwable t) {
               throw new RuntimeException(t);
           }
      }      
   });
 } catch (PrivilegedActionException e) {
   throw new RuntimeException("Unable to unmap the mapped buffer", e);
 }
}

Note: "this can be dangerous unless you know everything about what is going on in the system". See comments.

Taken from GraphHopper that was inspired from Lucene its code. Furthmore it has code for Android too and has working solutions for older JDK9 versions where different compiler switches were necessary (see git history).

If you call this often better have a look into the Lucene code (which is a bit more tricky) which creates an 'uncleaner' once and just calls clean for the unmap.

Karussell
  • 17,085
  • 16
  • 97
  • 197
  • 2
    This will indeed unmap the buffer, but you should include a warning that this can be dangerous unless you know everything about what is going on in the system. In particular, if any threads still attempt to access the now-unmapped buffer, they will most likely crash the entire JVM. – Stuart Marks Jun 30 '17 at 01:47
  • 1
    the whole initialization of the method handle should be moved out of that block into a static initializer so you only have to invoke the MH. otherwise this method will be far more costly than necessary. from the linked source: `// TODO avoid reflection on every call` – the8472 Jun 30 '17 at 22:08
  • 1
    @the8472 If you're concerned about security -- and you should be -- then you should be careful about caching a method handle somewhere such that any caller can invoke it. This is because the security check is done at lookup time, not invocation time. – Stuart Marks Jul 01 '17 at 18:11
  • 5
    By the way, the history behind why there is no public `MappedByteBuffer.unmap()` method is in [JDK-4724038](https://bugs.openjdk.java.net/browse/JDK-4724038). Since threads accessing the buffer after it's been unmapped cause a crash or data corruption, unmapping is done by GC, only after the buffer has become unreachable. The obvious solution of guarding every access imposes an unacceptable performance hit. Some ways of avoiding the performance hit are being investigated, but nothing's implemented yet. – Stuart Marks Jul 01 '17 at 18:24
  • @Stuart Marks: since anyone can call that public `cleanMappedByteBuffer` anyway, what additional security risk could caching the `MethodHandle` impose? This method even uses `doPrivileged` to ensure that every caller can perform this action. The specified reason not to provide an `unmap` method makes me wonder… [since objects can get garbage collected while still being in use](https://stackoverflow.com/a/26645534/2711488) (in the absence of thread synchronization), how is premature unmapping by the cleaner prevented without guarding? To me it sounds like special JVM support is needed anyway… – Holger Jul 04 '17 at 10:02
  • @Holger The `cleanMappedByteBuffer` method in this answer might very well be insecure. But one could easily see that since this has a `doPrivileged` call in it, that some checking of the caller or the context could be added to make it secure (or at least more secure). The danger with caching a method handle is that there are now two places -- lookup and use -- where security analysis needs to be done. One might easily refactor this to cache the method handle and forget to do security checking at the point of use. That's what I was warning about. – Stuart Marks Jul 05 '17 at 21:43
  • @Holger Regarding GC, in the other answer the object was "in use" because a method was active on it even though it wasn't reachable. It was only a problem in that answer because the object's finalizer had side effects that were visible on objects that were reachable from the running method. For `MappedByteBuffer` most of the methods end with `return this` so the buffer always remains reachable at the end of the method and cannot be GCed during an operation. An alternative that can be used where `return this` isn't feasible is `Reference.reachabilityFence`, new in 9. – Stuart Marks Jul 05 '17 at 21:51
  • @Stuart Marks: assuming that `return this` makes a difference is a contradiction to your other answer. The point is, after optimization only actual use counts, so the `MappedByteBuffer`’s methods ending in `return this;` is irrelevant if the caller doesn’t use that reference. The last actual use is the read of that `long` field before performing the native/intrinsic operation that only uses the non-heap memory addressed by the `long` value. Right between these two operations, the `MappedByteBuffer` instance could get collected as it is not touched by subsequent operations. – Holger Jul 06 '17 at 05:58
  • 1
    @Holger Whoops, yes, you're right, if `return this` occurs in a method that's inlined into the caller that ignores the return value, then `this` might become unreachable earlier than the `return this`. A buffer might indeed become unreachable after the last access to its `long address` field and before the code that accesses memory at that address. However, in practice the buffer can't be GCed, since there's (probably) no GC safepoint at this point, so the unreachability can't be detected. However, this is probably a place where `reachabilityFence` or similar is strictly necessary. – Stuart Marks Jul 07 '17 at 21:30