12

In OpenJDK 8, it was possible to access sun.misc.VM and call isDirectMemoryPageAligned and maxDirectMemory.
isDirectMemoryPageAligned is used to size correctly the direct memory to allocate, as done by DirectByteBuffer.
maxDirectMemory is used to report memory statistics as well as access giving the value configured for -XX:MaxDirectMemorySize. Internally, it will set a limit to the allowed consumption of direct memory.

Since OpenJDK 9, the class VM has been moved to jdk.internal.misc and is not available unless --add-export java.base/jdk.internal.misc=xyz is used when running the application.

Is there a "right" way to do this ?

I already tried to use ManagementFactory.getMemoryMXBean().getNonHeapMemoryUsage().getMax() as a replacement for maxDirectMemory but it always returned -1 - meaning that the value was not available. It is also possible to access java.nio.Bits#MAX_MEMORY by reflection, but it remains "hackish".

Note that one could be very very dirty and do the following - working for OpenJDK, Zulu and Oracle 11.0.1 - but it is not the target goal of this question.

public static void tryExportInternal() {
    final String moduleName = "jdk.internal.misc";
    final Module javaLang = Integer.class.getModule();

    // Method used only for tests by the JDK...
    final Method exporter;
    try {
        exporter = Module.class.getDeclaredMethod("implAddExports", String.class);
        exporter.setAccessible(true);
        exporter.invoke(javaLang, moduleName);
    } catch (NoSuchMethodException | IllegalAccessException e) {
        LOG.log(Level.INFO, "Cannot access internal Module method", e);
    } catch (InvocationTargetException e) {
        LOG.log(Level.INFO, "Cannot call internal Module method", e);
    }
}

In the source code, implAddExports is marked as @apiNote This method is for JDK tests only. :(

Kineolyan
  • 723
  • 8
  • 24
  • Are you trying to use `isDirectMemoryPageAligned` to allocate memory aligned with page e.g. `DriectByteBuffer`? – Karol Dowbecki Nov 29 '18 at 16:18
  • This is exactly it. – Kineolyan Nov 29 '18 at 16:34
  • “Note that one could be very dirty and do the following” I doubt you could. Classes in a module are encapsulated (unless the package is opened), which means they’re private and cannot be manipulated. – VGR Nov 29 '18 at 18:06
  • 1
    Have you tested it @VGR ? If so, please report me the VM you use. I shall try to find why not. The trick here is to open the package with the dynamic equivalent of `--add-exports`, which is done using a private method of java.lang.Module and reflection. See [this code](https://github.com/AdoptOpenJDK/openjdk-jdk11/blob/999dbd4192d0f819cb5224f26e9e7fa75ca6f289/src/java.base/share/classes/java/lang/Module.java#L771-L778) – Kineolyan Nov 30 '18 at 10:32
  • 2
    The hack included in the question is horrible and could break at any time. Direct buffers have not been page aligned for several releases so not clear why anything needs this now. More info on what the value of -XX:MaxDirectMemorySize= is needed too. – Alan Bateman Nov 30 '18 at 15:07
  • Thanks @Alan Bateman for your input. I would be glad to abandon this, but I still saw it in [the Direct Buffer templates](https://github.com/AdoptOpenJDK/openjdk-jdk11/blob/999dbd4192d0f819cb5224f26e9e7fa75ca6f289/src/java.base/share/classes/java/nio/Direct-X-Buffer.java.template#L112-L118). Are you sure about your assertion ? I am updating the question to detail the use of the two methods – Kineolyan Nov 30 '18 at 16:03
  • 3
    Buffers have not been page aligned since JDK 6. There is no supported interface for what you are doing so best to start by explaining why you need this? – Alan Bateman Nov 30 '18 at 16:47
  • The goal is to allocate direct memory the same way ByteBuffer.allocateDirect does. Though this is playing with the fire because this is not an official feature of any JVM, many software editors like us - ActiveViam - do so. If there is no official API, we will wait for one to be created - like the Var Handles were - or rely on tricks. – Kineolyan Nov 30 '18 at 22:14
  • 1
    and why don't you just call ByteBuffer.allocateDirect? – Alan Bateman Dec 01 '18 at 08:31
  • We optionally use it but for large machine, we reach the limits of these classes and of `malloc` itself. That's why we even develop our own allocator, particularly NUMA friendly and adapted for enormous machines (1TB of RAM). See [one of our talks](https://www.youtube.com/watch?v=LppgqvKOUKs&t=11m41s) for more details. It seems from all your comments that I have my answer ? Is it ok if I summarize them in a answer to this question ? – Kineolyan Dec 03 '18 at 12:42
  • 1
    If you are allocating the memory yourself then you can use JNI's NewDirectByteBuffer to wrap the region as a direct buffer, maybe that is what you are looking for. – Alan Bateman Dec 03 '18 at 13:28
  • FWIW, it's questionable whether your Module class hack would work in Java 12+. See https://bugs.openjdk.java.net/browse/JDK-8214696 – Stefan Zobel Dec 12 '18 at 18:02
  • Of course. And I was even surprised that it works in the first place. It was only present for the sake of completeness in my question. FYI, I don't plan to use it at all – Kineolyan Dec 13 '18 at 07:51

1 Answers1

1

This answer comes from the various comments of Alan Bateman to the question.

No, there are no standard API to access the two wanted methods.

Since JDK 6, DirectxxxBuffers are not paged aligned anymore. Thus, accessing VM.isDirectMemoryPageAligned is not needed to reproduce what DirectBuffers do.

About manually memory allocation, being the use-case behind the question, the only API to do direct memory allocation is currently ByteBuffer.allocateDirect, or its JNI alternative NewDirectByteBuffer.

Comment references: 1 2 3

Kineolyan
  • 723
  • 8
  • 24