Using sun.misc.Unsafe to get address of Java array items?

Question

I'm struggling to understand the documentation of sun.misc.Unsafe -- I guess as it's not intended for general use, nobody's really bothered with making it readable -- but I actually really need a way to find the address of an element in an array (so that I can pass a pointer to it to native code). Has anyone got any working code that does this? Is it reliable?

http://stackoverflow.com/questions/5574241/interesting-uses-of-sun-misc-unsafe — Wojciech Owczarczyk, Oct 28 '11 at 20:52
[Why Developers Should Not Write Programs That Call 'sun' Packages](http://www.oracle.com/technetwork/java/faq-sun-packages-142232.html) — BalusC, Oct 28 '11 at 20:58
imo, unsafe doc is pretty good, it's not intended for general public anyways. if you need some guidelines, start reading java.util.concurrent and java.util.concurrent.atomic... unsafe is pretty close to C (or assembler, if you prefer). If you have no experience if any, unsafe is not for you. How to get disassemble of your java code: http://wikis.sun.com/display/HotSpotInternals/PrintAssembly — bestsss, Oct 28 '11 at 21:27
@BalusC, unsafe is good and proper if you know what you do (i.e. not asking question on SO about it :) ) and it's probably more portable than JNI. — bestsss, Oct 28 '11 at 21:27
@Jules, to answer your question look at the source of java.util.concurrent.atomic.AtomicReferenceArray (for instance: http://grepcode.com/file/repository.grepcode.com/java/root/jdk/openjdk/6-b14/java/util/concurrent/atomic/AtomicReferenceArray.java ) — bestsss, Oct 28 '11 at 21:31
@bestsss - so do you know a reliable way to stop the GC from moving an object while you are using the pointer returned by `Unsafe`? — Stephen C, Oct 29 '11 at 04:04
@StephenC, what do mean? do you have any method in mind, unsafe is just intrinsics that are compiled (usually) to few assembler instructions (minus the class loading). Unsafe returns no pointers, besides the likes of `allocateMemory` which are not affected by the GC. If you mean how to keep a point to smth like byte[] in JNI `GetByteArrayElements() / ReleaseByteArrayElements()`, or copy them via GetByteArrayRegion. As far as I know by now: GetPrimitiveArrayCritical + ReleasePrimitiveArrayCritical are discouraged to use. Yet, w/ unsafe and directbuffer you can have native C memory+pointers to. — bestsss, Oct 29 '11 at 09:42
@StephenC - you're worrying about the possibility of something happening that simply doesn't happen. AFAICT, JVMs do not track pointers to objects as they move between storage locations in JIT-compiled code. For that reason, it turns out that the collector *must treat any value stored on the stack that looks like a reference to an object* as if it actually *were* a reference to an object (see https://stackoverflow.com/a/12097214/441899 for a description of how it does this). This means that objects *won't* move while you're using the pointers, and you don't need to do anything to prevent it. — Jules, Aug 08 '17 at 23:43
@Jules It can't "not move it". That simply doesn't make sense. The Java GCs are all generational (copying) collectors. When a non-garbage is found it is copied to a different space, and then the old space is erased. Objects that were "not moved" would end up as scorched earth. I think you are confusing the behavior of the Java GC with the behavior of a Conservative GC. — Stephen C, Aug 09 '17 at 00:14
And the question that you linked to is not describing the way that HotSpot collectors work. It seems to be talking about the behavior of something else. It doesn't help that its reference is a broken link. Sure, yes, you can implement a GC that way (either by using explicit tag bits, or by making conservative assumptions). However that is not how the HotSpot GCs and JVMs work. — Stephen C, Aug 09 '17 at 00:18
It maybe that we are confusing things here. Methods like JNI `allocateMemory` are actually allocating memory *outside if the heap*. That's why they don't move. But what @Jules is asking for is a way to get the address of an object in the heap. That's different. I believe that you can do that with Unsafe ... but it is unsafe! — Stephen C, Aug 09 '17 at 00:29
@StephenC There _is_ a way to prevent GC from moving objects around temporarily. It's not very pretty. You can call native code which uses GetPrimitiveArrayCritical / ReleasePrimitiveArrayCritical. In between calls to these funtions, GC is suspended (since the point of GetPrimitiveArrayCritical is to access heap memory shared with native code). You'd need to set a flag in the array (which could just be a byte[1]) when you wanted the native code to release the array and allow GC to resume. The native code would just spin/yield while waiting for the flag. — barneypitt, Oct 12 '22 at 11:04
Suspending GC for any length of time is nasty because you may end up blocking other threads. Indeed ... consider happens if the thread that has locked an object in memory then gets preempted by the OS. Unsafe! — Stephen C, Oct 12 '22 at 12:40

Wojciech Owczarczyk · Answer 1 · 2011-10-28T21:06:48.093

8

Here is a working sample. Please be careful however as you may easily crash your JVM with unappropriate usage of Unsafe class.

import java.lang.reflect.Field;

import sun.misc.Unsafe;

public class UnsafeTest {

    public static void main(String... args) {
        Unsafe unsafe = null;

        try {
            Field field = sun.misc.Unsafe.class.getDeclaredField("theUnsafe");
            field.setAccessible(true);
            unsafe = (sun.misc.Unsafe) field.get(null);
        } catch (Exception e) {
            throw new AssertionError(e);
        }

        int ten = 10;
        byte size = 1;
        long mem = unsafe.allocateMemory(size);
        unsafe.putAddress(mem, ten);
        long readValue = unsafe.getAddress(mem);
        System.out.println("Val: " + readValue);

    }
}

edited Oct 28 '11 at 21:06

answered Oct 28 '11 at 20:56

Wojciech Owczarczyk

5,595
2
33
55

2

That code must be perfectly safe, as it is exactly how DirectByteBuffer works. Unfortunately, it doesn't do what I want, which is to access the data in an existing array. – Jules Oct 29 '11 at 09:29
4

@StephenC, the code is perfectly fine, it's basically `char* mem = malloc(size);...` it's never touched by the GC and causes native C leak unless realesed. – bestsss Oct 29 '11 at 09:52
2

The code is fine, but you also need to consider freeing the allocated memory. See DirectByteBuffer for one way of doing it. Also consider that allocated memory is not zeroed, so you can not assume your direct array is zero initialized. – Nitsan Wakart Jan 09 '13 at 13:47

score 8 · Accepted Answer · answered Oct 28 '11 at 22:32

8

Instead of using an array you can use a ByteBuffer.allocateDirect() direct buffer. This has the address in a field and this address doesn't change for the life of the ByteBuffer. A direct ByteBuffer uses minimal heap space. You can get the address using reflection.

You can use Unsafe to get an address, the problem is that the GC can move it at any time. Objects are not fixed in memory.

In JNI you can use special methods to copy data to/from Java objects to avoid this issue (and others) I suggest you use these if you want to exchange data between Objects with C code.

answered Oct 28 '11 at 22:32

Peter Lawrey

525,659
79
751
1,130

1

I'd rather not use a direct byte buffer (which would make solving the problem rather easy) as I am stuck trying to implement an existing API which is defined in terms of byte arrays, and am trying to avoid the penalty of copying to and from an extra set of buffers. This would suffice as a last resort, but there must be some better way. Working with JNI would make this easier, but unfortunately I am working with JNA, which doesn't seem to have the interfaces necessary to do anything other than work with entire arrays. – Jules Oct 29 '11 at 09:27
@Jules, JNI will cause a copy on its own, unless you wish to go w/ `GetPrimitiveArrayCritical` which might hurt the GC. Bite the bullet and copy into direct (buffer) memory. That's the only feasible solution. For instance the impl. FileOutputStream (SocketOutputStream extends it) uses copy of the elements on the stack. Penalty to copy is not so high, since the highest cost comes w/ the load cost of the data (and cache misses, even) which you'd have to pay either way. The copying will also cause to prefetch the cachelines, so it's might be even better depeding how the native code works. – bestsss Oct 29 '11 at 09:59
@Jules, on a side note: even javax.net.ssl.SSLEngine allows use of buffers, all the algorithms are byte[] and they end up copying direct buffers into temporary arrays, this is the reverse story and the moral of it: do not use direct buffers w/ SSLEngine. – bestsss Oct 29 '11 at 10:03

score 0 · Answer 3 · answered Oct 29 '11 at 01:00

0

Why? There are plenty of facilities in JNI for dealing with the contents of Java arrays. You don't need to use undocumented internal Sun classes that mightn't be there next week.

answered Oct 29 '11 at 01:00

user207421

305,947
44
307
483

I'm using JNA, rather than JNI, and the function I'm interfacing with needs to have a pointer passed to the middle of an array, whereas JNA only appears to be able to produce a pointer to the start of an array. – Jules Oct 29 '11 at 09:17
@Jules, do not mix JNA and normal arrays, use direct buffers. – bestsss Oct 29 '11 at 09:46

Using sun.misc.Unsafe to get address of Java array items?

3 Answers3