5

Is it possible to perform the memory management by yourself. e.g. We allocate a chunk of memory outside the heap space so that it is not subject to GC. And we ourselves take care of allocation/deallocation of objects from this chunk of memory.

Some people have pointed to frameworks like Jmalloc/EHcache. Actually i more want to understand that how they actually do it.

I am fine with some direct approach or even some indirect approach ( e.g. first serialize java objects).

Peter Lawrey
  • 525,659
  • 79
  • 751
  • 1,130
MoveFast
  • 3,011
  • 2
  • 27
  • 53
  • 2
    Why do you want to do this? Just curious. – Fabian Barney Apr 02 '12 at 14:47
  • The [Unsafe](http://www.docjar.com/docs/api/sun/misc/Unsafe.html) class can possibly help you doing weird stuff... – assylias Apr 02 '12 at 14:47
  • See also: http://stackoverflow.com/questions/5574241/interesting-uses-of-sun-misc-unsafe – assylias Apr 02 '12 at 14:48
  • I have a use-case where I know a lot about the life time of items which I want to store. So I want to try out their Memory management out own my own..outside heap and GCs – MoveFast Apr 02 '12 at 14:49
  • See also: http://stackoverflow.com/questions/9050852/application-to-manage-its-own-virtual-memory/9057485#9057485 – Andrew Apr 02 '12 at 18:33

4 Answers4

5

You can not allocate Java objects in a foreign memory location, but you can map memory which is e.g. allocated in a native library into a direct ByteBuffer to use it from Java code.

Neet
  • 3,937
  • 15
  • 18
3

You can use the off the heap memory approach Look for example jmalloc

and this is also usefull link Difference between on and off the heap

Community
  • 1
  • 1
Julias
  • 5,752
  • 17
  • 59
  • 84
  • I more want to understand that how frameworks like jmalloc and EHCache for example are doing it. – MoveFast Apr 02 '12 at 15:01
  • @MonojGumber: They underlying mechanism ist most likely not written in Java, but in native code. The library then provides a Java interface. – Niklas B. Apr 02 '12 at 15:10
  • It can be read on the jmalloc project page that they are using serialization. This is a nice idea to store large objects, but not really suitable for performance. – Neet Apr 02 '12 at 15:32
3

I have a library which does this sort of thing. You create excerpts which can be used a rewritable objects or queued events. It keeps track of where objects start and end. The downside is that the library assumes you will cycle all the object once per day or week. i.e. there is no clean up as such. On the plus side its very efficient, can be used in an event driven manner, persisted and can be shared between processes. You can have hundreds of GB of data while using only a few MB of heap.

https://github.com/peter-lawrey/Java-Chronicle

BTW: It supports ByteBuffers or using Unsafe for extra performance.

Peter Lawrey
  • 525,659
  • 79
  • 751
  • 1,130
2

If you mean Java objects, then no, this isn't possible with standard VMs. Although you can always modify the VM if you want to experiment (Jikes RVM for example was made for this very purpose), but bear in mind that the result won't really be Java any more.

As for memory allocation for non-java data structures, that is possible and is being done regularly by native libraries and there is even some Java support for it (mentioned in the other answers), with the general caveat that you can very easily self-destruct with it.

biziclop
  • 48,926
  • 12
  • 77
  • 104
  • can you please give more details on allocation for non-java data structures. I am ok to convert my Objects to a serialized/byte buffer form. Will some java solution work for that? – MoveFast Apr 02 '12 at 14:56
  • @ManojGumber The other two answers contain hints about this: using direct ByteBuffers is one way or you can use a high-level library like jmalloc or BigMemory. But you should really only be using this in production if GC is your main bottleneck. – biziclop Apr 02 '12 at 15:06