5

We have a product that runs currently on a 32 bit 1.6 JRE. We're using Berkeley DB which consumes about 2.5 GB RAM of the 4 GB address space. That leaves us about 750 MB of memory for the JVM address space.

We're currently hitting OutOfMemory issues with the current setup. It would be nice to increase our JVM heap size to 1.5 GB while still retaining Berkeley DB's 2.5 GB space. Is there any way to access more than 4 GB RAM/heap in a 32-bit JVM ? I've considering the following solutions
1) use a JVM which does better GC -- this will give me marginal results -- I could get about 50-100 MB working memory
2) something like memcached or "out of process ehcache " -- this can get me as much as the hardware allows with the overhead of IPC/serialization.

Are there other solutions to increasing an application's addressable memory ?

The solution should work on solaris 10 running sparc.

*UPDATE : Because of using native shared libraries, right now, we're unable to switch to a 64-bit JVM even though the OS is 64-bit *

thank you,

trincot
  • 317,000
  • 35
  • 244
  • 286
anjanb
  • 12,999
  • 18
  • 77
  • 106
  • 1
    The OS is solaris 64-bit with lots of GB RAM. It's only the java process that needs more RAM/heap. – anjanb Nov 11 '11 at 10:37
  • anjanb, can you say exact version of BerkeleyDB? How so much memory was reserved to BerkeleyDB? – osgx Nov 11 '11 at 18:03

3 Answers3

5

Are there other solutions to increasing an application's addressable memory ?

  1. Split single application in several processes with non-shared memory (not a threads; but processes). First process can run DB and second can run other part of the project. You can use RMI or shared memory or sockets to communicate between processes.
  2. Lower memory, reserved for OS. E.g. on x86-32 PAE allows OS to reserve a smaller than 1 GB of 4 GB virtual address space. (e.g "4GB / 4Gb split", supported on Oracle Linux)
  3. Put some data to the disk. The disk can be a RAM-disk to better speed; or it can be real disk and OS will speed up access to files using "page cache".

Also, every real SPARC (not ancient SuperSparc or poor-man LION) is 64-bit really. So, it can be easier to switch to 64-bit version of OS. I don't know about Solaris, but in linux there is possible to run 32-bit applications on top of 64-bit OS. And 64bit OS will allow you to run 64-bit JVM.

UPDATE: There are ramdisks in Solaris http://wikis.sun.com/display/BigAdmin/Talking+about+RAM+disks+in+the+Solaris+OS and I think you should try them for storing database (or temporary files of DB). There will be no extra serialization/IPC like in case (1); only extra read/write or mmap/munmap. But Ramdisk is order faster than SSD and 3-4 orders faster than HDD.

osgx
  • 90,338
  • 53
  • 357
  • 513
  • Because of using native shared libraries, right now, we're unable to switch to a 64-bit JVM even though the OS is 64-bit – anjanb Nov 10 '11 at 16:20
  • 1
    Can you move native libs to separated process? Did you try to run 32bit JVM on 64-bit OS (this can help as not requiring 2/2 or 3/1 GB split)? – osgx Nov 10 '11 at 16:31
  • There is a quick test to check Memory splitting scheme: http://stackoverflow.com/questions/987219/max-amount-of-memory-per-java-process-in-windows/987576#987576 - you can run it in your current environment and in 64-bitOS + 32bit JVM combination – osgx Nov 10 '11 at 16:32
  • 1
    If he's already getting 3.2gb of the address space he already uses your 2nd point on whatever OS he's running (windows has something similar too). Although I don't see why a 64bit OS wouldn't give him the whole 32bit address space in this case - strange ([at least windows should make the whole 4gb available in that case](http://blogs.msdn.com/b/oldnewthing/archive/2005/06/01/423817.aspx)). – Voo Nov 10 '11 at 17:04
  • we are running the 32-bit JVM on a 64-bit OS -- solaris – anjanb Nov 11 '11 at 10:38
  • In our production environment, we can get more than 3 GB heap if we want to. But the point is, since the Berkeley DB takes up 2.5 of the 4 GB address space, the JVM itself needs some space to manage all the resources, the Java heap is limited to about 750 MB. – anjanb Nov 11 '11 at 10:45
  • So, if you have no more available virtual memory, you can't get more GBs for heap and the only you can do is to move some data to other place: Other process or Ram disk or smth like AWE (this is like HighMem for memory > 4 GB in 32 bit windows processes) – osgx Nov 11 '11 at 10:50
  • Yes, we're using a huge "page cache" so that the DB is mostly in RAM instead of slow disk seeks. – anjanb Nov 11 '11 at 10:51
  • "or smth like AWE (this is like HighMem for memory > 4 GB in 32 bit windows processes)" -- can u elaborate on this ? We use Solaris. – anjanb Nov 11 '11 at 10:53
  • "we're using a huge "page cache" so that the DB is mostly in RAM"... Hmm. Is in in RAM of OS or in Virtual Memory of the process itself? AWE is strange technology from Microsoft http://en.wikipedia.org/wiki/Address_Windowing_Extensions which allow SQL server to use >4GB via window (server must ask OS to remap some parts of >4GB into <4GB address space). – osgx Nov 11 '11 at 12:42
  • The VLM is like AWE for linux: http://download.oracle.com/docs/cd/B28359_01/server.111/b32009/appi_vlm.htm – osgx Nov 11 '11 at 12:43
3

32-bit programs are incapable of handling more than 4GB of memory addresses. They just don't have enough bits to represent more memory.

2^32 = 4 294 967 296

Your best bet would be to upgrade to a 64-bit JRE.

StriplingWarrior
  • 151,543
  • 27
  • 246
  • 315
  • Because of using native shared libraries, right now, we're unable to switch to a 64-bit JVM even though the OS is 64-bit – anjanb Nov 10 '11 at 16:20
  • @anjanb: In that case, you probably have to split the application into multiple processes, as osgx recommends. – StriplingWarrior Nov 10 '11 at 16:38
3

I suggest you run your 32-bit shared native libraries in a 32-bit JVM and run everything else in a 64-bit JVM. You can have the 64-bit JVM call the 32-bit JVM to do whatever it does. I assume the bulk of your data/memory requirement can be moved to a 64-bit JVM.

Peter Lawrey
  • 525,659
  • 79
  • 751
  • 1,130
  • that is something we will have to experiment and see if it makes sense for the throughput that is expected of the system. – anjanb Nov 11 '11 at 10:39
  • You should be able to achieve a throughput of 500 MB/s to 3000 MB/s between two processes on the same machine (depending on your hardware). – Peter Lawrey Nov 11 '11 at 10:43
  • can u suggest the best approach for a 64-bit Java app to talk to 32-bit java app. I'm aware of the usual unix/linux IPC mechanisms but which of these work best in 64-bit java and 32-bit java scenario ? – anjanb Nov 11 '11 at 11:01
  • 1
    The bit-ness of the JVM shouldn't matter. I would use a plain Socket using blocking IO or NIO might be the simplest. You can use DataOutput/InputStream with IO or ByteBuffers with NIO. You can send millions of messages per second with a single TCP connection with round trip latencies below 20 micro-seconds. – Peter Lawrey Nov 11 '11 at 11:06