3

i just came to the point where whether google nor my knowledge bring me forwards.

Think about the following situation: I read in a lot (up to millions) of large objects (up to 500mb each) and sometimes i read in millions of objects with only 500kb, this completely depends on the user of my software. Each object is gonna be processed in a pipeline so they don't need to be all in the memory for all the time, only a reference would be needed to find the objects again on my harddisk after serializing it so that i can deserialize it again. So it's something like a persistent cache for large objects.

so here come my questions:

  1. Is there a solution (any framework) which does exactly what i need? this includes: arbitrary serialization of large objects after determining somehow if the cache is full?

  2. if there isn't: is there a way to somehow intelligent check weather an object should be serialized or not? e.g. checking somehow the memory size? Or something like a listener on a softreference (when it get's released?).

Thanks alot,

Christian

AZ_
  • 21,688
  • 25
  • 143
  • 191
user984161
  • 31
  • 2

2 Answers2

0

Storing millions of objects sounds like a database application.

michael667
  • 3,241
  • 24
  • 32
  • Yes of course. but at somepoint i need to use a lot of objects at the same time in my "software" to compute them somehow together in a batch. these objects are images btw. ;) – user984161 Oct 07 '11 at 14:39
  • I assume you already have a data structure for your images that contains filename etc., but not the image data itself. Why not store the files (image data) on the disk/database and retrieve them when needed? – michael667 Oct 07 '11 at 14:42
  • Actually I have both. Image data and structure. But you are right, this is exactly my questions: how to determine weather i need to serialize an object or not (according to the free memory). Are there intelligent ways? e.g. using a softreference would mean, that i always need to serialize an object when i create or change it as i don't know when java kicks it off the heap. or can i listen to that somehow? – user984161 Oct 07 '11 at 14:48
0

Check out Oracle Coherence. There are a few open source alternatives as well but, Coherence is the most feature rich.

EDIT: Coherence Alternatives

Community
  • 1
  • 1
RHT
  • 4,974
  • 3
  • 26
  • 32
  • Thanks for the hint. Could you name a opensource project? actually the software we provide should be opensource. – user984161 Oct 07 '11 at 14:49