5

I have a set of images that stored in a 3D array of type Int16 , the number of images i cache can reach up to 600 images or more which affect the memory performance the garbage collection performance. I heard about memory tiling that can enhance the memory management as it sets the sub-array elements in a separate chunk.
- can I use memory tiling in my case?
- if yes, how can I use it?

Cœur
  • 37,241
  • 25
  • 195
  • 267
Sara S.
  • 1,365
  • 1
  • 15
  • 33
  • 1
    How is your array defined? `new short[n][][]` or `new short[x,y,z]`? In other words, is it really a multidimensional array or is it a jagged array? How do you know it affects the GC performance? – svick Jul 17 '11 at 14:23
  • 1
    600-ish objects are nothing. Grab a memory profiler and see what's really going on. – sisve Jul 17 '11 at 14:44
  • @svick my array is in the short[x,y,z] form, because i have searched alot for reason of the memory leak i have and i found that large objects effect the performance of GC and some people have to call GC.Collect in such case, so what's better short[x,y,z] or short [x][y][z]?,Thanks for your reply – Sara S. Jul 17 '11 at 17:15
  • @Simon they are medical images so the total needed memory may exceed 200MB in some cases. the problem that i cache all of them into one 3D array not as individual images – Sara S. Jul 17 '11 at 17:16
  • 1
    Array larger than about 85kb (implementation details in the gc) are stored in the large object heap (loh). The loh is collected (old objects removed) but not compacted ("defragmented"). Have you looked into memory-mapped [temporary] files instead? What type of cache? How are you using the images? – sisve Jul 17 '11 at 20:16
  • hmm, i dont understand what you mean by memory-mapped files instead. but My application needs to have the cached volume(3D array) in to the VM because i can generate images from these set of images all the time.i'm doing multi planar reconstruction – Sara S. Jul 18 '11 at 06:37

2 Answers2

3

can i use memory tiling in my case?

Yes. A 3d array of int16 [,,] has to be ONE block of memory. Put that into slices and they are smaller already. ([][,]).

The rest depends on your exact needs.

TomTom
  • 61,059
  • 10
  • 88
  • 148
  • so short [x][y,z] is better than [x,y,z] . and the best is short [x][y][z] is what i understand is right? – Sara S. Jul 17 '11 at 17:20
  • 2
    @Sara, it's far from that simple. What form is better depends on what exactly you are doing. – svick Jul 17 '11 at 17:32
  • Yes. It is b etter for using memory as it ois not ONE BIG BLOCK (every array is one), but the optimum size depends on your exact needs. It is complicated. I use, for example, for buffers up to 4mb pages that get recycled (there is an unused queue). – TomTom Jul 17 '11 at 18:15
  • I think one should look at this question too to know the expense of saving memory in case of 3D :http://stackoverflow.com/questions/597720/what-is-differences-between-multidimensional-array-and-array-of-arrays-in-c – Sara S. Jul 18 '11 at 07:00
1

Garbage collection will benefit from having as few things to collect as possible. A multi-dimensional array will be treated as a single allocation:

short[,,] a = new short[2,3,4];

A jagged array will use many different allocations. Its smaller allocation sizes might be much faster to allocate, but it might put more strain on the GC during collection:

short[][][] a = new short[2][][];

I've never heard of "memory tiling" before so if you can explain what that is, I might be able to answer your question better.

Cory Nelson
  • 29,236
  • 5
  • 72
  • 110
  • Many computer applications require the manipulation of large data arrays. These applications can behave badly under a paged virtual memory (VM) system, due to poor memory access patterns. one solution to this problem is tiling, a technique in which the arrays are partitioned into sub-arrays that map one-to-one with VM pages. ==>copied from this pdf :http://www.usenix.org/publications/library/proceedings/sa92/franklin.pdf – Sara S. Jul 17 '11 at 17:26