2

A customer is complaining that my program is using too much memory. However, after working with them for a while, I've realised that:

  1. They've turned off their page file (on their terminal services box).
  2. They're worried about the size of the "private working set" figure in task manager for my program.

So, my question is, if I just trim the size of the working set with EmptyWorkingSet() after my program has started up (it uses lots of memory during XML parsing but then deletes it, but the working set doesn't seem to go down) I can make the working set figure go right down. However, will this actually help the customer? I have a feeling this just means that the working set will be paged and I believe if you have the page file turned off, the working set is backed by real memory anyway....

Is it true to say that what task manager reports as "private working set" is really how much my program has new/malloced?

Benj
  • 31,668
  • 17
  • 78
  • 127
  • 3
    They are idiots for turning off the page file. – nobody Feb 04 '11 at 14:38
  • 1
    @Andrew: ... maybe they're from a linux background, where it's actually plausible that you can run a worthwhile program in 2GB RAM without hitting swap ;-p – Steve Jessop Feb 04 '11 at 14:57
  • Yes I totally agree, unfortunately management insist that I fix it rather than tell the customer to turn their page file on.... :-( – Benj Feb 04 '11 at 15:44

3 Answers3

3

At least in a sense of standard terminology, "private working set" is the amount of memory your program has mapped that is not backed by files (the program executable, dlls, or manually memory-mapped files) on disk or other shared resources. If swap (paging) were enabled, it's the amount of swap space your program would occupy if it were entirely swapped out of memory.

I would agree with your management that you need to fix your bloated program. Turning off swap is a very sane decision for a customer with low-latency requirements. If your program is using 2GB of memory, perhaps you need to rethink whatever libraries you're using to represent XML data in memory.

R.. GitHub STOP HELPING ICE
  • 208,859
  • 35
  • 376
  • 711
  • 2Gb! It's not that bad ;-) Or perhaps it is, it's using about 100Mb per session, but sadly they have 100 sessions, so erm...10Gb! Gulp.. – Benj Feb 05 '11 at 11:34
  • Is that working set perhaps largely the same data for each session? If so, you might consider moving the storage backing to a database so it could be shared, or simply parsing and extracting the data from the xml as you need it rather than keeping a representation of the whole thing in ram. – R.. GitHub STOP HELPING ICE Feb 05 '11 at 13:47
1

You've noted that your working set goes up after new/malloc. This is because they ask the OS for memory. You've also noted that it does not go down after delete/free. This is because they don't return memory to the OS. On a normal, sane system, this is not a problem. The unused memory space for your process will end up in swap, untouched and out of RAM.

On this special box, you'd be better off to override operator new with a direct call to HeapAlloc and operator delete with a call to HeapFree. Do enable the low-fragmentation heap for Server 2003; it's already the default for 2008.

MSalters
  • 173,980
  • 10
  • 155
  • 350
0

The private working set seems to be the virtual memory your program alone uses and needs, so I'm not sure that resetting is going to help you. I'd find out why your program is using so much memory, rather then trying to play around with the private working set.

Memory leaks?

Tony The Lion
  • 61,704
  • 67
  • 242
  • 415
  • I'm not sure it's so much a memory leak because my program doesn't grow once it's started. It may be that I really do need to use this much memory though because this customer has given me a very large config XML file most of which I need to keep in memmory (If I unload it the memory does go right down). Perhaps I need a profiling tool to work out where most of the memory is so that I can start pruning. – Benj Feb 04 '11 at 14:42