I have a C# service running as the LocalSystem account which launches numerous other processes depending on what its needs are. This has been going fine for a few months. Just this week, some of the sub-processes are crashing. I've attached a remote debugger to them, and they're failing in memory allocations (C++ new operator returns 0x0), which is the indirect cause of the crash.
Funny thing is, if I RDP into the machine, I can easily launch the process from CMD no problems. Yet when the service launches it, no go.
The machine is running Windows XP SP3. It isn't out of the commit charge is about 80% of physical RAM.
Are there some special limitations of how many processes or how much memory can be used by a service, including processes spawned by that service??
Any other ideas why these processes would be unable to allocate memory.
EDIT:
I've had a good look at the crashing scenario with Procmon from SysInternals, and it doesn't reveal anything (that I can see). It all looks like it's going normal, then suddenly crashes. I can confirm from attaching a remote debugger that it is crashing after dereferencing a null pointer from a c++ new call. This is one of the first objects allocated in the app, it should never fail.
I also discovered that if I enable to services option: Allow services to interact with desktop, then all of the child processes launch correctly. The do, however, appear on the desktop when you connect via RDP and are unfortunately terminated if you log out via RDP = YUK! This still isn't an ideal solution, though - I'd really like to know why the child processes were unable to allocate memory after the 6th child process.