0

I've got a problem with a C# application and a COM component allocating memory:

C# program calls a function in a COM DLL written in C++ which does matrix processing. The function allocates a lot of memory (around 800MB in eight 100MB chunks). This fails (malloc returns "bad allocation" when calling the function from C#.

If I run the same function from a C program, allocating the same amount of memory, then there's no problem allocating memory.

I've got 8GB RAM, Win7 x64 and there are plenty of free memory.

How to fix that it works to allocate memory when calling from the C# application? I tried to google it, but didn't really know what to search for. Searched for setting heap size etc, but that didn't give anything.

Feel a bit lost! All help are appreciated!

user1309971
  • 607
  • 1
  • 5
  • 11

1 Answers1

3

Amount of physical memory (8 GB) is not the constraint that limits memory consumption of your application. Supposedly, you built 32-bit application which has a fundamental limit of 4 GB of directly addressable bytes. For historical reasons, the application not doing any magic has only half of this - 2 GB. This is where you allocate from, and this space is used for other needs. 100 MB chucks are large enough to reduce the effectively usable space because of memory/address fragmentation (you want not just 100 chunks, you request continuous ones).

The easiest solution here is to build 64-bit applications. The limits there are distant.

If you still want 32-bit code:

  • enable /LARGEADDRESSWARE on the hosting application binary to extend limit from 2 to 4 GB
  • use file mappings, which you can keep in physical memory with your data and map into metered address space on demand
  • allocate smaller chunks
Community
  • 1
  • 1
Roman R.
  • 68,205
  • 6
  • 94
  • 158
  • But it do work just fine with 32-bit C application. Why should it not work in 32 bit C#? The result is a plugin for Excel, and that it 32-bit anyway, if that matters. – user1309971 Dec 03 '13 at 15:57
  • 1
    OK, 32-bit C application will stumble on 13th instead of 8th. What is taking place is C# app has a larger footprint on the process address space. – Roman R. Dec 03 '13 at 16:00
  • 1
    You're 32-bit C application isn't starting up an instance of the CLR which alone uses memory to set up the initial managed heap and load the required libraries like your C# application is doing. – Allan Elder Dec 03 '13 at 16:01
  • Note that your app will have to share address space with Excel as well, so you're having way less space to allocate from - and you will have to live with it, since you're playing away match here. – Roman R. Dec 03 '13 at 16:03
  • Another thing is that I can't change how the external matrix library is allocating memory (DLib using OpenBLAS). I just send in one row of data to be processed and then do that row by row to use less memory. – user1309971 Dec 03 '13 at 16:03
  • Is there any solution if I can't reduce the memory sent in to the matrix library? Excel call my COM addin with a row of data which calls the matrix library. – user1309971 Dec 03 '13 at 16:07
  • 1
    You can use out of process COM server, or otherwise split your addin so that it does its job in a separate process (which can be 64-bit, or dedicated 32-bit with still much more space than within Excel). This makes things more complicated, of course, but it is still doable and this is what I would perhaps do having this challenge in front of me. – Roman R. Dec 03 '13 at 16:09