allocate huge memory in MFC

Question

Sometimes I need to deal with huge PDF files, in such case, I use some new operator in a loop, and this is a so huge loop so that more than 10 GB memory will be used. As we all know, I must use some WIN API functions to use hard disk. But I don't know how to implement it. Anyone can help me?

Here is a code snippet:

CMyObject gg_data;
for(__int64 i = 0; i < up_limit(__int64); ++i)
{
   //add new data to the PDF file
   CCAry pData = new CCAry("myData");
   gg_data.AddRef(pData);
}

//after the loop, save file and free memory so that no memory leaks occur...
//Caution: the "save" is a very expensive work, it will take several hours!!!
gg_data.save();

Can you elaborate on what it is exactly that you're allocating memory for and how you're using this memory? Entire files? If so, do you modify the files in memory or not? Also, is your system 32-bit or 64-bit? — Alexey Frunze, Jan 16 '13 at 12:54
I want to add some of my own data to the existing PDF file and do some modification work. I hope my app can work both on x86 and x64.And I'm using Visual C++ 2010. — , Jan 16 '13 at 12:56
you probably need something like [file mapping](http://msdn.microsoft.com/en-us/library/windows/desktop/aa366883(v=vs.85).aspx) — AndersK, Jan 16 '13 at 12:58
Here is on limitations of file mapping - http://stackoverflow.com/questions/726471/how-big-can-a-memory-mapped-file-be — SChepurin, Jan 16 '13 at 13:30

score 2 · Accepted Answer · answered Jan 16 '13 at 13:00

2

Memory mapped files are what you want. This maps a file into a 'buffer' of memory and the OS manages it for you so that you can manipulate the file as if it was a huge chunk of RAM.

answered Jan 16 '13 at 13:00

gbjbaanb

51,617
12
104
148

3

If files are bigger than ~1GB, memory mapping will be tricky on 32-bit Windows. – Alexey Frunze Jan 16 '13 at 13:01
Could you give me more specific details? – Triumphant Jan 16 '13 at 13:03
1

@AlexeyFrunze But so will allocating the memory, so memory-mapped files aren't worse than the alternative – sashoalm Jan 16 '13 at 16:40

score 0 · Answer 2 · answered Jan 16 '13 at 13:00

0

If you want to use a Windows-specific allocation call, you should probably go for VirtualAlloc(). But I guess that using new is already allocating virtual memory so paging to hard disk should be taken care of by Windows as well.

You might need to increase the size of the pagefile on the hard disk if you are working with such huge amounts of memory.

answered Jan 16 '13 at 13:00

Christoph Freundl

627
7
20

Why does the code you provided in the original question not work for you? – Christoph Freundl Jan 16 '13 at 13:20

allocate huge memory in MFC

2 Answers2