Compiling big C files

Question

I've a big C file 400M approx. The file size is due to big resources within the code itself. I'm trying to compile with MinGW32 but with the mingw32-gcc.exe -O0 -g3 -ggdb3 -o file.exe file.c but the compiler shows me this error cc1.exe: out of memory allocating #N bytes. How can I solve this problem?

Answer 1

As been mentioned in the comments, the best solution is to not have huge files.

In your particular case, where you mentioned that the main function just consists of millions of nop instructions (add that to the question btw) I'd say that the solution is to allocate memory dynamically and then use a loop to create the nop instructions. After that you make the memory executable and call it. I see that you're using mingw32 so I found this answer that describes how to do that in Windows.

The code (copied from mentioned answer) would with a slight modification look like this:

// Allocate some memory as readable+writable
// TODO: Check return value for error
LPVOID memPtr = VirtualAlloc(NULL, sizeof(myarr), MEM_COMMIT, PAGE_READWRITE);

// Write NOP:s
for(int i=0; i<size; i++)
    memPtr[i] = // NOP code

// Change memory protection to readable+executable
// Again, TODO: Error checking
DWORD oldProtection; // Not used but required for the function
VirtualProtect(memPtr, sizeof(myarr), PAGE_EXECUTE_READ, &oldProtection);    

// Assign and call the function
(void (*)()) myfunc = (void (*)()) memPtr;
myfunc();

// Free the memory
VirtualFree(memPtr, 0, MEM_RELEASE);

Answer 2

Assuming x86(-64), since you're on Windows, the following code performs 400 million nop instructions:

00401000: B9 00 84 D7 17     mov         ecx,17D78400h
00401005: 90                 nop
00401006: E2 FD              loop        00401005
00401008: 33 C0              xor         eax,eax
0040100A: C3                 ret

Notice that this is 11 bytes of code, not 400 million.

Equivalent C code:

int i;
for(i = 0; i < 400000000; i++)
{
    __asm__("nop;");
}

Answer 3

Since your ACTUAL problem is that you need a huge executable, the solution is something completely different from what you tried, and this is why you always should tell what your actual problem is, instead of assuming that your attempt is correct or suitable. As mentioned in the comments, this is an XY-problem

Here is a pretty easy solution. First, create the boilerplate for an assembly program. On my computer, using Linux and nasm it looks like this:

section .text
global  _start
_start:

I named this file nop.asm

Now, I used a simple bash loop to add 10 million nop:s:

$ for i in $(seq 10000000); do echo nop >> nop.asm  ; done

Add the rest of the boilerplate:

$ echo "mov eax,1" >> nop.asm
$ echo "int 0x80" >> nop.asm

Note: The boilerplate may look different on your system

Now you will have a file called nop.asm that looks like this:

section .text
global  _start
_start:
nop
nop
...
nop
nop
mov eax,1
int 0x80

Compile and link it:

$ nasm -f elf64 nop.asm
$ ld -s -o nop nop.o

And now you have a pretty big binary:

$ ls -lh
total 94M
-rwxr-xr-x 1 klutt klutt 16M Oct 30 21:35 nop
-rw-r--r-- 1 klutt klutt 63M Oct 30 21:33 nop.asm
-rw-r--r-- 1 klutt klutt 16M Oct 30 21:34 nop.o

If I would get the same problem with nasm as you got with gcc, I would consider writing a program that writes the executable directly.