10

I have a reasonably large C++ program (~11mb exe) compiled under VS2008 and was interested to see if whole program optimization would significantly affect its performance. However, turning on whole program optimization and link time code generation causes the link to fail as follows;

1>c:\cpp\Win32\Atlas\tin\TINDoc.Cpp : fatal error C1083: Cannot open compiler intermediate file: '.\releaseopt\TINDoc.obj': Not enough space
1>LINK : fatal error LNK1257: code generation failed

Looking at task manager, I can see the linker using more and more memory until it runs out and bombs out. The compiler is running on XP 32bit with 2GB or ram and 2gb page file. Is WPO limited to smaller applications and/or bigger environments, or is there any way to get the linker to be a bit more frugal in memory usage.

n.b. already turned of precompiled headers, which was causing the compilation to fail before linking, and turned off output of debug info and anything else that might take extra resources. The help for C1083 suggests missing header files or inadequate file handles rather than lack of space.

Edit: Got it working under VS2010, albeit without precompiled headers, but the performance gains aren't that significant. I'll leave this option alone until I move onto a beefier 64bit platform with a more robust version of VS2010.

SmacL
  • 22,555
  • 12
  • 95
  • 149
  • 6
    Interesting question. What about free disk space; are you perhaps running low? I find [this bug on Microsoft Connect](http://connect.microsoft.com/VisualStudio/feedback/details/581207/visual-studio-2005-sp1-reproducible-linker-error-lkn1257-caused-by-c1083) that seems to match your symptoms. The comments say it's been fixed in VS 2010. If it's an option, you could try that. – Cody Gray - on strike Mar 02 '11 at 10:09
  • 1
    Seems to be the same bug ok. I'll try on VS2010 when I get a chance, and post my results. Thanks for the feedback. BTW, loads of available disk space. – SmacL Mar 02 '11 at 10:33
  • 2
    VC2008 is a fragile beast. The optimiser just doesn't work for some cases, and looks like you have one such case. – Michael J Mar 02 '11 at 16:34
  • @Michael, I'm starting to realise this. I ran some performance tests on a release version with all optimization rather than explicit inlining turned off, and it ran quite a bit faster than the optimized code (inline any + optimize for speed). Going back to explicit inlining + optimize for speed was still quite a bit slower than explicit inlining only. It really illustrates the importance of profiling rather than relying on black box technology. – SmacL Mar 03 '11 at 11:14
  • @michael, why not make that an answer? – Ben Mar 11 '11 at 13:12
  • @Ben I think Cody's is a better answer: the connect bug – Rup Mar 11 '11 at 13:41
  • @Ben - I did that (see below). Also expanded it somewhat. Hope it helps. – Michael J Mar 12 '11 at 18:56

2 Answers2

5

VC2008 is a fragile beast. The optimiser just doesn't work for some cases, and looks like you might possibly have one such case.

Examples of "not working" include

  • De-optimised code (slow!)
  • Compiler or (more frequently) linker crashes
  • Obscure compile/link error messages
  • Incorrect code execution (this one is rare, but not unknown).

Actually, if you look at some of the tricks used by modern optimisers it is pretty amazing that it works as often as it does. The complexity is quite astounding.

Addressing Shane's problem specifically, some things that might cause your error:

  1. Running out of file handles used to be a big problem in DOS and early Windows versions. You hardly ever see it in modern Windows versions. I'm not even certain there is still a limit on file handles.

  2. A compiler bug could be doing an infinite loop, meaning that no amount of resources will be enough.

  3. A recursive include file could also cause something similar. Do all your include files have "#pragma once" or "#if !defined(FOO_INCLUDED)"?

  4. Is it possible you've included the file TINDoc.obj twice in the project? The 2008 compiler is aggressively multi-threaded and there might be contention between two threads trying to access the file. Actually this could happen via a compiler bug, even if you haven't included the file twice.

  5. If the program really is just too big, consider breaking it up into one or more DLLs and building it piecemeal, or splitting the contentious source file into multiple files to compiler in multiple phases.

Don't assume that, because it says "not enough space" that has to be the reason. Some programs (including some compilers) will guess a reason for an error, instead of checking.

You can monitor the use of memory, handles, etc. using task manager or perfmon, or (my preference) Process Explorer

I already posted the first part of this as a comment (above) but I an making it an answer, as suggested by Ben (http://stackoverflow.com/users/587803/ben) -- and expanded it somewhat.

Michael J
  • 7,631
  • 2
  • 24
  • 30
  • Good synopsis Michael, I do use '#pragma once' on all headers, and was watching a resource graph so the space error seems reasonable. It's a big code base generating ~11mb of an exe, with a fair number of 3rd party DLLs already. I got the whole program optimization working using VS2010, and it doesn't yield much improvement in my case, so I'm just going to avoid this option for now. – SmacL Mar 15 '11 at 11:19
3

You can try to boot XP with the /3G switch to give the linker a 3GB address space. In Vista/7, use BCDEDIT /Set IncreaseUserVa 3072 in the command line to get the same effect.

This has fixed problems creating .ilk-files for me.

kbjorklu
  • 1,338
  • 8
  • 10