LoadLibrary() fails with error 8 (ERROR_NOT_ENOUGH_MEMORY)

Question

Later edit: After more investigation, the Windows Updates and the OpenGL DLL were red herrings. The cause of these symptoms was a LoadLibrary() call failing with GetLastError() == ERROR_NOT_ENOUGH_MEMORY. See my answer for how to solve such issues. Below is the original question for historical interest. /edit

A map viewer I wrote in Python/wxPython for Windows with a C++ backend suddenly stopped working, without any code changes or even recompiling. The very same executables had been working for weeks before (same Python, same DLLs, ...).

Now, when querying Windows for a pixel format to use with OpenGL (with ChoosePixelFormat()), I get a MessageBox saying:

LoadLibrary failed with error 8:
Not enough storage is available to process this command

The error message is displayed when executing the following code fragment:

void DevContext::SetPixelFormat() {
    PIXELFORMATDESCRIPTOR pfd;
    memset(&pfd, 0, sizeof(pfd));
    pfd.nSize        = sizeof(pfd);
    pfd.nVersion     = 1;
    pfd.dwFlags      = PFD_DRAW_TO_WINDOW | PFD_SUPPORT_OPENGL;
    pfd.iPixelType   = PFD_TYPE_RGBA;
    pfd.cColorBits   = 32;

    int pf = ChoosePixelFormat(m_hdc, &pfd);    // <-- ERROR OCCURS IN HERE
    if (pf == 0) {
        throw std::runtime_error("No suitable pixel format.");
    }

    if (::SetPixelFormat(m_hdc, pf, &pfd) == FALSE) {
        throw std::runtime_error("Cannot set pixel format.");
    }
}

It's actually an ATI GL driver DLL showing the message box. The relevant part of the call stack is this:

                  ... More MessageBox stuff
0027e860 770cfcf1 USER32!MessageBoxTimeoutA+0x76
0027e880 770cfd36 USER32!MessageBoxExA+0x1b
*** ERROR: Symbol file not found. Defaulted to export symbols for C:\Windows\SysWOW64\atiglpxx.dll -
0027e89c 58471df1 USER32!MessageBoxA+0x18
0027e9d4 58472065 atiglpxx+0x1df1
0027e9dc 57acaf0b atiglpxx!DrvValidateVersion+0x13
0027ea00 57acb0f3 OPENGL32!wglSwapMultipleBuffers+0xc5e
0027edf0 57acb1a9 OPENGL32!wglSwapMultipleBuffers+0xe46
0027edf8 57acc6a4 OPENGL32!wglSwapMultipleBuffers+0xefc
0027ee0c 57ad5658 OPENGL32!wglGetProcAddress+0x45f
0027ee28 57ad5dd4 OPENGL32!wglGetPixelFormat+0x70
0027eec8 57ad6559 OPENGL32!wglDescribePixelFormat+0xa2
0027ef48 751c5ac7 OPENGL32!wglChoosePixelFormat+0x3e
0027ef60 57c78491 GDI32!ChoosePixelFormat+0x28
0027f0b0 57c7867a OutdoorMapper!DevContext::SetPixelFormat+0x71 [winwrap.cpp @ 42]
0027f1a0 57ce3120 OutdoorMapper!OGLContext::OGLContext+0x6a [winwrap.cpp @ 61]
0027f224 1e0acdf2 maplib_sip!func_CreateOGLDisplay+0xc0 [maps.sip @ 96]
0027f240 1e0fac79 python33!PyCFunction_Call+0x52
                  ... More Python stuff

I did a Windows Update two weeks ago and noticed some glitches (e.g. when resizing the window), but my program still worked mostly OK. Just now I rebooted, Windows installed 1 more update, and I don't get past ChoosePixelFormat() any more. However, the last installed update was KB2998527, a Russia timezone update?!

Things that I already checked:

• Recompiling doesn't make it work.
• Rebooting and running without other programs running doesn't work.
• Memory consumption of my program is only 67 MB, I'm not out of memory.
• Plenty of diskspace free (~50 GB).
• The HDC m_hdc is obtained from the display panel's HWND and seems to be valid.
• Changing my linker commandline doesn't work.

Should I update my graphics drivers or roll back the updates? Any other ideas?

System data dump: Windows 7 Ultimate SP1 x64, 4GB RAM; HP EliteBook 8470p; Python 3.3, wxPython 3.0.1.dev76673 msw (phoenix); access to C++ data structures via SIP 4.15.4; C++ code compiled with Visual Studio 2010 Express, Debug build with /MDd.

Answer 1

I was running out of virtual address space.

By default, LibTIFF reads TIF images by memory-mapping them (mmap() or CreateFileMapping()). This is fine for pictures of your wife, but it turns out it's a bad idea for gigabytes worth of topographic raster-maps of the Alps.

This was difficult to diagnose, because LibTIFF silently fell back to read() if the memory mapping failed, so there never was an explicit error before. Further, mapped memory is not accounted as working memory by Windows, so the Task-Manager was showing 67MB, when in fact nearly all virtual address space used up.

This blew up now because I added more TIF images to my database recently. LoadLibrary() started failing because it couldn't find any address space to put the new library. GetLastError() returned 8, which is ERROR_NOT_ENOUGH_MEMORY. That this happened within ATI's OpenGL library was just coincidence.

The solution was to pass "m" as flag to TiffOpen() to disable memory mapped IO.

Diagnosing this is easy with the Windows SysInternals tool VMMap (documentation link), which shows you how much of the virtual address space of a process is taken up by code/heap/stack/mapped files/shareable data/etc.

This should be the first thing to check if LoadLibrary() or CreateFileMapping() fails with ERROR_NOT_ENOUGH_MEMORY.