Visual Studio C++ Console Application Won't Run

Question

What can be done when a C++ console application builds and launches but won't run any code, at least not any that the debugger can see? I have a program with a main function:

int main (int argc, char *argv[])
{
    short retval = 0;
    retval = samain ( (short) argc, argv);
    return ((int) retval);
}

I put a breakpoint on the first line (retval=0) and the line after it just to be safe, and run. It shows a console window and hangs until I stop the debugger. The debug output shows a number of DLLs being loaded but nothing else. It ends with this line that looks very similar to the dozens before it:

'fcmtsysmd.exe' (Win32): Loaded 'C:\Windows\SysWOW64\RpcRtRemote.dll'. Cannot find or open the PDB file.

This is a program I know runs and functions in a "normal" environment because hundreds of people are using it and I haven't changed the code. The things different in my environment (because I'm in the process of rewriting our installation process) are:

I might not have all the dependencies installed properly.
One of the DLL dependencies that was causing an error earlier has been replaced by a newly built version with an Isolated COM reference instead of its usual COM reference because the COM object it was trying to access is not registered. (I'm trying to implement isolated COM in our new installation.)

So my question is, if an error is occurring before the first line of code executed, how can I track down the source of the problem?

I can't get a sensible call stack because even if I single step (F10 key) to start the program, it's hung right away. If I break at that point, I see the following call stack:

ntdll.dll!7743fdd1()    Unknown
[Frames below may be incorrect and/or missing, no symbols loaded for ntdll.dll] 
ntdll.dll!7743fdd1()    Unknown
ntdll.dll!77475f86()    Unknown
ntdll.dll!77459809()    Unknown

When I remove the isolated COM settings from the dependent DLL, I can get a access violation exception and stop on some code in the debugger. It's stopped on a function being called during the initialization of a static variable in another DLL. The strange thing is, when I add the isolated COM settings to the first DLL, and put a breakpoint on the other DLL, the breakpoint doesn't even get hit.

If you break inside the debugger it should tell you where it stopped by looking at the stack trace. It may be inside a function being called by a static object initialization. Make sure you're running a debug build or the breakpoints won't be reliable. — Mark Ransom, Nov 20 '15 at 22:07
your best bet is to load this module in depends.exe and view all the dependencies. If it reports any missing dependencies, copy those libraries in the same folder as this exe and re-run — Nandu, Nov 20 '15 at 22:09
@Nandu if there were missing dependencies you wouldn't get a hang, it would refuse to run altogether. — Mark Ransom, Nov 20 '15 at 22:10
If it hangs before application entry point, attaching debugger won't get u far. 1) built application with optimization turned off, 2.) when the app hangs, go to taskmgr, right click on the app, and create dump. Analyze the dump pointing to symbol server. — Nandu, Nov 20 '15 at 22:16
Very strange. Changing the isolated COM settings on the dependent DLL caused another DLL's static initialization to not even be seen by the debugger even though the isolated COM settings were introduced to resolve a dependency that happened after that code ran. — BlueMonkMN, Nov 20 '15 at 22:40
@Nandu good question, I haven't worked often enough with delay loaded DLLs. — Mark Ransom, Nov 20 '15 at 22:44

score 0 · Accepted Answer · edited May 23 '17 at 11:59

It turns out that the application was hanging during a call to CoCreateInstance triggered by some static initialization logic in one of the dependent DLLs. I guess the trick is trying to identify static initialization logic in dependent DLLs and putting breakpoints there. Unfortunately there doesn't appear to be any debug output to help identify which DLL's static initialization is executing when, so it's impossible to tell which DLL is the problem.

I will ask about how to identify the source of a hanging CoCreateInstance call in a separate question if there isn't already a question covering it: Application hung during CoCreateInstance when using Isolated COM

The final answer ended up being at https://stackoverflow.com/a/34076433/78162 (which I had to create myself).

Visual Studio C++ Console Application Won't Run

1 Answers1