17

I'm trying to debug an issue in which an executable produces repeatable output (which I want) when executed directly from Visual Studio, but does not produce repeatable output when executed from the command prompt. It's a single-threaded application, so there shouldn't be any strange behaviour there in terms of timing.

Can somebody enumerate what possible differences could be between the two environments?

I'm sure the actual executable is the same -- they're both release builds and are running the same .exe file.

Here are the environments and the outcomes:

  1. Run directly from command prompt (cmd): Non-repeatable output
  2. Run from Visual Studio with Debugging (F5): Repeatable output
  3. Run from Visual Studio without Debugging (Ctrl-F5): Non-repeatable output

I know that the working directory can possibly be different, but I'm manually adjusting that to make sure that the working directory is identical.

Based on these results, it looks like running "with Debugging" (even in a Release build) somehow fixes the problem. Does this point to a likely culprit? What are the differences between running an executable with debugging and without?

SOLUTION: As pointed out in the accepted answer, the debug heap was the issue. The problem was that deep in the bowels of our code, somebody was accessing parts of a large array before they were initialized. They had allocated memory with a malloc and had not initialized the memory to 0. The debug heap would (I assume) fill the array with some repeatable value, whereas when the debugger wasn't attached (i.e. when run from the command line or with Ctrl-F5) the values were more random and would sometimes cause tiny deviations in the behaviour of the program. Unfortunately, the adjustment was so subtle as to almost be unnoticeable, and the memory in question was properly reset after the first "frame" of processing, but the initial conditions were already slightly different and the damage had been done. Chaos theory in action! Thanks for the guidance.

One great debugging tip that helped out: write a custom malloc that immediately fills memory with completely random data. That way, you can make sure you're properly initializing it yourself before using it, otherwise your results will be (hopefully) crazy every time you run it -- even in debug mode with the debug heap!

aardvarkk
  • 14,955
  • 7
  • 67
  • 96
  • 1
    Depends on your code. You may be triggering undefined behaviour, that only surfaces at higher optimisation levels. – Alex B May 22 '13 at 16:02
  • 1
    But theoretically isn't the only difference that a debugger isn't attached? The build is identical -- Release build both times. The degree of optimization isn't changing. – aardvarkk May 22 '13 at 16:03
  • I know you say you're adjusting for the working directory, but that seems to be the likely culprit. The other issue I would look for is installed libraries. VS handles dependencies (and if you build and install through ClickOnce, that handles your dependencies), but running directly on the command line may not. – Scott Mermelstein May 22 '13 at 16:05
  • My VS is rusty, are you certain it's running the same release executable (as opposed to the debug one)? What about linked libraries? – Alex B May 22 '13 at 16:07
  • 1
    It seems like my question title is actually not quite accurate -- the issue is reproducible just by running the application in two different ways. If I run with F5, I get the results I want. If I run with Ctrl-F5, I don't get the results I want. So it seems something related to attaching the debugger alters the behaviour of the executable. This should eliminate working directory and linked libraries as issues, no? – aardvarkk May 22 '13 at 16:10
  • This blog itself is pretty much useless: http://blogs.msdn.com/b/zainnab/archive/2010/11/01/start-debugging-vs-start-without-debugging-vstipdebug0037.aspx But the comments are interesting: "Start without debugging and your program won't use the Windows debug heap", and "Everytime you load a new module, the debugger looks for the PDB, loads it, etc." Neither are documented, but they might point you somewhere. – Scott Mermelstein May 22 '13 at 16:10
  • What is the issue that you're seeing? You only say "repeatable output" and "non-repeatable output." Can you narrow down the field by explaining what's changing? – Dan Puzey May 22 '13 at 16:12
  • This SO post seems potentially useful, too: http://stackoverflow.com/questions/5837070/release-build-not-working-when-run-without-debugging-in-vs2010 – Scott Mermelstein May 22 '13 at 16:13
  • @ScottMermelstein Yeah, I had been to that MSDN blog, but hadn't noticed the comment about the Windows debug heap. That seems interesting, as does your other link. I'll look into potential memory issues. – aardvarkk May 22 '13 at 16:15
  • 1
    @DanPuzey Well, more specifically I'm generating a massive file log. I load a bunch of files, process them, and spit out a bunch of output. When I run without debugging, the files match for a while, and then the results are suddenly slightly "off" (by a tiny amount). When I run with debugging, the files match identically. – aardvarkk May 22 '13 at 16:16
  • 1
    I have updated the title to be more descriptive. – Mats Petersson May 22 '13 at 16:25
  • This SO question is very similar, but not an exact duplicate: http://stackoverflow.com/questions/5311407/whats-the-difference-between-running-a-debug-app-hosted-in-visual-studio-and-ru?rq=1 – Adrian McCarthy May 22 '13 at 16:56
  • "Off" in what way? In terms of timing, or content, or does the output suddenly not work? You've not given much to go on in terms of what's actually going wrong. There are presumably lines of code that are behaving differently; if you could share some detail on what's changing then you'll get much more useful answers. – Dan Puzey May 22 '13 at 17:09
  • 1
    @DanPuzey It's quite a complicated image processing pipeline. By "off", I mean that the results of the calculations are off by a pixel, for instance. Everything matches, and then suddenly the results for a given frame don't match the results from the last time it was run. – aardvarkk May 22 '13 at 18:04
  • You might consider filling your allocations not with "totally random" values, but with fairly random, fairly recognisable values that are probably invalid when taken as addresses, e.g. random combinations of hex digits a-f. – PJTraill May 10 '15 at 15:34
  • *"The debug heap would (I assume) fill the array with some repeatable value,"* - Does the debugger fill the array *before* the program starts execution? So, if we fill memory with completely random data when our program starts, then it doesn't matter whether the debugger filled the array with some repeatable value or not? (because our custom malloc "overwrites" the memory with random value, even the debugger filled that memory previously) – starriet Sep 01 '22 at 04:04
  • And, what do you mean by this?: *"...and the memory in question **was** properly reset after the first "frame" of processing, ..."* (btw, thanks for posting this important question!) – starriet Sep 01 '22 at 04:09

2 Answers2

18

Windows Heap behaves differently if process is started under the debugger. To disable this behavior (in order to find a problem while debugging) add _NO_DEBUG_HEAP=1 to environment (like in this question).

Alternatively you can attach to process early in program execution. Heap will not enter the debug mode then. Add DebugBreak() line somewhere in the beginning of the execution, run with Ctrl+F5, and start debugging when asked to.

Dialecticus
  • 16,400
  • 7
  • 43
  • 103
  • Great link addition there, too. This seems like a very likely issue to me -- some kind of sentinel value that's repeatable when a debugger is attached, but is just garbage data otherwise (and thus the non-repeatable behaviour!) – aardvarkk May 22 '13 at 17:03
  • This pointed me in the right direction. I'll edit the question with some details. Thanks! – aardvarkk May 24 '13 at 19:13
  • The first link doesn't work. Is there any way to see the article? Thanks for the great hint though. – starriet Sep 01 '22 at 04:18
0

Well, it is difficult to say without knowing a bit more about your code. However, I had a similar problem with a program doing lots of floating-point arithmetics (double precision numbers).

The issue would show up when I was dealing with numbers that were slightly different, but numerically indistinguishable for the machine. If two doubles differ by less than numeric_limits<double>::epsilon(), they are seen as the same number for the machine. Hence, expressions of the type:

if (num1==num2)...

or

if (num1<num2)...
...

can result in colourful effects.

These colourful effects can vary when run in debug or release mode. The reason is that debug/release run-time libraries are different. Also, and crucially, the compilation is done with different code optimisations. The difference between the command-line debug version and the debug-window version (F5) is also explained by subtle optimisation differences.

If you're using VS, you can have a look at the effect of the different compilation options and optimisations in the C/C++ and Linker section of the Properties menu.

To avoid this problem, I recommend using the numeric_limits facilities from the <limits> STL library. As an example, the implementation of a less-than operator should be something like this:

bool operator<(double num1, double num2) {
    double difference=fabs(num1-num2);
    if (difference>numeric_limits<double>::epsilon()) {
        if (num1 < num2) return true;
        return false;
    }
    return false;
}
DrD
  • 419
  • 4
  • 14