1

I found various sources that there should be little to no performance difference with the debug and release version of Visual Studio.

I am using Visual Studio 2015 and wrote a event driven state machine framework. After I did the first performance tests a was bumped because the framework made only about 13.000 state switches per second.

But as soon as I compiled it as release version I got 1.5 Million states switches per second. Thats a huge increase for me, but I cant explain where this comes from.

The sad part is, that I cant share my code because it belongs to my company.

But i found out that already the for-loop like this runs much quicker with the release Version.

for (auto i = 0; i < 2000000; i++)      
        my_machine.PushEvent(event);

Furthermore the memorey usage went down from like 255 MB to 67MB with the release version. The memory gets taken up by two queues.

I hope this gets not marked as duplicate, but I would like to hear some Input about the performance differences.

HWilmer
  • 458
  • 5
  • 14
  • 6
    "I found various sources that there should be little to no performance difference with the debug and release version" and they are (other than wrong)? – UKMonkey Jul 16 '19 at 16:35
  • I dont know if they are wrong, they were posted here on SO with a good amount of upvotes. So I thought there is some truth behind them. – HWilmer Jul 16 '19 at 16:40
  • 6
    "I found various sources that there should be little to no performance difference with the debug and release version of Visual Studio." - Disregard those sources. They clearly don't know what they are talking/writing about. There's a *huge* performance difference between a debug (unoptimized) and a release (optimized) build with the Visual Studio compiler (as well as with any other compiler). – Jesper Juhl Jul 16 '19 at 16:41
  • 2
    I have seen tasks that take a few minutes in Release and over a day in Debug mode. The slowdown is highly dependent on the code you are debugging. In the case above the algorithm did a lot of small allocations. – drescherjm Jul 16 '19 at 16:43
  • @HWilmer the reason I asked for the sources is to see if it was the source - or a misunderstanding of the source. You've not posted them after explict asking; which doesn't help – UKMonkey Jul 16 '19 at 16:44
  • Can you link to these sources? Perhaps there's a misunderstanding worth clearing up (I highly doubt that something stated simply as you posted it would get highly upvoted). – Angew is no longer proud of SO Jul 16 '19 at 16:44
  • I took this source [Source]( https://stackoverflow.com/questions/2446027/debug-vs-release-performance) – HWilmer Jul 16 '19 at 16:46
  • 3
    Your source is from a completely different programming language and also managed code with the .net framework versus native code with no .net. – drescherjm Jul 16 '19 at 16:47
  • 3
    "But as soon as I compiled it as release version I got 1.5 Million states switches per second. Thats a huge increase for me, but I cant explain where this comes from." - It comes courtesy of your compilers *optimizer*. C++ is designed to have lots of costly abstractions be completely removable by the compiler and the optimizer also has *lots* of tricks up its sleeve to turn slow code you write into very efficient code. When you don't enable the optimizer, code will be easy to debug, but slow. When you enable the optimizer, code will be fast, but *hard* to debug. – Jesper Juhl Jul 16 '19 at 17:06
  • 1
    "The sad part is, that I cant share my code because it belongs to my company" - But what's stopping you from writing (and posting) a *new* [mcve] that shows the same problem? – Jesper Juhl Jul 16 '19 at 17:47
  • In addition to the optimizations that everyone mentioned in debug mode in Visual Stuido there are also runtime checks (heap corruption, uninitialized variables, iterator checks ...) that are not performed in release mode. These checks are not free. – drescherjm Jul 16 '19 at 18:41

2 Answers2

4

I found various sources that there should be little to no performance difference with the debug and release version of Visual Studio.

This is quite probably incorrect, or misinterpreted information ... or, apparently, information about a different language. In case of misinterpretation, the original statement may have been that debug symbol information has no effect on performance, which would be correct.

Regardless, extra debug operations enabled by _DEBUG (Visual studio specific) or disabled by NDEBUG (standard macro that controls assertions) do have overhead. How significant overhead is depends on what the program does. If it spends most of the time waiting for harddrive or network, then probably not very significant. If it does a lot of operations on containers, then overhead is probably more significant.

Even more significant performance difference will come from lack of optimisations, that are enabled in Release build, and not in Debug build.

eerorika
  • 232,697
  • 12
  • 197
  • 326
1

Release builds in many ecosystems have debugging information off and performance optimizations on. It's likely that the latter accounts for the difference you're seeing.

If you'd like to test this, you could raise the optimization level of your debug build and see if it yields more comparable performance. Here's a list of optimization level arguments you could use - my guess is the release build is using /O2 or /Ox.

elucent
  • 36
  • 4
  • Thank you I will look into that, but dont you think the performance increase is a bit insane? – HWilmer Jul 16 '19 at 16:41
  • 3
    Not really. You're describing roughly a 100x speed improvement, and while that's definitely pretty considerable there are also definitely a few compiler optimizations that can yield pretty substantial gains in the right circumstances ([this comes to mind](https://stackoverflow.com/questions/9936132/why-does-the-order-of-the-loops-affect-performance-when-iterating-over-a-2d-arra)). – elucent Jul 16 '19 at 16:45
  • 4
    The inclusion of debug info does not, in general, slow down your application. It just takes up a bit more disk space. What slows down a debug build is *not running the optimizer*. Where I work we actually ship our software in optimized form (for performance) but with debug symbols enabled (for debugability of crash dumps). Having debug symbols enabled does not slow down the application, but makes it slightly less painful to debug crashes in optimized code. – Jesper Juhl Jul 16 '19 at 16:46
  • 2
    @HWilmer a loop writing over a million elements in an array could be completely removed by the compiler with optimisations if it turns out that the array is never read later. So no; 100x seems fair – UKMonkey Jul 16 '19 at 16:49