Compare two .exe for differences (both created on same machine)

Question

I am trying to test that Visual Studio generates a predictable/repeatable executable from the same code.

To do this, I'm creating a small program (one .cpp file) and building a .exe (A), making some changes and making another .exe (B), then un-doing the changes and making another .exe (C).

My theory is that the information for A and C will be the same and confirm that MSVS generates predictable/repeatable .exe's from the same code.

Several problems:

(i've saved the results of the commands in .txt's)

I've used dumpbin /rawdata to get rid of time/date stamp data and save the raw contents of each section of the file (not entirely sure what "raw" means), but that leaves 2 lines of difference when comparing with windiff.

Running dumpbin /headers shows differences in the .rdata section (Raw Data #2) under Debug Directories; the differences are in the time (which is expected) and in the Format: X, {Y}, "difference here", Z column.

I've searched forums and msdn for hours and cannot find a solution using dumpbin. Similar forum posts have ended with shrugged shoulders.

Can anyone give me a hand? I will do my best to add more specificity as requested.

Thank you, ZayJay

Viewed References:
http://support.microsoft.com/kb/177429
http://support.microsoft.com/kb/164151
http://msdn.microsoft.com/en-us/windows/hardware/gg463119.aspx
http://www.ms-news.net/f3614/how-can-i-compare-2-executables-1980031.html
+ others...

Also, running a comparison between my .cpp and a new .cpp with a main that only returns 0 yielded differences in the same areas. I expected there to be differences in the .text (Raw Data #1) section of the dumpbin /headers results... Anything to read or straight up answers would be great! thanks!

i believe those are in the .text (Raw Data #1) section of the results from `dumpbin /headers` and even comparing a .cpp with a few functions vs a .cpp with a main that returns 0 showed no differences there, only on the .rdata (Raw Data #2), File Headers Values (date/time), and Optional Header Values (checksum) — ZayJay, Apr 21 '11 at 16:57
So you want to know the meaning of the number between the PDB's GUID and the filename, or you want to generate exactly the same debug data, or something else? Why are you worried about the debug information reference being different? — Rup, Apr 21 '11 at 16:58
I'm creating a 'Release' executable (sorry if terminology is off). I don't care if the Debug Info is different, but find it strange that that is one of the only areas showing a difference when I compare two .exe's generated from difference source code. --edit-- also, what does PDB's GUID stand for? never seen the term, I'll read this later: http://www.wintellect.com/CS/blogs/jrobbins/archive/2009/05/11/pdb-files-what-every-developer-must-know.aspx — ZayJay, Apr 21 '11 at 17:01
@ Rup -- I see that I've got a lot of learning to do with PDB and GUID. I want to verify that MS Visual Studio 2005 will generate .exe's from the same source code that will operate in the predicted manner. Also, I do not know the meaning of the number you described, but that's something I'll try to do now — ZayJay, Apr 21 '11 at 17:11
just found this http://stackoverflow.com/questions/3316505/how-can-i-know-if-an-assembly-really-did-change and am looking into it; you helped out in it too Rup, so an extended thanks for those posts. — ZayJay, Apr 21 '11 at 17:24
------------ANSWER----------- It appears I cannot answer my own question, but here was my error: The compiler was optimizing my code and generating the same executable regardless of what I commented out because the result was always the same. First I just declared and initialized 3 integers, then returned 0. But because I didn't do anything with them, I assume the .exe didn't really have any executing code in it. Therefore, when I commented out one of the declarations and expected a change, there actually wasn't any difference in the resulting executable. --------CONTINUED------- — ZayJay, Apr 21 '11 at 21:06
Second, I declared and initialized an int, then called to functions to multiply and divide it by two. Again, because the end result was the same number no matter which functions I commented out, the executable remained the same (because of compiler optimization). Finally, using the 2nd link in my original post, I was able to find a difference when I call a function to multiply by 3 and then divide by 2, ending with different results no matter what I comment out. This caused it to act as expected. PS-- I also had the solution generating debug info on compilation, which was unnecessary THANK YOU — ZayJay, Apr 21 '11 at 21:11
PSS -- also, it only works for me if I output the data (no differences show up using `windiff` if I do not print the number with `printf()`) — ZayJay, Apr 21 '11 at 21:16
Just a shot in the dark, can it output assembly, and can you compare those, perhaps some sort of optimization is occuring behind the scenes?, edit: crap, I'm using my phone and your "answer" comment wasn't displayed, sorry, congrats I guess — onaclov2000, Apr 23 '11 at 06:19
Could be some thing inside the _resources_ section of the binary. Version may be? — Salman A, Apr 23 '11 at 17:16
Thanks onaclov2000 and Salman A. You were both right; there was data in the resources section and there was optimization going on behind the scenes. Thanks! — ZayJay, Apr 24 '11 at 15:16
Many inputs can produce same output, but each input has only one output, is necessary but not sufficient rule. You could have a small input change result it a huge output change, this happens with optimising compilers, but the behavioural change is still small(appropriate). Therefore if you do not trust the compiler, you must test the software. As this is something you should be doing anyway it is not a problem. — ctrl-alt-delor, Dec 13 '13 at 11:44
This question was really well written. Kudos ZayJay. I am trying to prove to a customer that VS2013 v120xp platform toolset builds build the same *.exe as a VS2019 v120xp platform toolset build, and ran into the PE variablilty issue. Hashes were always different, so was looking a way to prove that the two compilers are doing what I say they are doing... namely generating the same binary. — Ross Youngblood, Jun 19 '20 at 20:33

score 2 · Answer 1 · answered Apr 23 '11 at 17:10

There are a few fields in the portable executable format which will change every compilation (PE format is used for .exe and .dll). Writing a parsing app that memory maps such an exe and walks its internal datastructures isn't too hard (it is, in fact, what dumpbin does). You'll basically need to ensure that you exclude any GUIDs, date stamps, and anything else that might reasonably change.

The references section of http://en.wikipedia.org/wiki/Portable_Executable (the PE format docs from Microsoft) explain the format. I've written a similar app (mine was to extract version information of the resources section) and found the documentation to be pretty good.

That said, this seems like a bit of a waste of time. Modern compilers are fantastic at what they do. And they generally don't have randomness thrown into them, so you should expect the .rdata, .data and .text sections to be generally the same. In general, when you start questioning your compiler (especially a big one like Visual Studio or gcc), chances are you should take a deep breath and go back and look at your own code. Bugs are few and far between. And the few that there are usually have MSDN/StackOverflow articles written about them already.

(In other words: what exactly are you trying to accomplish?)

Thanks Smokey.Canoe. Haha, it's not exactly a waste of time but I can see what you're saying. I'm an intern at a company and they want to verify that software used on projects works as expected. This means ALL software. So I made some requirements for MS VS and had to come up with ways to test them. That is all really. Great learning experience though :) I also learned that I could answer my own questions....after typing it in my comments >.< In short, I was just trying to confirm that the two .exe's are the same for validation reasons — ZayJay, Apr 24 '11 at 15:14

Compare two .exe for differences (both created on same machine)

1 Answers1