0

I've been stuck trying to investigate a very strange program crash. The stack trace basically look like this

msvcr100.dll!malloc(unsigned int size) Line 89  C
msvcr100.dll!operator new(unsigned int size) Line 59    C++
msvcp100.dll!std::num_put<char,std::ostreambuf_iterator<char,std::char_traits<char> > >::_Getcat(const std::locale::facet * * _Ppf, const std::locale * _Ploc) Line 1054    C++
msvcp100.dll!std::use_facet<std::num_put<char,std::ostreambuf_iterator<char,std::char_traits<char> > > >(const std::locale & _Loc) Line 518 C++
msvcp100.dll!std::basic_ostream<char,std::char_traits<char> >::operator<<(int) Line 287 C++

What comes next after that is a line from my code which is just a simple cout << operator to print an integer

 cout << "Error is " << ret << endl;

I can't work out what causes this crash. I knew I'm not running out of memory since the process only uses only around 2000 kbytes right before crashing.

Since it's only printing a local integer variable, I don't see illegal memory access involved either.

Program was compiled with Visual Studio 2010 (v100) with Win32 Release profile and I've confirmed the production server has Microsoft Visual C++ 2010 x86 Redistributable installed.

Where else should I look for clues for the cause of this?

gerrytan
  • 40,313
  • 9
  • 84
  • 99
  • 1
    Does it also do it in Debug mode? – Adam Feb 10 '14 at 02:52
  • 2
    I find these sort of bugs tend to be caused by somehow mangling malloc's accounting data through buffer overruns or other misuses of pointers. A debug malloc library can help find such problems (it uses canary values and does more sanity checks). In other words, the problem is probably on a different line than the segfault. – Adam Feb 10 '14 at 02:54
  • Thanks @Adam, problem does not happen if program compiled in Debug mode. However VC++ 2010 redist package doesn't include the debug library so I can't deploy the debug version to production. I'll try to search buffer overruns on other lines. Is there recommended way to look for this as there are hundreds of lines? – gerrytan Feb 10 '14 at 03:27
  • With no debug library, it's time to start ifdef'ing out code trying to narrow down where the problem lies. Besides overwriting memory, you could be double-freeing something. – Graham Perks Feb 10 '14 at 04:23
  • @gerrytan The VC redist package might not do enough checks, even the debug version. It's been a while since I developed for Windows, but on Unix you can pre-link your own `malloc` to any process. This is how we would get taught to build a malloc, actually. So there are several such debug libraries available, the most common is `valgrind`. It does much more, but it can do this too. I don't know if there's a Windows version, but there should definitely be an equivalent. It will do all the checks mentioned and more, and you can run your release code against it. – Adam Feb 10 '14 at 05:09

2 Answers2

1

You should run your code with GFlags and "Full Page Heap"! I assume, that there is some kind of memory overwrite...

For more info see:

Also you can use Application Verifier to find other problems...

Community
  • 1
  • 1
Jochen Kalmbach
  • 3,549
  • 17
  • 18
1

My bet is your heap was hosed in a call before that one. If you are failing in malloc, it likely means that the internal structures C uses to keep track of memory are jacked up, and that often means you overran something somewhere. Or deleted something twice. Or some combination of that.

Consider looking at this:

http://msdn.microsoft.com/en-us/library/974tc9t1.aspx

TJ Bandrowsky
  • 842
  • 8
  • 12