1

Ours is a MFC application, which links to a win32 DLL. When the application invokes this DLL function, the argument "buffer" becomes a "Bad Pointer", after entering the function stack. This results in an application crash.

static MyClass* Instance(string& buffer);

I changed the argument type to "char *", but it only pushes the crash to next statement in the function. How to detect this heap corruption?

Few hints

  • This crash is reproducible, even if I invoke this DLL function from the very start of our application (CWinApp constructor). Could this memory corruption be caused by loading of resources, manifest etc?
  • The crash is ocurring in Vista and Win7, but not in XP.
  • Both these projects were recently migrated from Visual Studio 2002 to VS2008.

Code that invokes the function

CString data = "some string";
string str = data.GetBuffer();
data.ReleaseBuffer();
MyClass *obj = MyClass::Instance(str);
Community
  • 1
  • 1
Karthik Murugan
  • 1,429
  • 3
  • 17
  • 28
  • What is the target os bit version? 32-bit or 64-bit? – Kangkan Sep 27 '11 at 04:51
  • Its a 32 bit application – Karthik Murugan Sep 27 '11 at 04:56
  • Post code that shows the call to Instance. Are both exe and dll using shared crt of the same type (multithread dll/single thread dll)? – sean e Sep 27 '11 at 05:10
  • Posted invoking code. Both exe and dll are using shared CRT of multithread dll – Karthik Murugan Sep 27 '11 at 05:28
  • Does the memory that the string has allocated actually change during the invocation of Instance? View the string contents in the memory window - do not rely on the quickwatch window for str. Note that you should be able to assign data to str like "str = (LPCTSTR) data;" and then the ReleaseBuffer call can be removed. – sean e Sep 27 '11 at 06:36
  • As I said, it now crashes in the next statement that allocates a memory. – Karthik Murugan Sep 27 '11 at 09:32
  • @e I'm getting this WARNING, while building the DLL. Any clues? `LINK : warning LNK4098: defaultlib 'MSVCRTD' conflicts with use of other libs; use /NODEFAULTLIB:library` – Karthik Murugan Sep 27 '11 at 09:34

2 Answers2

1

There were two mistakes:

  1. Couple of custom built C++ files were not compiled with MD switch. We had to add -MD to the custom build script to make the CRT consistant with other objects.

  2. There were LNK2005 conflicts between LIBCMT.LIB and MSVCRT.LIB, which were otherwise ignored due to the /FORCE switch. We resolved these conflicts by removing LIBCMT.LIB in Linker->Input

Thanks all for your help.

Karthik Murugan
  • 1,429
  • 3
  • 17
  • 28
0

My guess is this is wrong usage of calling conventions or a CRT mismatch (i go with calling conventions).

Try building a stub DLL with the same function signature (which does nothing) and make it work with your MFC app.

Here's an example...

HTH

Community
  • 1
  • 1
Hertzel Guinness
  • 5,912
  • 3
  • 38
  • 43