Injected DLL Main loop crashes the process

Question

I'm currently trying to inject a DLL into a project, but everytime i use a while loop the process crashes. This is the code:

BOOL APIENTRY DllMain( HMODULE hModule, DWORD  ul_reason_for_call,LPVOID lpReserved ){
    switch(ul_reason_for_call){
    case DLL_PROCESS_ATTACH:
        while(char c = getch()){
            if(c == 'p'){
                MessageBox(0,L"CAPT",L"CAPT",MB_OK);
            }
        }   
        break;
    }
return TRUE;
}

"I'm currently trying to inject a DLL into a project" i succesfully inject the dll into the process, i have tried to test with a message box on dllmain and it showed up, but every time i try to use a while loop it crashes. — Cornelius Aurel, Apr 11 '16 at 20:21
Are you using Detours or are you manually allocating and injecting into the process space? Did you ensure that your page is executable? — zetavolt, Apr 11 '16 at 20:25
I do it manually, but i don't really understand why the loop crashes the process, it pretty much works with everything else — Cornelius Aurel, Apr 11 '16 at 20:27
The easiest way to diagnose your problem is for you to say what error you are getting. Illegal instruction? Segmentation Fault? — zetavolt, Apr 11 '16 at 20:27
The process just gets stuck, not responding, not saying anything.(happen only when using while loops) — Cornelius Aurel, Apr 11 '16 at 20:50
You said before that "the process crashes." Now you've said it "just gets stuck, not responding." Those aren't the same things. When something crashes, that means it encountered an error and terminated. It will sometimes display an error message prior to terminating. When a program stops responding, that's called *hanging*. So, which is it: Does your program crash or does it hang? — Rob Kennedy, Apr 11 '16 at 20:59
@RobKennedy I'm sorry for the confusion, i wanted to say that it hangs. I have to end the process from the task manager because it becomes unresponsive and i can't do shit. — Cornelius Aurel, Apr 11 '16 at 21:03
When you use the debugger, where does it tell you the process is hung? — Rob Kennedy, Apr 11 '16 at 21:07
@CorneliusAurel I've updated my answer, it's probably because of `MessageBox`, see my answer below. — A.Fagrell, Apr 11 '16 at 21:07
@A.Fagrell it's not because of `MessageBox`, its because of the loop, still hangs even if i remove what's inside it: `while(char c=getch()){ }` — Cornelius Aurel, Apr 11 '16 at 21:18
@CorneliusAurel I don't think you should call `getch()` there, it will probably create a deadlock as well — A.Fagrell, Apr 11 '16 at 21:24
@A.Fagrell still no succes, i have tried: `while(1); for(;;) while(char c= getch())` and always the same result. What alternatives do i have? — Cornelius Aurel, Apr 11 '16 at 21:31
@CorneliusAurel of course your software will get stuck there if you have an infinite loop? — A.Fagrell, Apr 11 '16 at 21:36
You absolutely, positively **must** return from `DllMain`, as soon as possible. And don't do anything that might lead to another module being loaded. That's as much a deadlock as not returning from `DllMain`. Calling `getch()` is also not a good idea. Even if the process has a console attached, it's not yours to abuse. — IInspectable, Apr 11 '16 at 21:53
Remove everything from your DllMain. Everything. Do nothing there. Add in a line to return `TRUE` something missing from the code you posted. Which makes me think that you didn't post the real code. — David Heffernan, Apr 11 '16 at 21:53

score 2 · Accepted Answer · edited May 23 '17 at 12:15

Since you changed your original question, I'll rearrange my answer a bit...

There are significant limits on what you can safely do in a DLL entry point.

Please read carefully through the remaks on the following page: https://msdn.microsoft.com/en-us/library/windows/desktop/ms682583(v=vs.85).aspx and https://msdn.microsoft.com/en-us/library/windows/desktop/aa370448(v=vs.85).aspx

For example, it shouldn't take more than 300ms (recommended time) during the DLL_PROCESS_ATTACH. You're also not returning anything (should be a boolean).

However, I think the main reason your software hangs is because MessageBox and/or getch() probably creates a deadlock there. Regarding the MessageBox use OutputDebugString instead, refer to this answer: https://stackoverflow.com/a/10981735/5874704

Also as suggested in comments:

Don't put the while loop in DllMain. Use CreateThread in DllMain to launch a new thread. Put the while loop there

Previously you also asked about the definition of the DLLMain. This is the "bare"-function of the DllMain:

BOOL WINAPI DllMain( 
   HINSTANCE hDllHandle, 
   DWORD     nReason, 
   LPVOID    Reserved)
{
   BOOL bSuccess = TRUE;
   switch ( nReason )
   {
      case DLL_PROCESS_ATTACH:
          break;
      case DLL_PROCESS_DETACH:
         break;  
      case DLL_THREAD_ATTACH:
         break;
      case DLL_THREAD_DETACH:
         break;
   }

   return bSuccess;

}

still can't get the while loop to work, keeps crashing the process when injected to it . — Cornelius Aurel, Apr 11 '16 at 20:51
Don't put the while loop in DllMain. Use CreateThread in DllMain to launch a new thread. Put the while loop there. — Harry Johnston, Apr 12 '16 at 02:27

Injected DLL Main loop crashes the process

1 Answers1