2

I've read in several sources that CreateProcess must not be called from a DllMain function.

CreateProcess :

Do not call CreateProcess from a DllMain function. This causes the application to stop responding.

Dynamic-Link Library Best Practices:

You should never perform the following tasks from within DllMain: Call CreateProcess. Creating a process can load another DLL.

Question

Why is that? it states that it causes the application to stop responding but this is just a symptom. what is the real reason?

The reason I'm asking is that I tried creating a process from a DllMain function and it sees to work just fine.

Community
  • 1
  • 1
idanshmu
  • 5,061
  • 6
  • 46
  • 92
  • Because it might create a deadlock but it does not have to - it is not promised to work fine. Your try was successful and some other would fail/lock. – Roman R. Nov 05 '14 at 10:00
  • OK. but is it out of my control or Can I code in such way that deadlocks are impossible? – idanshmu Nov 05 '14 at 10:02
  • This is completely outside your control. Even if you make sure that the executable image used to create that new process doesn't have any dependencies that need to be resolved, a system can be configured to load modules behind your back (AppInit_DLLs, global hooks, etc.). Heed the warning, there is nothing you can do about it. – IInspectable Nov 05 '14 at 15:58

2 Answers2

5

DllMain executes whilst the loader lock is held. As explained by the documentation you referenced, CreateProcess may result in a DLL being loaded. And that can lead to dead lock on the loader lock. The dead lock occurs because the loader lock is already held.

The documentation is clear. Don't call CreateProcess from DllMain. The standard way to get things done from DllMain is to create a thread to do the work. Although you must not wait on that thread because that leads to exactly the same dead lock.

David Heffernan
  • 601,492
  • 42
  • 1,072
  • 1,490
  • If the thread gets a pointer to a function defined inside `DllMain` scope and the thread is executed after `DllMain` has exited, wouldn't it cause the thread to address an invalid memory address? – idanshmu Nov 05 '14 at 10:13
  • Why would you do that. Don't do that. – David Heffernan Nov 05 '14 at 10:18
  • If CreateThread is called withing the scope of `DllMain` then it must get a pointer to a function in that scope. Am I lost? – idanshmu Nov 05 '14 at 10:24
  • No. Pass a pointer to memory with extended scope. For instance heap allocated memory. This is not special to DllMain. That's how passing parameters to threads always works. – David Heffernan Nov 05 '14 at 11:24
  • @Harry the DllMain docs in the desktop docs say not to call CreateProcess. – David Heffernan Nov 05 '14 at 22:27
  • On the other hand, that same document says not to call CreateThread either, and we know that's wrong, so I'm not sure how seriously to take the rest of it. Nonetheless, I agree that you shouldn't call CreateProcess. – Harry Johnston Nov 05 '14 at 22:33
  • @HarryJohnston That's an odd thing for it to say. Perhaps they just don't want to explain to you that you must not block on the thread and so bring back the same deadlock, and instead issue a blanket statement. – David Heffernan Nov 05 '14 at 22:45
  • That's always been my guess. :-) – Harry Johnston Nov 05 '14 at 22:51
  • @idanshmu: it isn't obvious to me what you mean by "function in that scope". Are you perhaps under the impression that the DLL itself will be unloaded when DllMain exits? – Harry Johnston Nov 05 '14 at 22:53
  • @HarryJohnston yes! _the DLL itself will be unloaded when DllMain exits_ **if** `DllMain` returns `FALSE`. so this makes it kinda risky. you must create the thread iff `DllMain` returns `TRUE`. – idanshmu Nov 06 '14 at 05:55
  • Why are you returning `FALSE`? – David Heffernan Nov 06 '14 at 07:00
  • `DllMain` may return `FALSE`. I want to stress the fact that `DllMain` may exit in such way that it causes DLL to be unloaded and thus invalidates the thread. It may seem obvious but it is likely someone finds this information useful. – idanshmu Nov 06 '14 at 10:38
  • 1
    Well, it would be an obvious mistake to return false to indicate that the DLL loaded, but then to continue working. All the same, I think your question is answered now. – David Heffernan Nov 06 '14 at 10:47
1

MSDN says:

Therefore, the entry-point function can call functions in Kernel32.dll that do not load other DLLs. [...] Unfortunately, there is not a comprehensive list of safe functions in Kernel32.dll.

Then it expands the statement explaining that more complicated APIs (CreateProcess included) might involve usafe API calling:

Calling functions that require DLLs other than Kernel32.dll may result in problems that are difficult to diagnose. For example, calling User, Shell, and COM functions can cause access violation errors, because some functions load other system components. Conversely, calling functions such as these during termination can cause access violation errors because the corresponding component may already have been unloaded or uninitialized.

This is what stands behind best practices advise to not call CreateProcess. It is out of your control whether CreateProcess will or will not load other DLLs. In your control is to avoid unsafe API call and create process later.

Roman R.
  • 68,205
  • 6
  • 94
  • 158