How to Fast Fail a single threaded application with Windows 10 Parallel Loading?

Question

I have a utility application to spawn (or respawn) other processes, set up the console window and its title and icon and connect the standard handles to file-like objects (file, named pipe, socket).

In case of any failure this application has to terminate immediately and as fast as possible.

if ( !success )
  ExitProcess( 1 );

Problems come with "Window 10 parallel library loading support in ntdll.dll".

As described in this answer and in this article this feature prevents the fast fail.

Programs which execute in less than 30 seconds will appear to hang due to ntdll!TppWorkerThreadwaiting for the idle timeout before the process terminates.

Even if I set MaxLoaderThreads in the Registry, the tread pool is created when I call LoadLibrary.

My Question is, how to terminate the process without waiting for the loader-threads, without calling ZwCreateUserProcess directly and without modifying the Registry on the target machine.

My idea is to have a replacement for ExitProcess which calls TerminateProcess as follows (and it seems to work).

VOID WINAPI ExitProcessNow( UINT uExitCode )
{
  if ( TerminateProcess( GetCurrentProcess(),
                         uExitCode ) )
    ExitThread( uExitCode );
  else
    ExitProcess( uExitCode );
}

---

EDIT Some clarification

Assume the following program:

#include <Windows.h>
#ifndef _DEBUG
#pragma comment ( linker, "/ENTRY:EntryPoint" )
#else
#define EntryPoint main
#endif

#define NUMBER_OF_WORKER_THREADS  3

DWORD __stdcall WorkerThreadProc( PVOID __unused )
{
  for ( ;; )
    Sleep( 1000 );
}

void EntryPoint()
{
  for ( int ii = NUMBER_OF_WORKER_THREADS; ii; --ii )
    CloseHandle( CreateThread( NULL, 0, WorkerThreadProc, NULL, 0, NULL ) );
  Sleep( 10000 );
  // For demonstration do not call ExitProcess.
  // ExitProcess( 0 );
}

The initial thread starts three workers, waits 10 seconds and then exits. The workers are running indefinitely. If _DEBUG is not defined, there is no CRT code which calls ExitProcess at the end of main() and the process continues running with three threads at WorkerThreadProc.

This was the situation I observed. My process and/or a spawned process where running with three threads with an entry point at ntdll!Tp....

My code definitively called ExitProcess (after replacing ExitProcess by TerminateProcess it worked).

The point is, in cases when the program has to exit it has to exit immediately, independent of any locks in any library or any thread. Maybe it has something to do with the parallel loading or not, maybe there is as deadlock in my or the spawned code, it doesn't matter. The program has to exit and the process has to become signaled.

The main use case of this program is to respawn other programs when they exit for any reason as /sbin/init does when configured with respawn in /etc/inittab.

Answer 1

My Question is, how to terminate the process without waiting for the loader-threads

simply call ExitProcess as usual. this call just terminate process, and of course not wait for loader threads.

Programs which execute in less than 30 seconds will appear to hang due to ntdll!TppWorkerThreadwaiting for the idle timeout before the process terminates.

this is wrong or poorly worded. this is related to case when process not call ExitProcess but simply return from entry point (or terminate self thread).

process exit if:

• TerminateProcess called (with no error) - unconditionally cause a process to exit. no any user mode cleanup.
• ExitProcess called. however because here exist user mode cleanup - it can deadlock under some conditions.
• all threads exit in process. the your quote related exactly to this case. if your "single threaded" application just exit from entry point without call ExitProcess - process exit only in case no other threads in process. with parallel loader exist working threads which usually live up to 30 seconds if no tasks. as result process will exit not immediately. but after 30 seconds. but again - this is only if process not call ExitProcess direct.

in your case - process (spawn7.exe ) :

• or not call ExitProcess (in this case it can exit after 30 seconds)
• or deadlock in call ExitProcess - in this case need look it call-stack and debug it

---

OP say that

Even if I call ExitProcess (withing the 30 seconds while the thread pool ist active) the process is still there with the three threads from ntdll.dll but without my EntryPoint.

this is false and can not be. if ExitProcess called - first what it do - terminate all of the threads in the process, except the calling thread. or deadlock even before this (in call RtlLockHeap for main process heap). so will be:

• or still exist "EntryPoint" thread in process
• or only single thread in process (again your "EntryPoint" thread)

if "still there with the three threads from ntdll.dll but no other" - i can definitely say - ExitProcess not called (or hooked)

Answer 2

I cannot reproduce this at the moment, maybe it depends on the day(or night)time.

This situation (three threads in ntdll.dll!TpReleaseWork after exit) not only (sometimes) appears to my program "spawn(7).exe", but also (sometimes) to other (well-known) applications (I can see it in Process Explorer) which I cannot modify. Might be an odd situation in the "new" Feature "Window 10 parallel library loading support in ntdll.dll". Maybe I missed an update, maybe whatever.

Anyway, I need to "fail fast" and because I use the Processes for synchronisation (WaitForSingleObject) I now have two helper funtions.

ExitProcessNow to "kill" myself and WaitForProcessOrMainThread to wait for the target process (or its main thread) recently created with a call to CreateProcess.

VOID
__stdcall
ExitProcessNow( UINT uExitCode )
{
  if ( TerminateProcess( GetCurrentProcess(),
                         uExitCode ) )
    ExitThread( uExitCode );
  else
    ExitProcess( uExitCode );
}

VOID
__stdcall
WaitForProcessOrMainThread( PPROCESS_INFORMATION pProcessInformation,
                            BOOL fWaitForThread )
{
  if ( !fWaitForThread )
    WaitForSingleObject( pProcessInformation->hProcess, INFINITE );
  else
  {
    DWORD ExitCode = 0;
    WaitForSingleObject( pProcessInformation->hThread, INFINITE );
    GetExitCodeThread( pProcessInformation->hThread, &ExitCode );
    TerminateProcess( pProcessInformation->hProcess, ExitCode );
  }
  CloseHandle( pProcessInformation->hThread );
  CloseHandle( pProcessInformation->hProcess );
  pProcessInformation->hThread = NULL;
  pProcessInformation->hProcess = NULL;
}

BTW:

Interestingly, Windows 10 contains a default entry for chrome.exe with MaxLoaderThreads set to 1 to disable parallel loading. Windows 10 Parallel Loading Breakdown

There might be a reason for it. ;-)