Synchronize, CheckSynchronize and WaitFor in Apache Web Application DLL

Question

Considering the following SO Question: Is it dangerous to use synchronize in a non-VCL application?

I was wondering how TThread's Synchronize, CheckSynchronize and WaitFor is affected when using them inside an Apache Web Application DLL.

I started investigating this when I realized WaitFor was locking/hanging. Inside WaitFor it checks if CurrentThread.ThreadID = MainThreadID and then repeats checking the result of MsgWaitForMultipleObjects and based on the result it will do CheckSynchronize or PeekMessage and when it received a WAIT_OBJECT_0 it will finally exit the loop. If the MainThreadID is not the same as the CurrentThreadID, then WaitFor will do a single WaitForSingleObject.

So I created the following test Apache Web Module DLL to see what the MainThreadID is during the lifetime of a web request.

type
  TTheThread = class(TThread)
  private
  protected
    procedure Execute; override;
  public
  end;

  TWebModule1 = class(TWebModule)
    procedure WebModule1DefaultHandlerAction(Sender: TObject;
      Request: TWebRequest; Response: TWebResponse; var Handled: Boolean);
    procedure WebModuleCreate(Sender: TObject);
    procedure WebModuleDestroy(Sender: TObject);
  private
  public
  end;

procedure DoLog(AMsg : String);

var
  WebModuleClass: TComponentClass = TWebModule1;
  TestThread : TTheThread;
  Logger : TLogger;

implementation

{%CLASSGROUP 'System.Classes.TPersistent'}

{$R *.dfm}

procedure DoLog(AMsg : String);
begin
  AMsg := AMsg + #13#10 + 'MainThreadID: ' + InttoStr(MainThreadID) +
                 #13#10 + 'CurrThreadID: ' + InttoStr(GetCurrentThreadId);

  Logger.Log(ltNormal,AMsg);
end;

procedure TTheThread.Execute;
begin
  While not Terminated do
    begin
      Sleep(5000);
      DoLog('Thread Execute - Inside While - Terminated: ' + BoolToStr(Terminated,True));
    end;

  DoLog('Thread Execute - End - Terminated: ' + BoolToStr(Terminated,True));
end;


procedure TWebModule1.WebModule1DefaultHandlerAction(Sender: TObject;
  Request: TWebRequest; Response: TWebResponse; var Handled: Boolean);
begin
  Response.Content :=
    '<html>' +
    '<head><title>Web Server Application</title></head>' +
    '<body>Web Server Application</body>' +
    '</html>';

  DoLog('WebModule1DefaultHandlerAction');
end;

procedure TWebModule1.WebModuleCreate(Sender: TObject);
begin
  Logger := TLogger.Create(nil);
  {set some Logger properties}

  DoLog('WebModuleCreate - Before Thread Create');

  TestThread := TTheThread.Create(True);
  TestThread.Start;

  DoLog('WebModuleCreate - Created Thread');
end;

procedure TWebModule1.WebModuleDestroy(Sender: TObject);
begin
  DoLog('WebModuleDestroy Before Terminate');

  TestThread.Terminate;

  DoLog('WebModuleDestroy Before WaitFor');

  TestThread.WaitFor;

  DoLog('WebModuleDestroy Before Free');

  TestThread.Free;

  DoLog('WebModuleDestroy End');

  Logger.Free;
end;

end.

So I start Apache, go to the web page in a browser, count down about 12 seconds and then I stop Apache. I give it a while but the httpd service only destroys a few threads but then stays the same and never closes.

Here is the log:

2021-03-03 10:23:42 081  WebModuleCreate - Before Thread Create
2021-03-03 10:23:42 081    ^ MainThreadID: 2432
2021-03-03 10:23:42 081    ^ CurrThreadID: 6328
2021-03-03 10:23:42 081  WebModuleCreate - Created Thread
2021-03-03 10:23:42 081    ^ MainThreadID: 2432
2021-03-03 10:23:42 081    ^ CurrThreadID: 6328
2021-03-03 10:23:42 081  WebModule1DefaultHandlerAction
2021-03-03 10:23:42 081    ^ MainThreadID: 2432
2021-03-03 10:23:42 081    ^ CurrThreadID: 6328
2021-03-03 10:23:47 093  Thread Execute - Inside While - Terminated: False
2021-03-03 10:23:47 093    ^ MainThreadID: 2432
2021-03-03 10:23:47 093    ^ CurrThreadID: 2220
2021-03-03 10:23:52 100  Thread Execute - Inside While - Terminated: False
2021-03-03 10:23:52 100    ^ MainThreadID: 2432
2021-03-03 10:23:52 100    ^ CurrThreadID: 2220
2021-03-03 10:23:57 101  Thread Execute - Inside While - Terminated: False
2021-03-03 10:23:57 101    ^ MainThreadID: 2432
2021-03-03 10:23:57 101    ^ CurrThreadID: 2220
2021-03-03 10:23:58 313  WebModuleDestroy Before Terminate
2021-03-03 10:23:58 313    ^ MainThreadID: 2432
2021-03-03 10:23:58 313    ^ CurrThreadID: 2432
2021-03-03 10:23:58 313  WebModuleDestroy Before WaitFor
2021-03-03 10:23:58 313    ^ MainThreadID: 2432
2021-03-03 10:23:58 313    ^ CurrThreadID: 2432
2021-03-03 10:24:02 108  Thread Execute - Inside While - Terminated: True
2021-03-03 10:24:02 108    ^ MainThreadID: 2432
2021-03-03 10:24:02 108    ^ CurrThreadID: 2220
2021-03-03 10:24:02 108  Thread Execute - End - Terminated: True
2021-03-03 10:24:02 108    ^ MainThreadID: 2432
2021-03-03 10:24:02 108    ^ CurrThreadID: 2220

As you can see, WaitFor never finishes. Interesting that OnWebModuleCreate runs in a different thread than the assumed MainThreadID but OnWebModuleDestroy runs in the same MainThreadID and as such CurrentThreadID = MainThreadID when WaitFor is called.

So I suspect I will have to never use Synchronize, and WaitFor in this case. Is that correct or are there more to this than I see?

What do I use to do synchronization if by my assumption the so-called main thread may not be calling CheckSynchronize at all and can I just do a single WaitForSingleObject instead of WaitFor? I suspect synchronization as in the sense to a main thread might never be required here at all.

Update

After a lot of reading I came across the information about DLLMain/DLLProc and that it is called whenever a process is "attached/detached" and a thread is "attached/detached"

There is also a blog post by Raymond Chen that discusses thread deadlocks in DLLMain.

So it seems that because DLLMain is called when the the DLL is unloaded and also when a thread exits I cannot do a thread WaitFor in the DLL Unloading code (OnWebModuleDestroy) as that will obviously cause a deadlock as the one will be waiting for the other. I'll appreciate it if anyone can advise on this opinion.

I don’t have experience with Apache Webapplications, but if these behave the same as ISAPI dll on IIS, then you need to terminate the thread in the Termination callback routine `TerminateExtension`. — R. Hoek, Mar 03 '21 at 21:17

Blurry Sterk · Accepted Answer · 2021-05-11T11:02:13.057

To recap:

A DLL's DLLMain/DLLProc is called whenever a process is "attached/detached" and a thread is "attached/detached"

There is also a blog post by Raymond Chen that discusses thread deadlocks in DLLMain.

Because DLLMain is called when the the DLL is unloaded and also when a thread exits I cannot do a thread WaitFor in the DLL Unloading code (OnWebModuleDestroy) as that will obviously cause a deadlock as the one will be waiting for the other.

After R.Hoek's comment I started looking to see if Apache calls some sort of cleanup procedure before it actually unloads the module DLL and came upon its apr_pool_cleanup and the companion apr_pool_cleanup_register procedure which registers a hook into the the cleanup. Fortunately this has already been handled inside the Web.ApacheApp unit and its TApacheApplication class that has an OnTerminate "event" which is called by the apr_pool_cleanup.

Assigning cleanup code to the OnTerminate "event" allows you to properly get rid of your threads without fear of deadlocks in DLLMain:

TApacheApplication(Application).OnTerminate := OnApplicationTerminate;

Synchronize, CheckSynchronize and WaitFor in Apache Web Application DLL

Update

1 Answers1