Delphi: Speed up loading of interfaced plugins (DLL)

Question

A few days back I had help from SO members in creating a safe plugins system, using interfaces to communicate between the main app and the dll's. This solved some problems I was having with Access Violations and memory leaks, and all is working perfectly now, without errors or crashes.

So, I've been creating some long due plugins for this project, which lead me to another problem: Speed

What I'm doing right now is, when the main app starts, loads all dll's in a specific folder that follow a determined name patern.

The code I'm using to load them is the following:

if FindFirst(cfg.ExePath+cPathPlugins+'\np*.npl', faAnyFile, SR)<>0
   then Exit; // npl files are in fact renamed dll's

PluginHost := TPluginHost.Create as IPluginHost;
Plugins    := TObjectList<TPluginInfo>.Create(True);

repeat
  if (SR.Attr <> faDirectory)
  then begin
    dll := LoadLibrary(PChar(cfg.ExePath+cPathPlugins+SR.Name));
    if dll<>0
    then begin
      @PluginInit := GetProcAddress(dll, 'PluginInitialize');
      if Assigned(PluginInit)
      then begin
        Plugin := TPluginInfo.Create;
        try
          Plugin.Dll  := dll;
          Plugin.Intf := PluginInit(PluginHost);
          Plugins.Add(Plugin);
        except
          Plugin.Free;
        end;
      end
      else FreeLibrary(dll);
    end;
  end;
until FindNext(SR)<>0;
System.SysUtils.FindClose(SR);

This bit of code takes about 45s to load 7 plugins. None of these dll's have initialization code, and the PluginInitialize just passes the host interface and retrieves the plugin interface.

My questions are:

1. Can the number of methods on the interfaces affect the loading speed at this point? I don't believe so, but would like to confirm.

2. Could this loading time be cut somehow, while still loading them at startup?

I did thought of an alternative, having the names of the plugins to load in the database and loading the dll itself only upon first usage, but I'd prefer not to apply this, as some of these plugins must run automatically upon completion of some events during the app's execution, and not only though a menu option on demand.

I thought maybe this could be done loading the plugins in background, on a different thread, but I don't know if this could bring any risks, since I've never used threads. I believe the main risk with using threads is when one tries to access variables that are being modified by the other, is this right? In this case, that wouldn't happen, I think, as what comes after the plugin loading is grabing the plugins name (using one of its methods) and adding it to a TButtonGroup, that is created before starting the search for the dll's.

Do you believe this would be a good solution? If not, any alternative you can point me to?

Answer 1

Your problem is that the DLL's are large. You need to create the DLL's using run time packages. That way, the code is only loaded once. Each DLL will include duplicates of the same code. LoadLibrary will load the DLL and call the initialization code for each DLL. This means that package X would be linked into each plugin that uses it and would be initialized when each plug in is loaded. (corrected)

For standalone EXE file, taking off runtime packages is great. It makes deployment much simpler. When you want to start using a plugin system, it's best to switch to a system that includes runtime packages.

That doesn't mean that you need to keep every runtime package separate. For example, if you only use Dev Express controls in the main application or in a single plugin then you can let Delphi compile that package into the App/DLL.

To change which runtime packages you wish to keep separate and which ones you wish to include in the project go to the "Packages-Runtime Packages" page in the project options. There is a check box that lets you choose to link with runtime packages. Underneath is a text box. In this text box you can specify the names of the packages that you want to keep separate.