1

How can a shared library that is loaded at runtime from my main program use classes from another shared library that is also loaded at runtime?

I've modularized my game engine architecture. I've created a Core module that contains classes such as a custom String library. My Engine module uses the String class contained in Core throughout itself but currently can't compile because it doesn't know about the String class.

Is it possible to compile the Core module in a way that can provide Engine with enough detail to compile, while also preserving the value of a modules? If I statically linked Core to Engine, that would defeat the purpose.

If I dynamically linked Core to Engine, wouldn't that also defeat the purpose? The only thing I can think of is that this is necessary for modules to compile, but that I would still dynamically load them at runtime.

I ask because the .lib from DLL version of the Core module is enough to get Engine to compile, but will cause the DLL to be loaded at runtime if I kept with this. Is there a way to get engine to compile from basically what the .lib provides without actually causing them to be automatically loaded?

Stradigos
  • 814
  • 1
  • 13
  • 29
  • So I am guessing you're on Windows right ? Without any knowledge on the runyime loading library you're using there is little we can do. Going through that information should be the first thing you need to do. For example, yesterday I had another look at dlopen() on linux and from the man page I think your design is probably possible on linux with just that. – Arne J Jul 28 '19 at 05:47
  • How are you using the classes in the Engine DLL from your program? You are using `LoadLibrary` I assume? What would be different using the Core DLL from the Engine DLL? – Ted Lyngmo Jul 28 '19 at 07:39
  • Yes, I'm using Windows and am using the LoadLibrary call. Do you mean what would be different about using the DLLs at compile time instead of loading them at runtime? I'm hoping to be able to reload them as I work for one. The game itself is a DLL too that is selected and loaded at runtime. Can't know about that ahead of time. – Stradigos Jul 28 '19 at 11:55
  • I mean that if you've figured out how to load the Engine DLL at runtime and use the classes in that, you should be able to do the same for the Core DLL from the Engine DLL, perhaps in its DllMain. – Ted Lyngmo Jul 28 '19 at 12:53
  • You misunderstand. I can't compile the Engine module because it depends on classes defined in the Core module. But since I load both of those modules dynamically from my program at runtime Engine doesn't know anything about core at compile time. Thus, I can't compileYou are talking about runtime problems. I'm not even there yet. – Stradigos Jul 28 '19 at 13:01
  • But your goal is to load both at runtime, right? That's what I'm talking about. I just made [an example for Posix systems](https://stackoverflow.com/questions/56932785/automatically-creating-wrappers-for-classes-loaded-with-dlopen/56971263#56971263), but the princip is the same for Windows. You probably want to create an overload for `operator delete` too as shown in Remy's comment here https://newsgroups.embarcadero.com/thread.jspa?threadID=172448&tstart=465 – Ted Lyngmo Jul 28 '19 at 13:13
  • I do something similar to this for my Managers. I have an interface class for Render, for example, as well as a function to create or retrieve the actual manager using the interface. That makes sense to me because the interfaces don't have dependencies nor implementations. However, other examples I've seen do and I'm trying to reconcile how. – Stradigos Jul 28 '19 at 15:53
  • [Here's the Unreal Engine's string class called FString](https://github.com/EpicGames/UnrealEngine/blob/release/Engine/Source/Runtime/Core/Public/Containers/UnrealString.h). Notice the CORE_API symbol there to export the class. Core is compiled into a DLL. [Here an example of Engine using it](https://github.com/EpicGames/UnrealEngine/blob/release/Engine/Source/Runtime/Engine/Public/UnrealEngine.h) (Ctrl + F "FString". It comes in through CoreMinimal.h. – Stradigos Jul 28 '19 at 15:53
  • The Engine is its own module. These modules are loaded at runtime by the application. But how can they compile? I see in [Engine's build script](https://github.com/EpicGames/UnrealEngine/blob/release/Engine/Source/Runtime/Engine/Engine.Build.cs) that it references Core as a dependency but I don't know what that means in their custom build system. – Stradigos Jul 28 '19 at 15:54
  • This is almost like how, in another project, my EXE is also generating a .LIB file because [some of its functions are exported](https://stackoverflow.com/questions/2820694/why-does-my-visual-c-exe-project-build-create-lib-and-exp-files). A module it loads (at runtime) links with the .LIB so it knows about some of the functions in the executable and can call them. I'm basically trying to do this between two DLL's instead of an EXE and a DLL. **I just don't understand how, if I static link Core into Engine how that doesn't defeat the purpose of dynamically loading it later.** – Stradigos Jul 28 '19 at 16:06
  • @Stradigos All your links except the last one are dead. – Ted Lyngmo Jul 28 '19 at 21:50
  • @TedLyngmo, his issue is that by linking his `Engine` module against his `Core` module's `.lib` file, and using the entities contained within it as if they were in a static library, his `Core` module will be automatically loaded when his `Engine` module is loaded, without using `LoadLibrary()` (and, more pertinent, he won't be able to unload and replace it, as he wants). In effect, it would function more like an easier-to-update static library than a true dynamic library. – Justin Time - Reinstate Monica Jul 28 '19 at 21:53
  • @JustinTime I get that, but what OP is asking for is "_How can a shared library that is loaded at runtime from my main program use classes from another shared library that is also loaded at runtime?_" The Posix version I made as an example basically works for Windows too, by replacing the Posix functions with Windows ditos. The second link I shared showed some of the outlines for a Windows version. – Ted Lyngmo Jul 28 '19 at 21:59
  • @Stradigos To answer your other question: If you want to load both libraries at runtime you should not link any of them statically. – Ted Lyngmo Jul 28 '19 at 22:04
  • Seems like he's more concerned with them being coupled, @TedLyngmo, which is what I was talking about. From his description, he's concerned that implicitly linking to his `Core` DLL (which behaves _similar_ to static linking, but is a type of dynamic linking) will couple `Core` to `Engine`, and cause `Engine` to automatically load `Core` for him. He instead wants to be able to have `Engine` specify which module it dynamically links to, so he can swap it out. This isn't as simple as it is on POSIX, because Windows technically provides two separate types of dynamic linking. – Justin Time - Reinstate Monica Jul 28 '19 at 22:24
  • @JustinTime I agree that OP:s main concern isn't clear. The subject of the question and first sentence inside it is what I focused on since that seems to be the ultimate goal. – Ted Lyngmo Jul 28 '19 at 22:28
  • I think what I'm noticing is that Unreal's Core module is dynamically linked by other modules that ARE loaded at runtime. It's the only thing that makes sense. Since Core contains stuff like a String class, it's okay that each DLL has it's own copy. Modules that need access to other modules must do so through the same interface that the main application uses, and they are able to compile because abstract interfaces don't have an implementation to worry about. I will need to embed a sense of sense of module dependency and the ability to return a handle to a module that's already been loaded. – Stradigos Jul 28 '19 at 22:34
  • Sorry for the unclear question. It was difficult to articulate. Thank you both. – Stradigos Jul 28 '19 at 22:34
  • Understandable, @TedLyngmo, especially considering linking with DLLs is a bit wonky. @Stradigos, I _believe_ the issue is that you're concerned that [implicit linking](https://docs.microsoft.com/en-us/cpp/build/linking-an-executable-to-a-dll?view=vs-2019) (automatic dynamic linking immediately before execution, that functions like a dynamic form of static linking and doesn't use `LoadLibrary()` (the Win API `dlopen()` equivalent)) would couple your modules together, and wonder whether you can use explicit linking (manual dynamic linking during execution, with `LoadLibrary()`) instead, right? – Justin Time - Reinstate Monica Jul 28 '19 at 22:39
  • @JustinTime That's one way of phrasing the question, I suppose. If I could unload the DLL I implicitly linked I could at least reload it. But I can't. So then either I static link (boo), write an interface class for String and all other basic functionality classes so I can explicitly link with that later (nah), or live with implicit linking. Implicit linking is where I've arrived at for now, at least for Core. – Stradigos Jul 28 '19 at 23:56
  • It may be worth looking at [this question](https://stackoverflow.com/questions/4794032/implicit-vs-explicit-linking-to-a-dll), @Stradigos, as it seems that implicit linking is usually recommended over explicit (and conversely, that your case would fall into the "plug-in style DLL" category mentioned [here](https://stackoverflow.com/a/4794165/5386374)). ...That said, I don't see an easy solution for your issue, myself. – Justin Time - Reinstate Monica Jul 29 '19 at 19:17

0 Answers0