1

I'm having a difficult problem to solve. I'm having two Qt-based applications, first one is in the main folder and the second one is located in it's subdirectory (yes, I'm forced to have it this way). The issue I'm facing is that I have to deliver 5 exact the same DLL's files for each application. I wouldn't have problem with that, if they wouldn't weight so much (10 DLL files = 60~ MB). Which is, definately too much.

On my debug build I am able to set the PATH variable within Visual Studio settings, and I will be not able to do so on production machines.

Is there any way I could set one of those application to rely on DLL files located in subdirectory?

Categle
  • 514
  • 1
  • 8
  • 22
  • 1
    On a modern PC-based system 60MB is not much. If you want to decrese the size of the installation file or archive, have just one copy of the DLL files and have the installation program copy them to their destinations. – Some programmer dude Oct 16 '16 at 11:29
  • @JoachimPileborg If I'd be able to do so, I would surely to that! Unfortunately I need to use file archiver and the condition is not to reach 1GB of it's size. Currently it's 1030 MB. Using best compression available. – Categle Oct 16 '16 at 11:36
  • Uhm... you opted to use Qt. Don't be surprised that you bought into bloat. But anyway, one non-portable solution to share the same set DLLs is to enable delay-loading (see [Linker Support for Delay-Loaded DLLs](https://msdn.microsoft.com/en-us/library/151kt790.aspx)), and handle the directory lookup yourself. This is explained under [Developing Your Own Helper Function](https://msdn.microsoft.com/en-us/library/kdw905aa.aspx). – IInspectable Oct 16 '16 at 11:52
  • That's the whole idea of DLLs - reuse. And sure, deployment of Qt applications is far from elegant. Consider using a static build, you end up with a single executable with no external dependencies, less than 10 MB in size for a standard widget app, or about 20 MB if you throw in QML and whatnot. – dtech Oct 16 '16 at 12:01
  • On android there was (and maybe still is) a thing called "ministro" which allowed multiple applications to reuse the same set of Qt libraries, manage different versions of the libraries and whatnot. It would be a good thing if something like that existed for all platforms, but it is not like Qt makers are too concerned with efficiency... – dtech Oct 16 '16 at 12:06
  • @IInspectable There is no chance to delay-load my Qt libraries, issues with initializers and such: `fatal error LNK1194: cannot delay-load 'Qt5Core.dll' due to import of data symbol '"__declspec(dllimport) public: static struct QListData::Data const QListData::shared_null" (__imp_?shared_null@QListData@@2UData@1@B)'; link without /DELAYLOAD:Qt5Core.dll` same happens for other libraries. – Categle Oct 16 '16 at 12:14
  • @ddriver I wish such application could exists for VS! – Categle Oct 16 '16 at 12:16
  • The optimal solution would be to use static linking. – dtech Oct 16 '16 at 12:25
  • @ddriver I guess.. I will give it a try and see how many MB I can save. – Categle Oct 16 '16 at 12:28
  • ... create a small "launcher" for the second application in the first application's folder, that simply sets PATH and launches the second one? – peppe Oct 16 '16 at 12:35
  • Any idea if `SetDllDirectory()` could work something out here? – Categle Oct 16 '16 at 13:05
  • 1
    `AddDllDirectory`, `SetDllDirectory`, and `SetDefaultDllDirectories` modify the search path of the calling process. You would have to use them, **before** any DLLs are imported. In essence, you'd have to use delay-loading. And that will only work, if the respective libraries are dynamically loaded (vs. statically resolved, before any application code runs). It is the less capable approach to using delay-loading with user-defined helper functions. – IInspectable Oct 16 '16 at 13:34
  • 1
    @peppe: That only really works, if you can add your directory to the `PATH` environment variable. Both the environment variables as well as the complete environment have fairly low limits, that it's likely that you'll run into them. And since you work for the *"Qt experts"* you know the answer to the problem already: Make Qt suck less. – IInspectable Oct 16 '16 at 13:37
  • @IInspectable: first, don't make this an ad-hominem. Second, why does a `.bat` that does what I suggested does not work? Third, suppose to replace Qt with MyFancyLibrary.dll. How does it make this problem any different? – peppe Oct 16 '16 at 14:02
  • 1
    @peppe: `1` Ad hominem? Do you often feel hunted? `2` I believe I answered that question before you asked: The environment is limited in size. Adding to it can fail. `3` MyFancyLibrary.dll makes the problem different, because MyFancyLibrary.dll **can** be delay-loaded. – IInspectable Oct 16 '16 at 14:08
  • Is there no mechanism on Windows to make the runtime linker aware of directories where DLLs reside in? Or some standard locations that an installer can put DLLs into? – Kevin Krammer Oct 16 '16 at 16:43
  • 1
    @KevinKrammer: I outlined the procedure above, but Qt seems to not be prepared for this. The standard location an installer can put DLLs in is the application directory. The OP doesn't want this for one of their applications. – IInspectable Oct 16 '16 at 17:28
  • What about Deployment tool: http://doc.qt.io/qt-5/windows-deployment.html#the-windows-deployment-tool? – Lucas Oct 17 '16 at 05:03
  • @Lucas im not sure about that – Categle Oct 17 '16 at 09:03
  • @IInspectable: those function don't look like they could help, as this would require code to run before the runtime linker start working. But libraries linked to the program need to have been resolved before the program runs, otherwise it can't start, no? But even is that is supposed to work, how would a DLL "not be prepared" for this? Isn't that betwee the application and the runtime linker? If the runtime linker tries to load Qt DLLs before the application was able to execute these functions, how would Qt prevent that? Especially since the libraries are apperenty not found? – Kevin Krammer Oct 17 '16 at 17:58
  • I am really quite puzzled by this, having multiple lookup paths for runtime linked libraries sounds like one of the most basic features of a modern operating system to me. There are multiple ways to do this on Linux, even such trivial ones as setting LD_LIBRARY_PATH. Windows might have had problems with DLLs in the past but it must have gotten better at that then this – Kevin Krammer Oct 17 '16 at 18:01
  • @KevinKrammer: [That comment](http://stackoverflow.com/questions/40069701/can-two-independent-qt-applications-use-same-dll-files?noredirect=1#comment67413224_40069701) has the information. Delay-loading postpones import resolution until first use, and the user-provided helper functions can perform arbitrarily complex library lookup. Of course, if you want something as brittle as setting the `LD_LIBRARY_PATH`, you **can** set the `PATH` environment variable, too. And watch this kludge come tumbling down. – IInspectable Oct 17 '16 at 18:29
  • @IInspectable: I agree, setting `PATH` does sound like a clutch, definitely not as nice as a reliable standard solution as setting `LD_LIBRARY_PATH` . The delay-loading sounds like an alternative to runtime loading, but maybe it can indeed be used for startup linking as well. Does seem to have a couple of constraints though and apparently doesn't work consistently for all DLLs. Still, very interesting concept! – Kevin Krammer Oct 17 '16 at 18:44
  • @KevinKrammer: Delay-loading doesn't work with Qt due to static initializers. As I commented earlier, the solution is to *"make Qt suck less"*. Replace the static initializers with magic statics, for example, or a proper RAII scheme altogether. That would even allow to run leak detection tools (which currently report lots of phantom leaks when used with Qt). Besides, setting `LD_LIBRARY_PATH` is as brittle as changing the `PATH` environment variable. Either approach changes the **global** environment of a program, potentially pulling libraries from directories, that weren't meant to be used. – IInspectable Oct 17 '16 at 18:56
  • Ok, I don't consider `LD_LIBRARY_PATH` to be brittle as it always worked very reliably for me. And it would not lead to any libraries being pulled from directories that aren't ment to be used, after all it only contains directories that are meant to be used – Kevin Krammer Oct 18 '16 at 15:50
  • @KevinKrammer: You cannot limit, which **libraries** to pull from certain directories, when modifying the `LD_LIBRARY_PATH`. It's an all or nothing decision. You move a directory to the front, and all libraries that happen to be there are considered first, even those that were meant to be loaded from somewhere else. And frankly, the old *"has always worked for me"* is great famous-last-words material. – IInspectable Oct 18 '16 at 16:37
  • Well, that's the very definition of a search path, just like any of those used by the runtime linker due to its primary config. Nobody adds random paths to their LD_LIBRARY_PATH, especially not in their start scripts. And nobody puts random libraries into their application's library directory. The chance of loading an unwanted library that way is basically zero – Kevin Krammer Oct 19 '16 at 17:22
  • You wouldn't even need LD_LIBRARY_PATH. On an OS with proper support for dynamic libraries (read: not Windows) you'd be using RPATH in your binaries. – peppe Oct 19 '16 at 21:39
  • `QListData::shared_null` (mentioned before) is a const global POD, with no relocations (it's a bunch of integers), which even ends up in `.rodata` on Linux. *It does not have any runtime initialization, and no runtime cost*. Apparently, the mere fact of having global data is enough for delayed DLL not to work. – peppe Oct 19 '16 at 21:53
  • @peppe: Care to explain how Windows does not have *"proper support for dynamic libraries"*? – IInspectable Oct 20 '16 at 01:48

1 Answers1

1

I don't know what kind of an installer you're using, but the dlls should be stored only once in the archive, and the files should be hard linked on installation. So it's a non-issue unless your installer is broken or the install script is.

Kuba hasn't forgotten Monica
  • 95,931
  • 16
  • 151
  • 313