-1

I understand that dlopen/dlclose/dlsym/dlerror APIs are used to interface to a dynamic loading library but how does this work for a binary already compiled with the -L flag referring the .so library (shared library, background/load time linking). Edit: I am referring to live updating of shared library without restarting the app.

Rahul Deshpande
  • 31
  • 1
  • 5
  • 1
    You don't. You use `dlopen()` and friends when you want to do things like that. – Crowman Dec 29 '15 at 01:16
  • what are you planning to achieve with this ? – Pradheep Dec 29 '15 at 01:24
  • You can use LD_PRELOAD to load a library prior to the ones defined in the executable. See http://stackoverflow.com/questions/426230/what-is-the-ld-preload-trick. – esorton Dec 29 '15 at 03:16
  • @Pradheep I want to achieve live updating of the shared library and impact of the same on the application using it without the dlopen() family of calls and while keeping my current application build process intact. – Rahul Deshpande Dec 29 '15 at 22:15
  • @Paul Correct summarization. Please refer to my answer below for the details as per my research. – Rahul Deshpande Dec 29 '15 at 22:16

3 Answers3

2

Cong Ma provided some great pointers. Here is what I could conclude after some more research :

  • If an app is linked to an .so, and the shared library is replaced with a new copy of it, without stopping the app, the outcome would be 'non-deterministic' or 'unpredictable'.
  • During the 'linking/loading' phase of the app, the library gets mapped to the address space of the app/process.
  • Unix-like OS maintains COW, so for the reference, if copy is made on write, the app would have no impact though objective of being able to use new .so code would not be achieved
  • It all about the memory region that the app is accessing - if the access region does not have a change in address, the app won't experience any impact.
  • The new version of the lib may have an incremental nature, which might not impact the app - there can be compilation magic for the relocatable address generation too.
  • You may be lucky sometimes to fall in those above categories and have no issues while replacing the .so with a running app referring to it but practically you will encounter SIGSEGV/SIGBUS etc. most of the time, as the address/cross-reference gets jumbled up with a considerable change in the library.

dlopen() solves the problem as we reduce the window of access of the library. If you do dlopen() and keep it open for long enough you are exposing your app to the same problem. So the idea is to reduce the window of access to fail-proof the update scenario.

Rahul Deshpande
  • 31
  • 1
  • 5
2

I'd say it's possible to reload a dynamic library while an application is running. But to do this you'd have to design such "live reloads" into the library from the start.

Conceptually it's not that difficult.

First, you have to provide a mechanism to signal the application to reload the library. This is actually the easy part - at least in the Unix world: rename or unlink the original library and put the new one in its place, then have the application call dlclose() on the old library, and dlopen() on the new. Windows is probably more difficult. On Unix or Unix-type systems, you can even do the unlink/rename while the application is still running. In fact, on Unix-type systems you do not want to overwrite the existing library at all as that will cause you serious problems. The existing on-disk library is the backing store for the pages mapped from that library into the process's address space - physically overwriting the shared library's file contents will again result in undefined behavior - the OS loads a page it expects to find the entry point of the function foo() in but that address is now halfway through the bar() function. The unlink or rename is safe because the file contents won't be deleted from disk while they're still mapped by the process.

Second, a library can't be unloaded while it's in use (it's probably possible to unload a library while it's being used on some OSes, but I can't see how that could result in anything other than undefined behavior, so "can't" is de facto correct if not de jure...).

So you have to control access to the library when it's being reloaded. This isn't really all that difficult - have a shim library that presents your API to any application, and use a read/write lock in that library. Each call into the library has to be done with a read lock held on the read/write lock, and the library can only be reloaded when a write lock is held on the read/write lock.

Those are the easy parts.

The hard part is designing and coding your library so that nothing is lost or corrupted whenever it's reloaded. How you do that depends on your code. But if you make any mistake, more undefined behavior awaits.

Since you have to halt access to the library to reload it anyway, and you can't carry state over through the reload process, it's going to be a LOT easier to just bounce the application.

Andrew Henle
  • 32,625
  • 3
  • 24
  • 56
1

  1. Don't assume. Read about it... Inside stories on shared libraries and dynamic loading. tldr: Shared library is loaded (mostly) before the app is executed, rather than "in the background".

  2. Do you mean "live updating of shared library without restarting the app"? I don't think this is possible without hacking your own loader. See answers to this question.

Cong Ma
  • 10,692
  • 3
  • 31
  • 47
  • Actually,on modern OSes libraries are possibly loaded on-demand, i.e. when the first access occurs. This can very well be called "background" loading. And it is very well possible to load/unload a library dynamically. It is even mentioned in the document you linked. Note that this document is ~15 years old and techniques have advanced since then (I didn't read it, so the latter is just a general comment to be cautious when reading). – too honest for this site Dec 29 '15 at 02:48