2

I compiled 2 shared libs with --static-libstdc++ enabled.

The 2 shared libs have the same function f, which just outputs a string and an integer to stdout.

Main program will load the 2 shared libs using dlopen, and call the f in it using dlsym.

However, the second loaded shared lib failed to output the integer and the C++ stream cout becomes bad & fail.

ADD: After discussion, I know this is normal... However, I want to change my question to: what implementation of libstdc++ caused this issue? Is there any shared global state? I think if there is no shared global state, it shouldn't be a problem. I wrote the similar program in Windows by static linking to VCRuntime and using LoadLibrary, and it works normally. So why libstdc++ is designed like this?


The following is the code for 2 shared libraries. (They share the same code)
They will just cout a string and a integer.

// dll.cpp

#include <iostream>

using namespace std;

extern "C" void f()
{
    cout << "hi" << 1 << endl;

    bool is_eof = cout.eof();
    bool is_fail = cout.fail();
    bool is_bad = cout.bad();

    cout.clear();

    cout << endl;
    cout << "eof: " << to_string(is_eof) << endl;
    cout << "fail: " << to_string(is_fail) << endl;
    cout << "bad: " << to_string(is_bad) << endl;
}

This is the main program, which loads the shared libs and calls their f functions.

// main.cpp

#include <iostream>
#include <dlfcn.h>
#include <cassert>

using namespace std;

using fn_t = void(void);

void call_f_in_dll(const char *dll_path)
{
    auto dll = dlopen(dll_path, RTLD_LAZY);
    assert(dll);
    fn_t *fn = (fn_t *)dlsym(dll, "f");
    assert(fn);
    fn();
    dlclose(dll);
}

int main()
{
    call_f_in_dll("./libmydll.so");

    cout << endl;

    call_f_in_dll("./libmydll2.so");

    return 0;
}

Here's the CMakeLists.

# CMakeLists.txt

cmake_minimum_required(VERSION 3.16)
project (TestGCC)

add_link_options(-static-libgcc -static-libstdc++)

add_library(mydll SHARED dll.cpp)
add_library(mydll2 SHARED dll.cpp)

add_executable (main main.cpp)
target_link_libraries(main dl)

And the output is:

hox@HOX-PC:~/repos/test-gcc/out$ ./main
hi1

eof: 0
fail: 0
bad: 0

hi
eof: 0
fail: 1
bad: 1

Notice the second part, there is no 1 after hi and fail & bad become 1.

result

You can checkout the code here: https://github.com/xuhongxu96/dlopen-iostream-issue

Hongxu Xu
  • 63
  • 1
  • 7
  • What happens if you don't `dlclose()` the libs? – Simon Kraemer Aug 31 '20 at 07:45
  • @SimonKraemer I've tried and nothing changed... – Hongxu Xu Aug 31 '20 at 07:49
  • 1
    In [this post](https://web.archive.org/web/20160313071116/http://www.trilithium.com/johan/2005/06/static-libstdc/) the author mentions that "(...) for this to work reliably you must not use dynamically loaded C++ code, including code loaded with dlopen." – pptaszni Aug 31 '20 at 08:07
  • 1
    What is the reason that you want/need to use `-static-libgcc -static-libstdc++`? (Just to be sure that you really need that) – t.niese Aug 31 '20 at 08:09
  • Does this answer your question? [Linking libstdc++ statically: any gotchas?](https://stackoverflow.com/questions/13636513/linking-libstdc-statically-any-gotchas) – pptaszni Aug 31 '20 at 08:11
  • I just want to distribute our lib easier by reducing dependencies. The lib may be used in compliant server machines, and deploying binaries on it requires complex procedures. Of course I can use dynamic linking and copy the libstdc++.so with the lib. But I really curious about the issue. – Hongxu Xu Aug 31 '20 at 08:12
  • There are various posts about that topic with somehow contradicting conclusions. (like [Compiling with -static-libgcc -static-libstdc++ still results in dynamic dependency on libc.so](https://stackoverflow.com/questions/26304531) and [Linking libstdc++ statically: any gotchas?](https://stackoverflow.com/questions/13636513)). I would probably go with the suggestion of [this answer to Linking libstdc++ statically: any gotchas?](https://stackoverflow.com/a/14082540) `[…]Another option (and the one I prefer) is to deploy the newer libstdc++.so alongside your application […]` – t.niese Aug 31 '20 at 08:22
  • @pptaszni Thanks for your info. If I understood right, the article says there are some shared global states in libstdc++ and if I use dlopen to load 2 static-linked shared libs, they will conflict which could cause issues. – Hongxu Xu Aug 31 '20 at 08:22
  • 1
    @HongxuXu `[…]if I use dlopen to load 2 static-linked shared libs, they will conflict which could cause issues[…]` yes. – t.niese Aug 31 '20 at 08:24
  • @t.niese Thanks for info. I decide to continue using static linking now. But I'm still curious about the libstdc++ implementation of iostream/fstream and want to figure out what exactly caused the issue (maybe conflict shared global state?) – Hongxu Xu Aug 31 '20 at 08:25
  • I implemented the same pattern in Windows using MSVC by static linking to VC runtime and LoadLibrary to load the DLLs. It works normally. That's why I'm curious about the implementation :) – Hongxu Xu Aug 31 '20 at 08:27
  • @OP: That is perfectly normal, the reason is the multiple libstdc++. (Also your main program has to linked with libstdc++.) – Lorinczy Zsigmond Aug 31 '20 at 08:33
  • Can you try making dll global symbols local visible with auto dll = dlopen(dll_path, RTLD_LAZY | RTLD_LOCAL) ? – StPiere Aug 31 '20 at 08:43
  • @LorinczyZsigmond Thanks for answer. But If libstdc++ can be designed as an isolated lib, I don't think multiple instances could be a problem. I wonder what exactly the design of it makes it impossible. – Hongxu Xu Aug 31 '20 at 08:44
  • @HongxuXu `cout` needs to somehow pass the data to its destination, depending on what functionality it uses, the `stdlib` might use an indirection over another system feature or dynamically linked library or needs to manage the _"connection"_ to the console directly. If it manages the _"connection"_ to the console directly then the two static linked stdlibs would try to use the same socket/stream, which might conflict. – t.niese Aug 31 '20 at 08:47
  • @StPiere Thanks for info. Unfortunately, I just tried and it doesn't work. – Hongxu Xu Aug 31 '20 at 08:48
  • @Hongxu: ok. What I would try next would be RTLD_NODELETE and/or RTLD_NOSHARE. Unfortunately have no enough time right now to try out by myself. – StPiere Aug 31 '20 at 08:52
  • Also if parts of your programs were compiled with different compiler-settings (e.g. with/without pthreads) they won't be compatible. – Lorinczy Zsigmond Aug 31 '20 at 08:55
  • @t.niese Really thanks for the answer. I really want the detailed explanation like yours. I've been debugging for a while and found it actually threw `bad_cast` at `const __num_put_type& __np = __check_facet(this->_M_num_put);` (https://gcc.gnu.org/onlinedocs/gcc-4.8.3/libstdc++/api/a01401_source.html#l00072) – Hongxu Xu Aug 31 '20 at 09:26
  • @HongxuXu I don't know the exact reason why `cout` fails in you case, I just wanted to illustrate why such a problem could occur. Another thing that could be problematic, is to pass objects of a library `A` to or between dynamically linked libraries that link statically against that library `A`. E.g. if the library `A` somehow manages parts of their state internally (like a thread pool, or if memory allocation was changed in some way). – t.niese Aug 31 '20 at 12:06
  • 1
    "if I use dlopen to load 2 static-linked shared libs, they will conflict" They will not conflict with each other, but they WILL conflict with the state in your main program, possibly unless you use `RTLD_DEEPBIND`. – n. m. could be an AI Aug 31 '20 at 12:22
  • @n.'pronouns'm. Thanks for tips. I've tried DEEPBIND but it doesn't work. However, I finally made it work by using `dlmopen`, which provides more strict isolation. – Hongxu Xu Aug 31 '20 at 15:41

1 Answers1

0

Finally I found a way to resolve the issue in Linux (GNU Extensions).

Use dlmopen which provides better isolation btw objects.

auto dll = dlmopen(LM_ID_NEWLM, dll_path, RTLD_LAZY);

Great thanks for all commenters!

Still welcome to explain the details about what the conflict states are.

Hongxu Xu
  • 63
  • 1
  • 7
  • https://manpages.debian.org/testing/manpages-dev/dlmopen.3.en.html#dlmopen()_and_namespaces – Hongxu Xu Aug 31 '20 at 15:58
  • Possible uses of dlmopen() are plugins where the author of the plugin-loading framework can't trust the plugin authors and does not wish any undefined symbols from the plugin framework to be resolved to plugin symbols. Another use is to load the same object more than once. Without the use of dlmopen(), this would require the creation of distinct copies of the shared object file. Using dlmopen(), this can be achieved by loading the same shared object file into different namespaces. – Hongxu Xu Aug 31 '20 at 15:58