0
  • OS Linux Ubuntu 18.04, gcc 7.4.0

A program should load one SO which is dependent on other SOs. All exported functions of all SOs should be called by the program. I found serveral related questions but none dealing explicitly with this situation.

Here is an exmple what I am trying to do: There are 3 shared libraries and one application:

libtest1.so    has a extern "C" print function
 libtest2.so    has a base class "Base" with a function usig the C-function
 libtest3.so    has a derived class "Test" with a function using the C-function-
 DllTest        Application: loads the *.so's

This is what the application should to

pHandle = OpenSharedLib("./libtest1.so");      # works
 OpenSymbol(pHandle, "GetName");            # works
 CloseLib(pHandle);

 pHandle = OpenSharedLib("./libtest2.so");      # error
 OpenSymbol(pHandle, "GetName");
 OpenSymbol(pHandle, "printBase");
 CloseLib(pHandle);

 pHandle = OpenSharedLib("./libtest3.so");
 OpenSymbol(pHandle, "GetName");
 OpenSymbol(pHandle, "printBase");
 OpenSymbol(pHandle, "print");
 CloseLib(pHandle);

The error was that dlopen() failed to load due to an undefined symbol: ./libtest2.so: undefined symbol: GetName". The nm output shows that the symbol is missing, but I did not find out how I could prevent that.

The basic idea is to have a 'front SO' that is collecting all kind of seprate SO's and present a unified library to the program. In the example the program should only load libtest3.so. It should then be able to load any symbol as long as it was exposed by any of the single SOs.

My Question: Is it possible to do what I want and how? Or in other words: what is my error?

Here is the code and the commands I used to compile them is given below.

  lib1.h, lib1.cpp     for libtest1.so
  lib2.h, lib2.cpp     for libtest2.so
  lib3.cpp             for libtest3.so
  DllTest.cpp          the application

libtest1.so

header

extern "C" __attribute__((visibility("default"))) const char* GetName();

Cpp

#include <stdio.h>
#include <stdlib.h>
#include "lib1.h"
    
__attribute__((visibility("default"))) const char* GetName() 
{
   return "Hello from lib1";
} 

compiled with

g++ -c -fvisibility=hidden -fPIC -o lib1.o  lib1.cpp
g++ -shared -o libtest1.so lib1.o

libtest2.so

Header

#include <stdio.h>
#include <stdlib.h>
    
class __attribute__((visibility("default"))) Base 
{
public:
 Base();
 virtual ~Base();

 const char* printBase();
 int nTest;
};

Cpp

#include <stdio.h>
#include <stdlib.h>
#include "lib2.h"

#include "lib1.h"   // for GetName()
    
Base::Base() 
{ nTest=1; }

 
Base::~Base()
{ }

const char* Base::printBase()
{   return GetName(); }

compiled with

g++ -c -fvisibility=hidden -fPIC -o lib2.o  lib2.cpp
g++ -shared -o libtest2.so -L. -ltest1 lib2.o

libtest3.so

#include <stdio.h>
#include <stdlib.h>

#include "lib1.h"
#include "lib2.h"

class __attribute__((visibility("default"))) Test : public Base
{
public:
 Test();
 virtual ~Test();
 const char* print();
};

Test::Test()
{ }

Test::~Test()
{ }


const char* Test::print() {
 char* pChar = (char*)GetName();
 printf( "hello from lib3: %d", nTest);
 return "test3";
}

Compiled

g++ -c -fvisibility=hidden -fPIC -o lib3.o  lib3.cpp
g++ -shared -o libtest3.so -L. -ltest1 -ltest2 lib3.o

** Loading Application **

#include <stdio.h> 
#include <stdlib.h>  
#include <iostream>
#include <dlfcn.h> 

void OpenSymbol(void* pHandle, char* strName)
 {
 typedef char* (*pfnChar)(void);
 pfnChar pFunction   = NULL;
 char*  cError;

 printf(" Find symbol %s\n", strName);
 dlerror();
 pFunction = (pfnChar)dlsym( pHandle, strName );
 cError = dlerror();
 if (cError != 0)   {
  std::cout << cError << std::endl;
  exit(1);   }
 printf(" Exec symbol: %p\n", pFunction );
 std::cout << pFunction() << std::endl;
 }


void* OpenSharedLib(char* strName)
 {
 void*   pHandle; 
 char*  cError;

 printf(" open lib %s\n", strName);
 dlerror();
 pHandle = dlopen( strName, RTLD_NOW ); 
 cError = dlerror();
 if (cError != 0)  {
  std::cout << cError << std::endl;
  exit(1);  }
 printf(" found DLL %p\n", pHandle );
 return pHandle;
 }

void* CloseLib(void* pHandle)
 {  dlclose(pHandle);  }

main()
{
 void*   pHandle; 

 pHandle = OpenSharedLib("./libtest1.so");
 OpenSymbol(pHandle, "GetName");
 CloseLib(pHandle);

 pHandle = OpenSharedLib("./libtest2.so");
 OpenSymbol(pHandle, "GetName");
 OpenSymbol(pHandle, "printBase");
 CloseLib(pHandle);

 pHandle = OpenSharedLib("./libtest3.so");
 OpenSymbol(pHandle, "GetName");
 OpenSymbol(pHandle, "printBase");
 OpenSymbol(pHandle, "print");
 CloseLib(pHandle);

 std::cout << "done" << std::endl;
}

Running nm -DC shows that for the last two libraries some symbold are not exported.

  • symbols libtest1.so:
...
000000000000057a T GetName
  • symbols libtest2.so:
...
                 U GetName
                 U operator delete(void*, unsigned long)
000000000000094c T Base::printBase()
00000000000008da T Base::Base()
00000000000008da T Base::Base()
0000000000000920 T Base::~Base()
0000000000000902 T Base::~Base()
0000000000000902 T Base::~Base()
0000000000200e08 V typeinfo for Base
0000000000000969 V typeinfo name for Base
0000000000200de8 V vtable for Base
                 U vtable for __cxxabiv1::__class_type_info
  • symbols libtest3.so:
...
                 U GetName
                 U printf
                 U operator delete(void*, unsigned long)
                 U Base::Base()
                 U Base::~Base()
0000000000000ab2 T Test::print()
0000000000000a2a T Test::Test()
0000000000000a2a T Test::Test()
0000000000000a86 T Test::~Test()
0000000000000a58 T Test::~Test()
0000000000000a58 T Test::~Test()
                 U typeinfo for Base
0000000000200df0 V typeinfo for Test
0000000000000b0f V typeinfo name for Test
0000000000200dd0 V vtable for Test
                 U vtable for __cxxabiv1::__si_class_type_info
  • Finaly, the output DllTest
 open lib ./libtest1.so
 found DLL 0x55965d711ea0
 Find symbol GetName
 Exec symbol: 0x7f902c38157a
Hello from lib1
 open lib ./libtest2.so
./libtest2.so: undefined symbol: GetName

Edit after selected answer There are two main issues that code is not working. First, loading fails because the dynamic loader does not find the dependent shared objects, i.e. when loading libtest2.so it fails to locate libtest1.so and when loading libtest3.so it fails to locate libtest2.so and libtest1.so. That can be fixed by adding the path during linking. Second, accessing non extern "C" objects like classes require an object that must be created in the shared object. Therefore lib2.h/cpp, lib2.h/cpp and DllTest.cpp must be refactored.

For convenience here is the full compilation sequence with the corrections from the answer

g++ -c -fvisibility=hidden -fPIC -o lib1.o  lib1.cpp
g++ -shared -o libtest1.so lib1.o
  
g++ -c -fvisibility=hidden -fPIC -o lib2.o  lib2.cpp
g++ -shared -o libtest2.so -Wl,-rpath,$PWD -L.lib2.o -ltest1 

g++ -c -fvisibility=hidden -fPIC -o lib3.o  lib3.cpp
g++ -shared -o libtest3.so -Wl,-rpath,$PWD -L. lib3.o -ltest1 -ltest2 

g++ -o DllTest DllTest.cpp -ldl

With this the code allows the use of the const char* GetName() function no matter which shared object of the three is loaded.

CatMan
  • 173
  • 8
  • 1
    There is no `GetName` symbols in `libtest2.so` sooo...? There is also no `printBase` symbol, and it does not have `char* (*pfnChar)(void);` type, it's a `Base::printBase()` function, as you noted. ` Is it possible to do what I want and how?` yes `what is my error?` you are querying wrong symbols with wrong type from not these libraries. `<< pFunction()` if you want to call `Base::printBase()` you have to first define an instance of class `Base` to apply the function on. – KamilCuk Nov 19 '21 at 10:22
  • Thanks for the hint. So you are saying without adding a `create()` function in `libtest2.so` that creates an object I will not get a pointer from dlsym()? I am still a bit confused regaring the pointer and query. I tried to query `dlsym(pHandle,"Base:print"` or `Base::print()` but it does not seem to make a difference. Also its not clear why the function pointer would be different since also Base::print is of the type `const char* (void)`. Maybe coud show me how the `dlsym` line looks in that example. – CatMan Nov 19 '21 at 11:18
  • `dlsym(pHandle,"Base:print"` Research name mangling. You are using `nm -D` - remove `-D`. `an object I will not get a pointer from dlsym()?` Generally, yes. But the classes could be just named differently and you could declare them.. – KamilCuk Nov 19 '21 at 11:28
  • I had been trying "Base::printBase" before, but obviouly w/o creating the object. I see I need to check for answers for dlsym with classes. 'nm´ I was using with no option. Can you confirm that the code above at least should work to call `GetName` from `libtest2.so` as it is? – CatMan Nov 19 '21 at 11:50
  • 1
    Does this answer your question? [Why does the order in which libraries are linked sometimes cause errors in GCC?](https://stackoverflow.com/questions/45135/why-does-the-order-in-which-libraries-are-linked-sometimes-cause-errors-in-gcc) – n. m. could be an AI Nov 19 '21 at 11:56
  • `Can you confirm that the code above at least should work to call GetName from libtest2.so as it is?` No, there is no `GetName` symbol in `libtest2` `'nm´ I was using with no option` You stated: `Running nm -DC`. I made an error. Do not use `-C` - it's demangling symbols. – KamilCuk Nov 19 '21 at 12:31
  • @KamilCuk Seems to code works in respect to GetName when the library is created with the correct runpath. Interestingly, even though its working and `ldd libtest2.so` is showing that it knows the lib `nm libtest2.so` still does show this as 'undefined'. Of course it makes sense that the lib itself does not hold the symbol but loads it from `libtest1.so`, its dependency. I just hoped that there is a way to get `nm` to show me the the end result, i.e. how it would look like for a program loading that with `dlopen()`. But maybe thats a question on its own. – CatMan Nov 19 '21 at 13:12
  • @ n. 1.8e9-where's-my-share m. Thanks, answered my first remark on the answer. – CatMan Nov 19 '21 at 13:22

1 Answers1

2

As a start, change the order of the arguments in the link-commands

g++ -g -shared -o libtest2.so -L. lib2.o -ltest1
g++ -g -shared -o libtest3.so -L. lib3.o -ltest1 -ltest2

Note: it's not necessary with every linker, but the newer gold linker only resolves "from-left-to-right". Also there's a -Wl,--no-undefined option, which is very useful to catch these error.

Then add rpath into these commands so that the shared objects find their dependencies run-time:

g++ -g -shared -o libtest2.so "-Wl,-rpath,${PWD}" -L. lib2.o -ltest1
g++ -g -shared -o libtest3.so "-Wl,-rpath,${PWD}" -L. lib3.o -ltest1 -ltest2

After linkage, readelf -d should show the RPATH like this:

$ readelf -d libtest3.so |head -n10

Dynamic section at offset 0xdd8 contains 28 entries:
 Tag        Type                 Name/Value
 0x0000000000000001 (NEEDED)     Shared library: [libtest1.so]
 0x0000000000000001 (NEEDED)     Shared library: [libtest2.so]
 0x0000000000000001 (NEEDED)     Shared library: [libstdc++.so.6]
 0x0000000000000001 (NEEDED)     Shared library: [libc.so.6]
 0x000000000000001d (RUNPATH)    Library runpath: [/home/projects/proba/CatMan]

(Also there is libtool to deal with shared libraries.)

Mind you, because of name-mangling, the following change should be performed (though it is compiler-specific)

- OpenSymbol(pHandle, "printBase");
+ OpenSymbol(pHandle, "_ZN4Base9printBaseEv");
Lorinczy Zsigmond
  • 1,749
  • 1
  • 14
  • 21
  • Can you elaborate a bit why is the position of `lib2.o` important? I can see running `ldd libtest2.so` and `readelf -d libtest2.so` that even with your lines it can not find the required library `libtest1.so`. I would have thought that the `readelf` would list the current PWD as runpath. It does not show a runpath at all. I am still missing something... – CatMan Nov 19 '21 at 11:22
  • Thanks for pointing out libtool. Interesting read (and a lot). However I really would like to understand my error for this specific OS/compiler – CatMan Nov 19 '21 at 11:24
  • Added some more details. – Lorinczy Zsigmond Nov 19 '21 at 11:41
  • thanks for adding the `readeff`. Thats exactly the output I was expecting with your changes but I am not getting it. Currently trying to find out why....(probably something very stupid) – CatMan Nov 19 '21 at 12:14
  • 1
    I found my problem. I typed your command using the wrong brackets: `$(PWD)` instead of `${PWD}` or `$PWD`. Now it works as it should. And also I am getting the output from GetName() from libtest2.so. – CatMan Nov 19 '21 at 12:56
  • I am accepting your answer as its the root cause where I got stuck. It would be nice to have a way to find out if a given shared object actually could resolve an unknown symbol at the current place, i.e. `nm libtest2.so` shows `GetName` only as unknown. The workaround is to have a small test program and check the dopne()' errors. The code above does not fully work yet, since for the classes rework is needed as @KamilCuk commented, but there are other answers on that like [here](https://stackoverflow.com/questions/63452699/loading-a-c-class-with-polymorphism-using-dlsym) – CatMan Nov 19 '21 at 13:48