0

I have a main program that uses dlopen to load a library that is written in C++ and call a routine.

    handle = dlopen(LIBNAME,RTLD_LAZY);
    if(handle == NULL){
       fprintf(stderr,"Could not open library %s: %s\n",LIBNAME,dlerror());
       exit(1);
    }
    ...
    startptr = (int (*)(char*,char*,char*))dlsym(handle,"PluginStart");
    if (startptr == NULL){
       fprintf(stderr,"Could not find start\n");
       exit(1);
    }
    ...
    int res = (*startptr)(name,sig,descr);

The start code in the plugin creates an instance of a class, and spins of a thread to do the startup so that the start code can return to the main program. The thread runs a loop so that the plugin is running independently from the main program.

    Manager(){
       std::thread mgrThread(&Manager::threadFunc,this);
       mgrThread.detach();
    };
void Manager::threadFunc(void)
{
   //setup stuff that ends in a loop
}

This worked fine when the main program was written in C. I decided to add a map to the main program so I changed to C++, and the creation of the thread in the plugin now segfaults.

Program received signal SIGSEGV, Segmentation fault.
0x0000000000000000 in ?? ()
(gdb) where
#0  0x0000000000000000 in ?? ()
#1  0x00007ffff7e740a9 in std::thread::_M_start_thread(std::unique_ptr<std::thread::_State, std::default_delete<std::thread::_State> >, void (*)()) () from /lib/x86_64-linux-gnu/libstdc++.so.6
#2  0x00007ffff7fac0c7 in std::thread::thread<void (Manager::*)(), Manager*, void> (this=0x7fffffffddc0, __f=
    @0x7fffffffddd0: (void (Manager::*)(class Manager * const)) 0x7ffff7face36 <Manager::threadFunc()>)
    at /usr/include/c++/9/thread:130
#3  0x00007ffff7fabf76 in Manager::Manager (this=0x555555580500) at Manager.h:13
#4  0x00007ffff7fabc21 in PluginStart (outName=0x555555561080 <name> "", outSig=0x555555561480 <sig> "", 
    outDesc=0x555555561880 <descr> "") at testPlugin.cpp:35
#5  0x000055555555b678 in main (argc=1, argv=0x7fffffffdf68) at main.cpp:122

Creating a thread in the main program that does a single printf and exits fixes the failure.

void tstFunc(void){
   printf("xyzzy\n");
}
...
    std::thread tstThread(tstFunc);
    tstThread.detach();
...

So apparently the pthread library is can't dynamically load correctly from the plugin if the C++ library is loaded without pthread in the main program?
I was using -Wl,--dynamic-list=./mexports.txt in the link of the main program to make the callbacks the plugin uses visible to the plugin. I changed it to -Wl,--export-dynamic to see if a more generous symbol export worked, but it only works if I create and use a thread in the main program (for both link options).

Any help would be appreciated.

Ubuntu 20.04 LTS, g++ 9.4.0, no c++ std specified in the makefile.

Mimial Case: main.cpp (used to be main.c)

#include <stdio.h>
#include <stdlib.h>
#include <map>
#include <string>
#include <thread>
#include <dlfcn.h>
#include <unistd.h>
const char * LIBNAME = "./testPlugin.so";
void * handle;
// start in looping state
int looping = 1;

void tstFunc(void){
   printf("xyzzy\n");
}

// case 1 to stop crash - comment out to stop crashing
std::map<std::string,std::string> testMap;

int main(int argc, char * argv[]){

    handle = dlopen(LIBNAME,RTLD_LAZY);
    if(handle == NULL){
       fprintf(stderr,"Could not open library %s: %s\n",LIBNAME,dlerror());
       exit(1);
    }
    int (*startptr)(void);

    startptr = (int (*)(void))dlsym(handle,"PluginStart");
    if (startptr == NULL){
       fprintf(stderr,"Could not find start\n");
       exit(1);
    }

    // case 2 to stop crash - uncomment to stop crashing
    //std::thread tstThread(tstFunc);
    //tstThread.detach();

    // Call PluginStart
    int res = (*startptr)();

    printf("xyzzy\n");
    while(looping){
        sleep(2);
    }
    dlclose(handle);
}                   

file testPlugin.cpp

#include <thread>
#include <cstdlib>
#include <unistd.h>

#define PLUGIN_API extern "C" __attribute__((visibility("default")))

class testClass {
    public:
        testClass(){
            std::thread rosThread(&testClass::threadFunc,this);
            rosThread.detach();
      }
    private:
        void threadFunc(void);
};

void testClass::threadFunc(void){
   printf("in threadFunc\n");
   // infinite loop;
   while(1);
}
testClass * testptr = nullptr;
PLUGIN_API int PluginStart(void)
{

    printf("in plugin start\n");
    testptr = new testClass();

    return 0;
}

Makefile:

CCFLAGS=-fPIC -g
CPPFLAGS=-fPIC -g
MOBJS=main.o
all: bin/test2 testPlugin.so
LDFLAGS= -ldl -lpthread -lm
bin/test2: $(MOBJS)
    g++ -Wl,--export-dynamic -o bin/test2 $(MOBJS) -ldl -lpthread

main.o: main.cpp 

testPlugin.so: testPlugin.o
    g++ -m64 -static-libgcc -shared -Wl,--version-script=exports.txt -Wl,-rpath=. -o $@ testPlugin.o $(LDFLAGS)

testPlugin.o: testPlugin.cpp 

clean:
    rm *.o bin/test2 testPlugin.so

Ether comment out the map variable (resulting in only C code) or uncomment the thread call to stop the crash.

codegen
  • 118
  • 5
  • One extra comment, is that before adding the thread, if I commented out the map variable, then it worked. adding the map variable (the only C++ feature in the program) seems to be what triggered it. Compiling and linking when g++ and no C++ features worked. – codegen Feb 20 '23 at 01:04
  • Just because this is where the program crashes or reports an error doesn't mean this is where the problem is. C++ does not work this way. The problem can be anywhere in your code, but after the bug occurs the program keeps running for a little bit before it finally crashes here. This is why stackoverflow.com's [help] requires you to show a [mre] that everyone else can cut/paste ***exactly as shown***, then compile, run, and reproduce your problem. See [ask] for more information. Until you do that, it is unlikely that anyone will be able to answer your question. – Sam Varshavchik Feb 20 '23 at 01:11
  • 1
    How do you link the plugin? Does it have `g++ -pthread ...` in the link line? – Employed Russian Feb 20 '23 at 01:24
  • I link the plugin with -lpthread. I also found another thread with more searching for keywords https://stackoverflow.com/questions/20568235/using-c11-multithreading-in-shared-library-loaded-by-program-without-thread-su?rq=1. Using LD_PRELOAD also seems to work, so it may be the way in which symbols are resolved when the main program uses the c++ library. – codegen Feb 20 '23 at 03:38
  • _Every_ components have to be compiled/linked with option `-pthread`. Actually, every other compiler-options have to be the same. (And have to use the same compiler(version) -- different C++ compiler(version)s generate incompatible code.) – Zsigmond Lőrinczy Feb 20 '23 at 05:25
  • Also you cannot call C++ function from main program C. (Sometimes it seems working, but that just a coincidence.) – Zsigmond Lőrinczy Feb 20 '23 at 05:30
  • Working on a minimal case, but it is minimum of four files. All of the components were compiled with -lpthread, but my understanding is adding a library not he command line doesn't do anything if it isn't referenced. – codegen Feb 20 '23 at 17:31
  • added the minimal case requested by Sam Varshavick. – codegen Feb 20 '23 at 18:01
  • Be careful: `-pthread` and `-lpthread` aren't the same. (Though the former might include the latter.) – Lorinczy Zsigmond Feb 20 '23 at 20:40
  • Ok, that fixed it. -pthread. – codegen Feb 21 '23 at 00:37
  • I couldn't reproduce the problem (Linux/amd64 g++ 4.9.4 and 12.2.0). The program only prints `in plugin start\nxyzzy\nin ithreadFunc\n` then waits for Godot. – Lorinczy Zsigmond Feb 21 '23 at 16:08
  • Please quote the output of this command: `readelf -d bin/test2 | grep NEEDED` – Lorinczy Zsigmond Feb 21 '23 at 16:41

0 Answers0