dlopen from memory?

Question

I'm looking for a way to load generated object code directly from memory.

I understand that if I write it to a file, I can call dlopen to dynamically load its symbols and link them. However, this seems a bit of a roundabout way, considering that it starts off in memory, is written to disk, and then is reloaded in memory by dlopen. I'm wondering if there is some way to dynamically link object code that exists in memory. From what I can tell there might be a few different ways to do this:

1. Trick dlopen into thinking that your memory location is a file, even though it never leaves memory.

2. Find some other system call which does what I'm looking for (I don't think this exists).

3. Find some dynamic linking library which can link code directly in memory. Obviously, this one is a bit hard to google for, as "dynamic linking library" turns up information on how to dynamically link libraries, not on libraries which perform the task of dynamically linking.

4. Abstract some API from a linker and create a new library out its codebase. (obviously this is the least desirable option for me).

So which ones of these are possible? feasible? Could you point me to any of the things I hypothesized existed? Is there another way I haven't even thought of?

Answer 1

I needed a solution to this because I have a scriptable system that has no filesystem (using blobs from a database) and needs to load binary plugins to support some scripts. This is the solution I came up with which works on FreeBSD but may not be portable.

void *dlblob(const void *blob, size_t len) {
    /* Create shared-memory file descriptor */
    int fd = shm_open(SHM_ANON, O_RDWR, 0);
    ftruncate(fd, len);
    /* MemMap file descriptor, and load data */
    void *mem = mmap(NULL, len, PROT_WRITE, MAP_SHARED, fd, 0);
    memcpy(mem, blob, len);
    munmap(mem, len);
    /* Open Dynamic Library from SHM file descriptor */
    void *so = fdlopen(fd,RTLD_LAZY);
    close(fd);
    return so;
}

Obviously the code lacks any kind of error checking etc, but this is the core functionality.

ETA: My initial assumption that fdlopen is POSIX was wrong, this appears to be a FreeBSD-ism.

Answer 2

I don't see why you'd be considering dlopen, since that will require a lot more nonportable code to generate the right object format on disk (e.g. ELF) for loading. If you already know how to generate machine code for your architecture, just mmap memory with PROT_READ|PROT_WRITE|PROT_EXEC and put your code there, then assign the address to a function pointer and call it. Very simple.

Answer 3

There is no standard way to do it other than writing out the file and then loading it again with dlopen().

You may find some alternative method on your current specific platform. It will up to you to decide whether that is better than using the 'standard and (relatively) portable' approach.

Since generating the object code in the first place is rather platform specific, additional platform-specific techniques may not matter to you. But it is a judgement call - and in any case depends on there being a non-standard technique, which is relatively improbable.

Answer 4

We implemented a way to do this at Google. Unfortunately upstream glibc has failed to comprehend the need so it was never accepted. The feature request with patches has stalled. It's known as dlopen_from_offset.

The dlopen_with_offset glibc code is available in the glibc google/grte* branches. But nobody should enjoy modifying their own glibc.

Answer 5

Loading the solib from memory has an inherent limitation. Namely, the DT_NEEDED deps of a solib cannot refer to the memory buffer. This means, among other things, that you can't easily load solib with deps from the memory buffer. I am afraid, unless the ELF specification is extended to allow DT_NEEDED to refer to other objects than the file names, there will be no standard API for loading the solib from memory buffer.

I think you need to use posix's shm_open(), then mmap the shared memory, generate your solib there, then use plain dlopen() via /dev/shm mount point. This way also the deps can be handled: they can refer to the regular files or to the /dev/shm objects that have your generated solibs.

Answer 6

Here's how you can do it entirely in-memory on Linux (no writing to /tmp/xxx) using a memory fd with memfd_create:

user@system $ ./main < example-library.so
add(1, 2) = 3

// example-library.c
int add(int a, int b) { return a + b; }

#include <cstdio>
#include <dlfcn.h>
#include <fcntl.h>
#include <sys/mman.h>
#include <sys/syscall.h>
#include <unistd.h>
#include <vector>

// Compile and then invoke as:
// $ ./main < my-shared-lib.so
int main() {
  // Read the shared library contents from stdin
  std::vector<char> library_contents;
  char buffer[1024];
  ssize_t bytes_read;
  while ((bytes_read = read(STDIN_FILENO, buffer, sizeof(buffer))) > 0) {
    library_contents.insert(library_contents.end(), buffer,
                            buffer + bytes_read);
  }

  // Create a memory file descriptor using memfd_create
  int fd = memfd_create("shared_library", 0);
  if (fd == -1) {
    perror("memfd_create failed");
    return 1;
  }

  // Write the shared library contents to the file descriptor
  if (write(fd, library_contents.data(), library_contents.size()) !=
      static_cast<ssize_t>(library_contents.size())) {
    perror("write failed");
    return 1;
  }

  // Create a path to the file descriptor using /proc/self/fd
  // https://sourceware.org/bugzilla/show_bug.cgi?id=30100#c33
  char path[100]; // > 35 == strlen("/proc/self/fd/") + log10(pow(2, 64)) + 1
  snprintf(path, sizeof(path), "/proc/self/fd/%d", fd);

  // Use dlopen to dynamically load the shared library
  void *handle = dlopen(path, RTLD_NOW);
  if (handle == NULL) {
    fprintf(stderr, "dlopen failed: %s\n", dlerror());
    return 1;
  }

  // Use the shared library...
  // Get a pointer to the function "int add(int, int)"
  int (*add)(int, int) =
      reinterpret_cast<int (*)(int, int)>(dlsym(handle, "add"));

  if (add == NULL) {
    fprintf(stderr, "dlsym failed: %s\n", dlerror());
    return 1;
  }

  // Call the function "int add(int, int)"
  printf("add(1, 2) = %d\n", add(1, 2));

  // Cleanup
  dlclose(handle);
  close(fd);
  return 0;
}

Answer 7

You don't need to load the code generated in memory, since it is already in memory!

However, you can -in a non portable way- generate machine code in memory (provided it is in a memory segment mmap-ed with PROT_EXEC flag).

^{(in that case, no "linking" or relocation step is required, since you generate machine code with definitive absolute or relative addresses, in particular to call external functions)}

Some libraries exist which do that: On GNU/Linux under x86 or x86-64, I know of GNU Lightning (which generates quickly machine code which runs slowly), DotGNU LibJIT (which generates medium quality code), and LLVM & GCCJIT (which is able to generate quite optimized code in memory, but takes time to emit it). And LuaJit has some similar facility too. Since 2015 GCC 5 has a gccjit library.

And of course, you can still generate C code in a file, fork a compiler to compile it into a shared object, and dlopen that shared object file. I'm doing that in GCC MELT , a domain specific language to extend GCC. It does work quite well in practice.

addenda

If performance of writing the generated C file is a concern (it should not be, since compiling a C file is much slower than writing it) consider using some tmpfs file system for that (perhaps in /tmp/ which is often a tmpfs filesystem on Linux)

Answer 8

I found a solution to this, creating a memory file using memfd_create and then opening from dlopen.

Answer 9

It should be noted that using shm_open+dlopen loads the dynamic library from shared memory, if/dev/shm has the permission of noexec, the dynamic library will fail to load.