9

I would like to have a small "application loader" program that receives other binary application files over TCP from an external server and runs them.

I could do this by saving the transmitted file to the hard disk and using the system() call to run it. However, I am wondering if it would be possible to launch the new application from memory without it ever touching the hard drive.

The state of the loader application does not matter after loading a new application. I prefer to stick to C, but C++ solutions are welcome as well. I would also like to stick to standard Linux C functions and not use any external libraries, if possible.

Flip
  • 93
  • 1
  • 4
  • Yes, it's possible but it's somewhat complicated. You have to emulate the OS and map the binary to memory, etc. – Seth Carnegie May 09 '12 at 20:37
  • 2
    You could write it to a file on a ramdisk – TJD May 09 '12 at 20:40
  • 1
    I also tend to think that *any* level of OS security on a moderately recent CPU is going to do its best to ensure that this can't happen. It's certainly doable, but would be a real pain to use on real-world distributions (I hope) – BRPocock May 09 '12 at 20:43
  • (That said, you might look at what `glibc`'s `ld-linux` does, in fact, do, since this is what it does for normal, on-disc executables. It's not pretty…) – BRPocock May 09 '12 at 20:44
  • @BRPocock: actually [it's done very often](http://en.wikipedia.org/wiki/Executable_compression). – Yakov Galka May 09 '12 at 20:45
  • @ybungalobill - That's a case of an application unpacking itself into its own memory. Having it unload into another application would be a different story. – Mr. Llama May 09 '12 at 20:51
  • @GigaWatt: Read the question again: "The state of the loader application does not matter after loading a new application." so it's actually what the OP wants. So he can look at some of the opensource packers on the page I linked. – Yakov Galka May 09 '12 at 20:53
  • @GigaWatt: besides you can always spawn a child process that loads the program into its own memory. The effect is the same. – Yakov Galka May 09 '12 at 20:55
  • @ybungalobill Flashbacks to early-90's embedded work :-) But, last time I knew, UPX (at least) actually wrote out the decompressed file to `/tmp` and ran it… – BRPocock May 09 '12 at 21:54

3 Answers3

6

Short answer: no.

Long answer: It's possible but rather tricky to do this without writing it out to disk. You can theoretically write your own elf loader that reads the binary, maps some memory, handles the dynamic linking as required, and then transfers control but that's an awful lot of work, that's hardly ever going to be worth the effort.

The next best solution is to write it to disk and call unlink ASAP. The disk doesn't even have to be "real" disk, it can be tmpfs or similar.

The alternative I've been using recently is to not pass complete compiled binaries around, but to pass LLVM bytecode instead, which can then be JIT'd/interpreted/saved as fit. This also has the advantage of making your application work in heterogeneous environments.

It may be tempting to try a combination of fmemopen, fileno and fexecve, but this won't work for two reasons:

  1. From fexecve() manpage:

    "The file descriptor fd must be opened read-only, and the caller must have permission to execute the file that it refers to"

    I.e. it needs to be a fd that refers to a file.

  2. From fmemopen() manpage:

    "There is no file descriptor associated with the file stream returned by these functions (i.e., fileno(3) will return an error if called on the returned stream)"

Flexo
  • 87,323
  • 22
  • 191
  • 272
  • Thanks for the response; this is exactly what I was looking for. Googling the topic didn't get me very far and I needed to satisfy my curiosity! – Flip May 10 '12 at 00:43
  • Note that it is something of a standard these days to have a `tmpfs` mounted at `/dev/shm/`. – caf May 10 '12 at 06:10
  • It's now possible to run in-memory executable by using memfd_create() and fexecve(). See https://unix.stackexchange.com/questions/230472/can-i-exec-an-entirely-new-process-without-an-executable-file – ArtemB Feb 04 '19 at 07:29
0

Much easier than doing it is C would just to set up a tmpfs file system. You'd have all the advantages of the interface of a harddisk, from your program / server / whatever you could just do an exec. These types of virtual filesystems are quite efficient nowadays, there would be really just one copy of the executable in the page cache.

As Andy points out, for such scheme to be efficient you'd have to ensure that you don't use buffered writes to the file but that you "write" (in a broader sense) directly in place.

  • you'd have to know how large your executable will be
  • create a file on your tmpfs
  • scale it to that size with ftruncate
  • "map" that file into memory with mmap to obtain the addr of a buffer
  • pass that address directly to the recv call to write the data in place
  • munmap the file
  • call exec with the file
  • rm the file. can be done even when the executable is still running
Jens Gustedt
  • 76,821
  • 6
  • 102
  • 177
  • Honestly tmpfs is going to be no faster than a native filesystem unless your storage is already at capacity. Writes get buffered to memory such that an immediate execve() will simply map them out of cache. The only cost you pay is the eventual flush to disk, which is asynchronous. – Andy Ross May 09 '12 at 21:10
  • @AndyRoss, don't use buffered writes for such a thing, I'll update my answer. – Jens Gustedt May 09 '12 at 21:25
  • 1
    Even that just avoids a memory copy. What's the performance constraint here? I'm having trouble imagining a situation where the speed of writes through the filesystem is a limit but the speed of receiving packets over a network is not. Seems vastly overcomplicated. Just open/write to your regular filesystem and save the trouble. – Andy Ross May 09 '12 at 21:49
0

You might want to look at and reuse UPX, which decompresses the executable to memory, and then transfers control to ld-linux to start it.

Employed Russian
  • 199,314
  • 34
  • 295
  • 362