How to embed a file into an executable file?

Question

I have two problems, the first has been solved.

Current problem
If I embed a file that requires a library to load it, such as a jpeg image or a mp3 music, I will need to use the file as input to the library. However, each library is different and uses a way to get a file as input, the input may be the file name or a FILE* pointer (from libc's file interface).

I would like to know how to access an embedded file with a name. It will be inefficient if I create a temporary file, is there another way? Can I map a file name to memory? My platforms are Windows and Linux.

If show_file(const char* name) is a function from a library, I will need a string to open the file.
I have seen these questions:
How to get file descriptor of buffer in memory?
Getting Filename from file descriptor in C

and the following code is my solution. Is it a good solution? Is it inefficient?

# include <stdio.h>
# include <unistd.h>

extern char _binary_data_txt_start;
extern const void* _binary_data_txt_size;
const size_t len = (size_t)&_binary_data_txt_size;

void show_file(const char* name){
    FILE* file = fopen(name, "r");
    if (file == NULL){
        printf("Error (show_file): %s\n", name);
        return;
    }

    while (true){
        char ch = fgetc(file);
        if (feof(file) )
            break;
        putchar( ch );
    }
    printf("\n");

    fclose(file);
}

int main(){

    int fpipe[2];
    pipe(fpipe);

    if( !fork() ){
        for( int buffsize = len, done = 0; buffsize>done; ){
            done += write( fpipe[1], &_binary_data_txt_start + done, buffsize-done );
        }
        _exit(0);
    }

    close(fpipe[1]);
    char name[200];
    sprintf(name, "/proc/self/fd/%d", fpipe[0] );

    show_file(name);

    close(fpipe[0]);
}

---

The other problem (solved)
I tried to embed a file on Linux, with GCC, and it worked. However, I tried to do the same thing on Windows, with Mingw, and it did not compile.

The code is:

# include <stdio.h>

extern char _binary_data_txt_start;
extern char _binary_data_txt_end;

int main(){
    for (char* my_file = &_binary_data_txt_start; my_file <= &_binary_data_txt_end; my_file++)
        putchar(*my_file);
    printf("\n");
}

The compilation commands are:

objcopy --input-target binary --output-target elf32-i386 --binary-architecture i386 data.txt data.o
g++ main.cpp data.o -o test.exe

On Windows, I get the following compiler error:

undefined reference to `_binary_data_txt_start'
undefined reference to `_binary_data_txt_end'

I tried to replace elf32-i386 with i386-pc-mingw32, but I still get the same error.

Answer 1

I think that for this to work with MinGW you'll need to remove the leading underscore from the names in the .c file. See Embedding binary blobs using gcc mingw for some details.

See if using the following helps:

extern char binary_data_txt_start;
extern char binary_data_txt_end;

If you need the same source to work for Linux or MinGW builds, you might need to use the preprocessor to have the right name used in the different environments.

Answer 2

If you're using a library that requires a FILE* for reading data, then you can use fmemopen(3) to create a pseudofile out of a memory blob. This will avoid creating a temporary file on disk. Unfortunately, it's a GNU extension, so I don't know if it's available with MinGW (likely not).

However, most well-written libraries (such as libpng and the IJG's JPEG library) provide routines for opening a file from memory as opposed to from disk. libpng, in particular, even offers a streaming interface, where you can incrementally decode a PNG file before it's been completely read into memory. This is useful if, say, you're streaming an interlaced PNG from the network and you want to display the interlaced data as it loads for a better user experience.

Answer 3

On Windows, you can embed custom resource into executable file. You would need a .RC file and a resource compiler. With Visual Studio IDE you can do it without hassle.

In your code, you would use FindResource, LoadResource and LockResource functions to load the contents into memory at runtime. A sample code that reads the resource as long string:

void GetResourceAsString(int nResourceID, CStringA &strResourceString)
{
    HRSRC hResource = FindResource(NULL, MAKEINTRESOURCE(nResourceID),  L"DATA");

    HGLOBAL hResHandle = LoadResource(NULL, hResource);

    const char* lpData = static_cast<char*> ( LockResource(hResHandle) );

    strResourceString.SetString(lpData, SizeofResource(NULL, hResource));

    FreeResource(hResource);
}

Where nResourceID is the ID of resource under custom resource type DATA. DATA is just a name, you may choose another name. Other in-built resources are cursors, dialogs, string-tables etc.

Answer 4

I've created a small library called elfdataembed which provides a simple interface for extracting/referencing sections embedded using objcopy. This allows you to pass the offset/size to another tool, or reference it directly from the runtime using file descriptors. Hopefully this will help someone in the future.

It's worth mentioning this approach is more efficient than compiling to a symbol, as it allows external tools to reference the data without needing to be extracted, and it also doesn't require the entire binary to be loaded into memory in order to extract/reference it.

Answer 5

Use nm data.o to see what it named the symbols. It may be something as simple as the filesystem differences causing the filename-derived symbols to be different (eg filename capitalized).

Edit: Just saw your second question. If you are using threads you can make a pipe and pass that to the library (first using fdopen() if it wants a FILE *). If you are more specific about the API you need to talk to I can add more specific advice.