Read a file into a string at compile-time

Question

I would like to write something in a file (let's call it foo.cpp) and include it as a string into my program at compile time, similar to the way #include does it.

Right now I'm using this C preprocessor #define:

#define toString(src) #src

to convert a bunch of code to a string, used as in this example:

const char* str = toString(
  int x;
  void main(){}
);

You can read about macro stringification there if you want.

I would like to move that code to an external file, which would be "linked" at compile-time. I don't want the file to have to be distributed with the program, which would be the case if I were to read it a run-time.

I tried to use an #include directive as shown below, but the compiler refused it:

const char* str = toString(
#include "foo.cpp"
);

g++ seems to be completely confused, but clang++ gave me this error:

error: embedding a #include directive within macro arguments is not supported

Does anyone know if/how this can be done?

Note: I'm using this to write my GLSL shaders, although I doubt this information is of any use.

PS: Before you tell me this is a duplicate of this question, putting my code in a giant string in its own file or using an external tool (eg. xxd) to dump its hex representation are not "solutions" for me, as they are not better (ie. easier/cleaner) than my current method.

---

Update, a few years later:
I just realized I never got to answer this question as it was closed as duplicate. I found the answer I was looking for when I saw this commit, itself based on a comment on this article, and have been using it ever since.

In a nutshell, a small assembly file includes the files you want and exposes them under a given NAME, with the three variables NAME_begin, NAME_end and NAME_len allowing you to access their contents from your C code.

This way, you have a normal file containing nothing but the code you want, and it gets read automatically at compile time, instead of having to read it at runtime or having to jump through xxd hoops.

Answer 1

I am not quite sure what you are trying to accomplish but but Linux command line utility xxd may be what you are looking for:

xxd -i [filename]

will generate a C style header file containing an array with your file contents in full binary encoding and a variable with its length.

Example:

xxd -i /proc/cpuinfo

makes a file with

unsigned char _proc_cpuinfo[] = {
  0x70, 0x72, 0x6f, 0x63, 0x65, 0x73, 0x73, 0x6f, 0x72, 0x09, 0x3a, 0x20,
  0x30, 0x0a, 0x76, 0x65, 0x6e, 0x64, 0x6f, 0x72, 0x5f, 0x69, 0x64, 0x09,
...
};
unsigned int _proc_cpuinfo_len = 654390;

You can include the resulting header in your code and access the array and file length via those variables.

Answer 2

You say you don't like xxd because it leaves the file unreadable. Fair enough. It would be simple to write your own utility that encodes the data in a different format, since you specifically want strings as input and output. You can take advantage of string literal concatenation to make this easily readable.

const char* str =
#include "foo.h"
    ;

foo.h:

"  int x;" "\n"
"  void main(){}" "\n"

I briefly tried using C++11's raw string literals, which would have allowed using the file without any reformatting, but it did not work. The #include is considered to be part of the string rather than including the file as desired.

Answer 3

The simplest thing in this sort of case is to write a small preprocessor, which reads your file, and outputs it wrapping each line in quotes. I'd probably do this in Python, but it is pretty straight forward in C++ as well. Supposing that you've got inputFile, outputFile and variableName from somewhere (probably argv, but you might want to derive the latter two from the input filename:

void
wrapFile( std::istream& inputFile,
          std::ostream& outputFile,
          std::string const& variableName )
{
    outputFile << "extern char const " << variableName << "[] =\n";
    std::string line;
    while ( std::getline( inputFile, line ) ) {
        outputFile << "    \"" << line << "\\n\"\n";
    }
    std::outputFile << ";" << std::endl;
}

Depending on what is in the files you're including, you might have to mangle the line before outputting it, to escape things like " or \.

If you wanted to get fancy, you could add some tests to insert the semicolon on the last wrapped line, rather than a line of its own, but that's not really necessary.

This will result in a '\0' terminated string. Given the probable length of the strings, it might be preferable to add a second variable with the length:

std::outputFile << "extern int const "
                << variableName
                << "_len = sizeof(" << variableName << ") - 1;\n";

(Don't forget the -1, since you don't want to count the terminating '\0' that the compiler will add to your string literal.) If you're including the generated file where it will be used, you don't necessarily need this, since std::begin and std::end will provide the necessary information (but again, don't forget to use std::end( variableName ) - 1, to ignore the '\n').

If you're using make, it's fairly easy to make your generated file depend on the file being wrapped, and the executable which does the wrapping (which in turn depends on the source above, etc.). With Visual Studios, you'll need to create a separate project for the wrapping code, if you write it in C++ (one of the reasons I'd use Python for this), and you'll likely have some problems with dependency management; Visual Studios isn't really designed for professional work (where large blocks of code are regularly generated using such techniques).