43

I am wondering if it is possible in C++11/14 to actually read files at compile time. For example the following code will only compile if it can successfully read the file.

constexpr std::string shader_source = load("~/foo.glsl");

Do you think this could be possible?

I know that I could do this with some custom tool when building my application.

Maik Klein
  • 15,548
  • 27
  • 101
  • 197
  • 7
    It depends on what you wean by *load*. The only solution I know of is `#include`, and it means that the content of the file must be understand by the compiler. – Didier Trosset Oct 02 '14 at 13:02
  • 7
    What you probably want is an external variable declaration, which will be matched to the actual data by the linker. Combine that with a tool that embeds an arbitrary binary resource into an object file with an exported symbol of your choice, and you're golden. (I could suggest such a tool, but you didn't mention what toolchain you are using, and the tools are different for ELF vs CV vs PE vs a.out object file formats) Note that converting the resource to C/C++ code as a constant array initialized with hexadecimal literals is likely to be very very slow, so go straight to object file. – Ben Voigt Oct 02 '14 at 13:04
  • I can't believe that your `load` function is an `constexpr`. So the complete expression can't be constexpr. I have no idea how a external file content can be a constexpr string. The only way is to use a tool which simple loads the file and generate c/c++ code. – Klaus Oct 02 '14 at 13:12
  • 1
    Given C++ doesn't provide a way to do this, you're getting answers assuming a particular compiler/linker - you might want to specify your actual OS/portability needs. – Tony Delroy Oct 02 '14 at 14:02
  • yeah, the c++ source file can be read – weima Oct 03 '14 at 13:52

4 Answers4

24

Building on teivaz's idea, I wonder if the usual "stringize after expansion" trick will work:

#define STRINGIZE(...) #__VA_ARGS__
#define EXPAND_AND_STRINGIZE(...) STRINGIZE(__VA_ARGS__)

constexpr std::string shader_source = EXPAND_AND_STRINGIZE(
#include "~/.foo.glsl"
);

Still, I would go for a conventional extern const char[] declaration resolved to the content by the linker. The article "Embedding a File in an Executable, aka Hello World, Version 5967" has an example:

# objcopy --input binary \
          --output elf32-i386 \
          --binary-architecture i386 data.txt data.o

Naturally you should change the --output and --binary-architecture commands to match your platform. The filename from the object file ends up in the symbol name, so you can use it like so:

#include <stdio.h>

/* here "data" comes from the filename data.o */
extern "C" char _binary_data_txt_start;
extern "C" char _binary_data_txt_end;

main()
{
    char*  p = &_binary_data_txt_start;

    while ( p != &_binary_data_txt_end ) putchar(*p++);
}
Ben Voigt
  • 277,958
  • 43
  • 419
  • 720
11
#define STR(x) #x

const char* a =
{ 
#include "foo.glsl" 
};

and foo.glsl should enclose its content in STR( ... )

upd. This will properly handle commas

#define STRINGIFY(...) #__VA_ARGS__
#define STR(...) STRINGIFY(__VA_ARGS__)
Teivaz
  • 5,462
  • 4
  • 37
  • 75
  • 6
    Preprocessor declarations such as `#include` have to be the first non-whitespace on the line :( – Ben Voigt Oct 02 '14 at 13:07
  • But see http://stackoverflow.com/a/5566624/103167 . It will be ok to put a newline between the parentheses and the `#include`, as long as the data format is tolerant of those extra newlines appearing in the string (GLSL is tolerant). – Ben Voigt Oct 02 '14 at 13:09
  • Hmm, your edit fixed the preprocesssor invocation but broke stringizing. – Ben Voigt Oct 02 '14 at 13:10
  • @NeilKirk: Probably because it should be bare GLSL source code for compatibility with shader debuggers. – Ben Voigt Oct 02 '14 at 13:11
  • 1
    That use of the `STR` macro wouldn't work if the file contains commas. I think it would also change the whitespace in the file. – interjay Oct 02 '14 at 13:13
  • damn, that's clever – CoffeeTableEspresso Aug 06 '19 at 17:24
  • Does this work for arbitrary file types being treated as a string? I ran into some trouble with this trying to load an SSL cert/key file this way - compiler doesn't like the file contents. I was hoping it would just be a pure string, as these are basically .txt files. – netpoetica Mar 04 '21 at 13:28
1

I have done something like this. See if this will give you what you want.

Add a command line option to the program that checks for the existence and validity of the input file.
That option should exit the program with an error code if the file does not exist, or is not valid.

In your make file, add a call to the program (using that command line option), as the final build step.

Now when you build the program, you will get an error if the proper files are not available or not valid.

Louis Newstrom
  • 172
  • 1
  • 1
  • 13
0

This is my C solution, but for C++ is the same. Stringify #define does not work with #include in my case with modern compilers, so you can use raw string literals inside your file to embed the content inside a string at compile time.

test.c:

#include <stdio.h>

const char *text = {
#include "test.dat"
};

int main() {
    printf("%s\n", text);
}

test.dat:

R"(This is a line.
This is another line...)"
apopa
  • 405
  • 2
  • 12