Extending the compiler to evaluate "complicated" function (with known input values) at compile time, out of `constexpr` reach

Question

Please take a look at this example. Trying to use constexpr to obtain a compile time evaluation would be utterly difficult, if not impossible.

However, where the function is called all arguments are known at compile time. In theory, (in presence of some optimization attribute?), the compiler could pause parsing, create a small program with the function, compile it, run it, and get the result to create a char constant to be used in the main program compilation.

I understand one of the problem would be for example cross compiling: you could only run a program from the compiler if the compiler can build a program that can run on the machine that is building. But this doesn't seem impossible to overcome.

There is no doubt: I am not the first one to think about it. But I tried to search and I only could found constexpr and template stuff. Is there any plan to implement such a feature in any compiler in the future? The feature could be also rephrased as: Give the possibility to write a C++ program within your C++ source that the compiler can compile and run upon request to create a constant.

Answer 1

The constexpr and template are the only ways in C++ to force the compiler to do something at compile time. In many cases the optimizer is able to figure out more complicated functions as well. However, you may inadvertely step away from the compiler's ``comfort zone'', e.g. through some allocation and/or pointer arithmetic.

As Marc Glisse points out in the comments, in your concrete example you are using std::string rather than a native char* and due to magic done within the implementation of that class, the optimizer may get lost.

That being said, you may welcome C++14 which relaxes the requirements for a function to be constexpr. You can now declare local variables, have mutating objects and have basic control flow structures. You can also manipulate raw strings too.

https://isocpp.org/wiki/faq/cpp14-language#extended-constexpr

---

Looking at a broader picture, there is an ongoing research to bring staging to programming languages. Staging is an idea of a program being partially evaluated through an explicit code constructs, allowing even most complex functions to be executed at compile time. Some example approaches to the problem, taking from different angles:

• MetaML
• LMS
• Online Partial Evaluation
• Dynamic Staging

Note that neither of those tries to do that in C++ though.

Answer 2

You basically want something different than what standard C++ provides. Notice that language standards do not define what optimizations are required, but only most of what optimizations are legitimate.

CyghusX1 answer give some general hints.

If you are on POSIX (e.g. Linux), you could also generate some specialized C++ code at "runtime", fork a compilation of it, and use dynamic loading facilities (i.e. dlopen) on that generated "plugin"; to be more specific:

• have some model of the AST of the subset of C++ you want to generate
• compile and link your main program with -rdynamic -ldl
• define some conventions regarding how to call your plugin, e.g. deciding that you want some extern "C" int foo(const char*) function in it.
• write some code declaring types for that AST
• write the code emitting that AST as C++ code in some file, the generated C++ code should follow your plugin convention (so would provide the int foo(const char*) function you want; avoid name mangling)
• concretely, emit the C++ code in a temporary file /tmp/temp1234.cc
• fork its compilation as a shared object by running something like g++ -O -Wall -fPIC /tmp/temp1234.cc -shared -o /tmp/temp1234.so, e.g. with system(3) or something else
• call dlopen(3) on /tmp/temp1234.so with RTLD_NOW|RTLD_GLOBAL; check against failure (otherwise, use dlerror())
• call dlsym on the handle provided by dlopen with the foo external name. This gives a function pointer, which you can call later.

On Linux, you could repeat this many times (practically, many hundred thousands times at least, see my manydl.c example). My GCC MELT tool is doing such things.

Actually, GCC MELT is for customizing and extending the GCC compiler, and you could perhaps even use for your goals, since it enable customization of GCC; so you might add some new attributes or pragmas marking the functions you would want to specialize and code the MELT extensions doing that specialization (you'll need to understand GCC internals, notably Gimple, so that would take you more than a week, and your __COMPACT_PRETTY_FUNCTION__ could use some of your additional GCC builtin provided by your MELT extension).

So have something like

#define __COMPACT_PRETTY_FUNCTION__ _builtin_compact_pretty_function()

and code the MELT extension implementing your new _builtin_compact_pretty_function compiler builtin.