15

Sorry that this will be a long post, but I feel like you need all of the code to see what's going on.


So, I have been experimenting with an idea for compile time string to data structure parser. Think of something like a regex, where the string is "compiled" into a data structure at compile time but executed at runtime (so long as the input string is a constant of course). But I've run into an issue that I don't quite understand what's wrong:

Basically, my design is a 2 pass parser:

  • Pass 1: determine how many "opcodes" are in the input string
  • Pass 2: return an array whose size is determined by Pass 1, and filled in with the "opcodes"

Here's what things look like:

// a class to wrap string constants
class constexpr_string {
public:
    template <size_t N>
    constexpr constexpr_string(const char (&s)[N]) : string_(s), size_(N - 1) {}
public:
    constexpr size_t size() const     { return size_; }
    constexpr size_t capacity() const { return size(); }
    constexpr size_t empty() const    { return size() != 0; }
public:
    constexpr char operator[](size_t n) const { return string_[n]; }
private:
    const char *string_;
    size_t      size_;
};

// would have loved to use std::array, but ran into an issue so..
// wrapped in a struct so we can return it
template <class T, size_t N>
struct constexpr_array {
    T array[N] = {};
};

struct opcode { /* not relevant */ };

template <size_t N>
constexpr constexpr_array<opcode, N> compile_string(constexpr_string fmt) {
    constexpr_array<opcode, N> compiled;
    /* fill in compiled_format */
    return compiled;
}

constexpr size_t calculate_size(constexpr_string fmt) {
    size_t size = 0;
    /* calculate size */
    return size;
}

#if 0
// NOTE: Why doesn't **This** work?
constexpr int test(constexpr_string input) {

    constexpr size_t compiled_size = calculate_size(input);
    constexpr auto compiled_format = compile_string<compiled_size>(input);
    return 0;
}
#endif

int main() {
    // NOTE: when this works...
    constexpr char input[] = "...";
    constexpr size_t compiled_size = calculate_size(input);
    constexpr auto compiled = compile_string<compiled_size>(input);
    execute(compiled); // run it!
}

So far so good!

The problem arises when I try to just wrap those 2 lines into a function :-/. I don't understand why the same exact code works in main, but if I just try to pass the same constexpr object to another function, I start getting errors about things not being constexpr.


Here's the error message:

main.cpp: In function ‘constexpr int test(constexpr_string)’:
main.cpp:258:55: error: ‘input’ is not a constant expression
  constexpr size_t compiled_size = calculate_size(input);
                                                       ^
main.cpp:259:70: error: no matching function for call to ‘compile_string<compiled_size>(constexpr_string&)’
  constexpr auto compiled_format = compile_string<compiled_size>(input);
                                                                      ^
main.cpp:60:45: note: candidate: template<long unsigned int N> constexpr constexpr_array<opcode, N> compile_string(constexpr_string)
 constexpr constexpr_array<opcode, N> compile_string(constexpr_string fmt) {
                                             ^~~~~~~~~~~~~~
main.cpp:60:45: note:   template argument deduction/substitution failed:
Tas
  • 7,023
  • 3
  • 36
  • 51
Evan Teran
  • 87,561
  • 32
  • 179
  • 238
  • Since this question is tagged with c++17, have you ever considered using `std::string_view` in your implementation? `std::string_view` is build with `constexpr` in mind, so it is definitely worth looking. https://en.cppreference.com/w/cpp/header/string_view – HugoTeixeira Aug 08 '18 at 02:03
  • @HugoTeixeira Definitely. My plan is/was to switch to a string_view once I had the core concepts of what I was experimenting with working how I want. – Evan Teran Aug 08 '18 at 02:05
  • Is there any reason you have to use a string? Representing regex rules (e.g. the kleene star) using operators or objects would probably be significantly easier. I believe that's what Boost.Spirit does. – Pharap Aug 08 '18 at 02:05
  • well, "regex" is was intended as an analogy for what I want to do. As in: string -> state -> execute_state. I was hoping for a nice way to do the first part at compile time... It looks like it will be a bit complicated to do so though. – Evan Teran Aug 08 '18 at 02:07
  • You may be interested in the experimental **`constexpr` JSON parser** by Ben Deane and Jason Turner. Their experiments are titled *constexpr all the things*. There have been talks at [C++Now 2017](https://cppnow2017.sched.com/event/A8IX/constexpr-all-the-things) (https://youtu.be/HMB9oXFobJc) and [CppCon 2017](https://cppcon2017.sched.com/event/Bgt7/constexpr-all-the-things) (https://youtu.be/PJwd4JLYJJY). The [source code is on GitHub](https://github.com/lefticus/constexpr_all_the_things), but I recommend watching the talk. – Julius Aug 08 '18 at 06:14
  • The `/* fill in compiled_format */` comment inside `compile_string` should say `/* fill in compiled */` instead...? – user202729 Aug 08 '18 at 07:52

1 Answers1

19

Let's reduce this:

constexpr void f(int i) {
    constexpr int j = i; // error
}

int main() {
    constexpr int i = 0;
    constexpr int j = i; // OK
}

Function parameters are never constexpr, so i inside f is not a constant expression and can't be used to initialize j. Once you pass something through a function parameter, the constexpr-ness is lost.

T.C.
  • 133,968
  • 17
  • 288
  • 421
  • That certainly sounds about right :-/ Certainly I've read about people doing clever tricks to do compile time parsing using `constexpr`, is there something people are doing to get around this? – Evan Teran Aug 07 '18 at 23:40
  • 1
    The general workaround is to get the value into a template non-type parameter (or a pack thereof). For string literals this is particularly difficult and AFAIK requires a non-standard extension (supported by GCC and Clang only) before C++20. – T.C. Aug 07 '18 at 23:45
  • OK, sounds like I need to look into things like this: https://stackoverflow.com/questions/15858141/conveniently-declaring-compile-time-strings-in-c/15863804#15863804 – Evan Teran Aug 07 '18 at 23:46
  • 2
    OK, I think you're answer is fundamentally right... but your reduced code is not quite representative. I'm looking at something which makes no sense to me right now: this works: constexpr size_t Func(constexpr_string input) { if(input[0] == 'X') { return 1; } return 0; } this doesn't: constexpr size_t Func(constexpr_string input) { constexpr char ch = input[0]; if(ch == 'X') { return 1; } return 0; } – Evan Teran Aug 08 '18 at 00:14
  • Again, `input` isn't constexpr. You can branch on non-constexpr values just fine. You just can't use it to initialize a `constexpr` variable. – T.C. Aug 08 '18 at 01:17
  • Yea, that makes sense. I suppose my confusion is coming from the fact that I have a function which is constexpr, returning a compile time size and calculating it based on a constexpr_string parameter. That seems to work just fine somehow... (I am using it as a template parameter the array size). – Evan Teran Aug 08 '18 at 01:19
  • In other words, why does my "calculate_size" work? and be able to be used as a static array size. – Evan Teran Aug 08 '18 at 01:20
  • 3
    `constexpr size_t compiled_size = calculate_size(input);` is valid because `input` is constexpr here and `calculate_size(input)`, considered as a whole, is a valid constant expression. Inside `calculate_size`, its parameter isn't constexpr, but that doesn't stop the whole function call expression from being constexpr. – T.C. Aug 08 '18 at 01:32
  • Ahh, that is an interesting way to think about it. The **result** is constexpr, but parameter isn't. I think I have a clear picture. – Evan Teran Aug 08 '18 at 01:45
  • @EvanTeran consider that a `constexpr` function still needs to be able to be called at runtime in the case that any of its parameters aren't constant expressions. So the parameter may or may not be a constant expression, which means any code relying it definitely being one can't compile. – John Ilacqua Aug 08 '18 at 06:10
  • 1
    @T.C. Regarding *"The general workaround is to get the value into a template non-type parameter"* -- couldn't you use a `constexpr` lambda like `[]() constexpr { return "string"sv; }`? https://wandbox.org/permlink/Z2jEGcjmVB7fPwM2 – dyp Aug 08 '18 at 11:23