C++ Boost qi recursive rule construction

Question

[It seems my explanations and expectations are not clear at all, so I added precision on how I'd like to use the feature at the end of the post]

I'm currently working on grammars using boost qi. I had a loop construction for a rule cause I needed to build it from the elements of a vector. I have re-written it with simple types, and it looks like:

#include <string>

// using boost 1.43.0
#include <boost/spirit/include/qi.hpp>
#include <boost/spirit/include/qi_eps.hpp>
#include <boost/spirit/include/phoenix.hpp>

namespace bqi = boost::spirit::qi;

typedef const char* Iterator;

// function that you can find [here][1]
template<typename P> void test_phrase_parser(char const* input, P const& p, bool full_match = true);

int main()
{
    // my working rule type:
    bqi::rule<Iterator, std::string()> myLoopBuiltRule;
    std::vector<std::string> v;
    std::vector<std::string>::const_iterator iv;

    v.push_back("abc");
    v.push_back("def");
    v.push_back("ghi");
    v.push_back("jkl");

    myLoopBuiltRule = (! bqi::eps);
    for(iv = v.begin() ; iv != v.end() ; iv++)
    {
        myLoopBuiltRule =
                myLoopBuiltRule.copy()  [ bqi::_val = bqi::_1 ]
                | bqi::string(*iv)      [ bqi::_val = bqi::_1 ]
                ;
    }
    debug(myLoopBuiltRule);

    char s[] = "  abc ";

    test_phrase_parser(s, myLoopBuiltRule);
}

(Looks like here does not want to be replaced by corresponding hyperlink, so here is the address to find function test_phrase_parser(): http://www.boost.org/doc/libs/1_43_0/libs/spirit/doc/html/spirit/qi/reference/basics.html)

All was for the best in the best of all worlds... until I had to pass an argument to this rule. Here is the new rule type:

    // my not-anymore-working rule type:
    bqi::rule<Iterator, std::string(int*)> myLoopBuiltRule;

'int*' type is for example purpose only, my real pointer is adressing a much more complex class... but still a mere pointer.

I changed my 'for' loop accordingly, i.e.:

    for(iv = v.begin() ; iv != v.end() ; iv++)
    {
        myLoopBuiltRule =
                myLoopBuiltRule.copy()(bqi::_r1)    [ bqi::_val = bqi::_1 ]
                | bqi::string(*iv)      [ bqi::_val = bqi::_1 ]
                ;
    }

I had to add a new rule because test_phrase_parser() cannot guess which value is to be given to the int pointer:

bqi::rule<Iterator> myInitialRule;

And change everything that followed the for loop:

myInitialRule = myLoopBuiltRule((int*)NULL);

debug(myLoopBuiltRule);

char s[] = "  abc ";

test_phrase_parser(s, myInitialRule);

Then everything crashed:

/home/sylvain.darras/software/repository/software/external/include/boost/boost_1_43_0/boost/spirit/home/qi/nonterminal/rule.hpp:199: error: no matching function for call to ‘assertion_failed(mpl_::failed************ (boost::spirit::qi::rule<Iterator, T1, T2, T3, T4>::operator=(const Expr&)

Then I got crazy and tried:

myLoopBuiltRule =
        myLoopBuiltRule.copy(bqi::_r1)  [ bqi::_val = bqi::_1 ]
        | bqi::string(*iv)      [ bqi::_val = bqi::_1 ]

-->

error: no matching function for call to ‘boost::spirit::qi::rule<const char*, std::string(int*), boost::fusion::unused_type, boost::fusion::unused_type, boost::fusion::unused_type>::copy(const boost::phoenix::actor<boost::spirit::attribute<1> >&)’

Then I got mad and wrote:

myLoopBuiltRule =
        myLoopBuiltRule(bqi::_r1)   [ bqi::_val = bqi::_1 ]
        | bqi::string(*iv)      [ bqi::_val = bqi::_1 ]

Which compiles since it is perfectly syntactically correct, but which magnificently stack overflows coz it happily, nicely, recursively, calls itself to death...

Then I lost my mind and typed:

myLoopBuiltRule =
        jf jhsgf jshdg fjsdgh fjsg jhsdg jhg sjfg jsgh df

Which, as you probably expect, has failed to compile.

You imagine that before writing the above novel, I checked out on the web, but didn't find out anything related to copy() and argument passing in the same time. Has anyone already experienced this problem ? Have I missed something ?

Be assured that any help will be really really appreciated.

PS: Great thanks to hkaiser who has, without knowing it, answered a lot of my boost::qi problems through google (but this one).

---

Further information:

The purpose of my parser is to read files written in a given language L. The purpose of my post is to propagate my "context" (i.e.: variable definitions and especially constant values, so I can compute expressions).

The number of variable types I handle is small, but it's bound to grow, so I keep these types in a container class. I can loop on these managed types.

So, let's consider a pseudo-algorithm of what I would like to achive:

LTypeList myTypes;
LTypeList::const_iterator iTypes;

bqi::rule<Iterator, LType(LContext*)> myLoopBuiltRule;

myLoopBuiltRule = (! bqi::eps);
for(iTypes = myTypes.begin() ; iTypes != myTypes.end() ; iTypes++)
{
    myLoopBuiltRule =
        myLoopBuiltRule.copy()(bqi::_r1)    [ bqi::_val = bqi::_1 ]
        | iTypes->getRule()(bqi::_r1)           [ bqi::_val = bqi::_1 ]
}

This is done during initialization and then myLoopBuiltRule is used and reused with different LContext*, parsing multiple types. And since some L types can have bounds, which are integer expressions, and that these integer expressions can exhibit constants, I (think that I) need my inherited attribute to take my LContext around and be able to compute expression value.

Hope I've been clearer in my intentions.

Answer 1

Note I just extended my answer with a few more informational links. In this particular case I have a hunch that you could just get away with the Nabialek trick and replacing the inherited attribute with a corresponding qi::locals<> instead. If I have enough time, I might work out a demonstration later.

Caveats, expositioning the problem

Please be advised that there are issues when copying proto expression trees and spirit parser expressions in particular - it will create dangling references as the internals are not supposed to live past the end of the containing full expressions. See BOOST_SPIRIT_AUTO on Zero to 60 MPH in 2 seconds!

Also see these answers which also concerns themselves with building/composing rules on the fly (at runtime):

• Generating Spirit parser expressions from a variadic list of alternative parser expressions
• Can Boost Spirit Rules be parameterized which demonstrates how to return rules from a function using boost::proto::deepcopy (like BOOST_SPIRIT_AUTO does, actually)

Nabialek Trick

In general, I'd very strongly advise against combining rules at runtime. Instead, if you're looking to 'add alternatives' to a rule at runtime, you can always use qi::symbols<> instead. The trick is to store a rule in the symbol-table and use qi::lazy to call the rule. In particular, this is known as the Nabialek Trick.

I have a toy command-line arguments parser here that demonstrates how you could use this idiom to match a runtime-defined set of command line arguments:

• https://gist.github.com/sehe/2a556a8231606406fe36

Limitations of qi::lazy, what's next?

Unfortunately, qi::lazy does not support inherited arguments see e.g.

• http://boost.2283326.n4.nabble.com/pass-inhertited-attributes-to-nabialek-trick-td2679066.html

You might be better off writing a custom parser component, as documented here:

• http://boost-spirit.com/home/articles/qi-example/creating-your-own-parser-component-for-spirit-qi/

^{I'll try to find some time to work out a sample that replaces inherited arguments by qi::locals later.}