spirit::qi: Combining parser_factory fails

Question

In a previous post (How do you create a generic parser using qi?) I was given some very good advice on how to create a parser-factory. Basic examples work fine, but I have problems combining factory-generated parsers. I would like to understand why my parsers fail and if my problem can be solved or I have to live with the current situation.

Example code (trimmed down as much as I could):

#include <boost/spirit/include/qi.hpp>
#include <boost/variant.hpp>
#include <string>


using namespace boost::spirit::qi;
using parse_iter = std::string::const_iterator;

//  Rudimentary test of a boost::spirit qi parser
template<class Parser>
void check_parser(Parser& p,std::string s)
{
    parse_iter first { s.begin() };
    parse_iter end { s.end() };
    bool ok = phrase_parse(first,end,p,space);
    std::cout << "Parsing [" << s << "]. Status: " 
        << (ok ? "OK\n": "Failed!\n");
}


//  Parser factory. Should have a static get() method
//  that returns something that can parse a valid T.
template<class T> struct parse_factory;

//  An example of using the factory
//  Specialising rule for int
template<>
struct parse_factory<int> 
{
    static int_type get() { return int_; } 
};


//  This ugly macro assures that we use the same rule for strings
//#define PARSE_STR as_string[lexeme[lit('"') >> *( ~char_("\""))>> lit('"')]| +alnum]
#define PARSE_STR (lexeme[lit('"') >> *( ~char_("\""))>> lit('"')]| +alnum)

template<>
struct parse_factory<std::string>
{
    static typename boost::proto::terminal<rule<parse_iter,std::string()>>::type get()
    {
        rule<parse_iter,std::string()> res;
//        debug(res);
        res = PARSE_STR;
        return res.copy();
    }
};


    //  Testing union
//  In case you wonder about naming: the real code is related to ASN.1 
//  parsing where "CHOICE" corresponds to a boost::variant
    using my_choice = boost::variant<int,std::string>;


template<>
struct parse_factory<my_choice>
{
    static boost::proto::terminal<rule<parse_iter,my_choice()>>::type get()
    {
        rule<parse_iter,my_choice()> r;
        r = int_ | PARSE_STR;
        return r.copy();
    }
};

void test_choice()
{
    auto choice_rule1 = int_ | PARSE_STR;

    check_parser(choice_rule1,"1");         //  Prints OK
    check_parser(choice_rule1,"Hello1");     //  Prints OK


    auto choice_rule2 = (parse_factory<int>::get() 
               | parse_factory<std::string>::get());

    check_parser(choice_rule2,"2");         //  Prints OK
    check_parser(choice_rule2,"Hello2");     //  Prints Failed! <=========================
    check_parser(parse_factory<std::string>::get(),"Hello");


    auto choice_rule3 = parse_factory<my_choice>::get();
    check_parser(choice_rule3,"3");         //  Prints OK
    check_parser(choice_rule3,"Hello3");     //  Prints OK


    auto choice_rule4 = int_ | parse_factory<std::string>::get();

    check_parser(choice_rule4,"4");         //  Prints OK
    check_parser(choice_rule4,"Hello4");     //  Prints Failed! <=========================


    auto choice_rule5 = parse_factory<int>::get() | PARSE_STR;

    check_parser(choice_rule5,"5");         //  Prints OK
    check_parser(choice_rule5,"Hello5");     //  Prints OK
}

int main() 
{
    test_choice(); 
}

So e.g. using rule #2

(parse_factory<int>::get() | parse_factory<std::string>::get()) 

causes the parsing to fail. I can successfully parse an int, but not a std::string. Update: If I change "get" for int, nothing compiles (both int and std::string input fails) when I use the factory-version for int. Unless I use the my_choice factory which continues to work. Interestingly, reversing the order of the alternatives changes nothing: I can parse the int, not the string. So - any ideas if/how this problem can be solved?

Answer 1

As I commented before, your program used auto-assignment of Proto expressions.

This means you copied temporaries, making the held references dangling.

That created UB.

You need boost::proto::deep_copy (or qi::copy in recent versions) to actually copy the expression template safely.

Here's a fixed version that works. Note the simplifications:

Live On Coliru

#define BOOST_SPIRIT_DEBUG
#include <boost/spirit/include/qi.hpp>

namespace qi = boost::spirit::qi;
using parse_iter = std::string::const_iterator;

//  Rudimentary test of a boost::spirit qi parser
template<class Parser>
void check_parser(Parser const& p,std::string s)
{
    parse_iter first { s.begin() };
    parse_iter end { s.end() };
    bool ok = qi::phrase_parse(first,end,p,qi::space);
    std::cout << "Parsing [" << s << "]. Status: " 
        << (ok ? "OK\n": "Failed!\n");
}


//  Parser factory. Should have a static get() method
//  that returns something that can parse a valid T.
template<class T> struct parse_factory;

//  An example of using the factory
//  Specialising rule for int
template<>
struct parse_factory<int> 
{
    static qi::int_type get() { return qi::int_; } 
};


//  This ugly macro assures that we use the same rule for strings
#define PARSE_STR (qi::lexeme['"' >> *~qi::char_('"') >> '"']| +qi::alnum)

template<>
struct parse_factory<std::string>
{
    //static typename boost::proto::terminal<rule<parse_iter,std::string()>>::type get()
    static typename qi::rule<parse_iter,std::string()> get()
    {
        qi::rule<parse_iter,std::string()> res;
        debug(res);
        res = PARSE_STR;
        return res;
    }
};


    //  Testing union
//  In case you wonder about naming: the real code is related to ASN.1 
//  parsing where "CHOICE" corresponds to a boost::variant
    using my_choice = boost::variant<int,std::string>;


template<>
struct parse_factory<my_choice>
{
    static qi::rule<parse_iter,my_choice()> get()
    {
        qi::rule<parse_iter,my_choice()> r;
        r = qi::int_ | PARSE_STR;
        return r;
    }
};

void test_choice()
{
    auto choice_rule1 = qi::copy(qi::int_ | PARSE_STR);

    check_parser(choice_rule1,"1");         //  Prints OK
    check_parser(choice_rule1,"Hello1");     //  Prints OK


    auto choice_rule2 = qi::copy(parse_factory<int>::get() 
            | parse_factory<std::string>::get());

    check_parser(choice_rule2,"2");         //  Prints OK
    check_parser(choice_rule2,"Hello2");     //  Prints Failed! <=========================
    check_parser(parse_factory<std::string>::get(),"Hello");


    auto choice_rule3 = qi::copy(parse_factory<my_choice>::get());
    check_parser(choice_rule3,"3");         //  Prints OK
    check_parser(choice_rule3,"Hello3");     //  Prints OK


    auto choice_rule4 = qi::copy(qi::int_ | parse_factory<std::string>::get());

    check_parser(choice_rule4,"4");         //  Prints OK
    check_parser(choice_rule4,"Hello4");     //  Prints Failed! <=========================


    auto choice_rule5 = qi::copy(parse_factory<int>::get() | PARSE_STR);

    check_parser(choice_rule5,"5");         //  Prints OK
    check_parser(choice_rule5,"Hello5");     //  Prints OK
}

int main() 
{
    test_choice(); 
}

Prints

Parsing [1]. Status: OK
Parsing [Hello1]. Status: OK
Parsing [2]. Status: OK
Parsing [Hello2]. Status: OK
Parsing [Hello]. Status: OK
Parsing [3]. Status: OK
Parsing [Hello3]. Status: OK
Parsing [4]. Status: OK
Parsing [Hello4]. Status: OK
Parsing [5]. Status: OK
Parsing [Hello5]. Status: OK

And it runs clean under valgrind.

Answer 2

I have found a solution to my problems, at least currently anything works so far. The core issue is to have the rule as a static member and return a const reference to that rule. So basically, what I do is to initialise a static rule in a static local memberfunction. This allows me to initialise the rule. My get() function for T then simply has the signature

static rule<Iter,T(),spacer> const& get() 
{ 
    rule<Iter,T(),spacer> const& t_rule = create_rule(); 
    return t_rule;
}

where create_rule creates the rule, stores it in a local static and returns a const reference to the static.

I would like to thank jv_ and sehe for their help in my long and twisted journey trying to make qi do what I want it to do.