boost::spirit parser returns empty vector

Question

Why the parser using the rules below returns an empty container? There're 3 rules. One is for parsing a string of characters except double quote, the second parses a pair (e.g. "col1" : 2) and the third parses the vector of such pairs. The ouput of the program below in MSVS2012 is

parse success
result: '' : 0
result: '' : 0
result: '' : 0

.

namespace parsers
{

            spirit::qi::rule< iterator, column_name_t() > quoted_string = 
                    spirit::qi::lexeme["\"" >> +~spirit::qi::char_("\"") >> "\""];
                spirit::qi::rule< iterator, column_and_aggregate(), spirit::qi::space_type > agg_pair =
                    quoted_string//::boost::bind( &apply_col_and_aggr_visitor, spirit::qi::_val, spirit::qi::_1 )]
                    > ':'
                    // A rule validation technic is used below.
                    > spirit::int_[spirit::qi::_pass = (spirit::qi::_1 >=AVG && spirit::qi::_1<=SUM)];//::boost::bind( &apply_col_and_aggr_visitor, spirit::qi::_val, spirit::qi::_1 )];
                spirit::qi::rule< iterator, column_and_aggregate_container(), spirit::qi::space_type > aggregates_parser =
                      '{'
                    > agg_pair/*[phoenix::push_back(spirit::qi::_val, spirit::qi::_1)]*/ % ',' // N.B.!!! list-redux technic
                    > '}';
}
using namespace parsers;
using namespace boost::spirit;
bool doParse(const std::string& input)
{
    typedef std::string::const_iterator It;
    auto f(begin(input)), l(end(input));

    //parser<It, qi::space_type> p;
    column_and_aggregate_container data;
    typedef BOOST_TYPEOF(qi::space) skipper_type; 
    try
    {
        bool ok = qi::phrase_parse(f,l,aggregates_parser,qi::space,data);
        if (ok)   
        {
            std::cout << "parse success\n";
            for (auto& pair : data)
                std::cout << "result: '" << pair.first << "' : " << (int) pair.second << "\n";
        }
        else      std::cerr << "parse failed: '" << std::string(f,l) << "'\n";

        if (f!=l) std::cerr << "trailing unparsed: '" << std::string(f,l) << "'\n";
        return ok;
    }
    catch(const qi::expectation_failure<It>& e)
    {
        std::string frag(e.first, e.last);
        std::cerr << e.what() << "'" << frag << "'\n";
    }

    return false;
}

int main()
{
    //bool ok = doParse("{ 'column 1' : 1, 'column 2' : 0, 'column 3' : 1 }");
    doParse("{ \"column 1\" : 1, \"column 2\" : 0, \"column 3\" : 1 }");
    //return ok? 0 : 255;
}


template <typename it, typename skipper = qi::space_type>
struct quoted_string_parser
{
    quoted_string_parser()
    {
        using namespace qi;
        quoted_string %= lexeme['"' >> *~char_('"') >> '"'];
        BOOST_SPIRIT_DEBUG_NODE(quoted_string);
    }
    qi::rule<it, std::string(), skipper> quoted_string;
};
template <typename it, typename skipper = qi::space_type>
struct aggregates_parser : qi::grammar<it, column_and_aggregate_container(), skipper>
{
    aggregates_parser() : aggregates_parser::base_type(aggregates_parser_)
    {
        using namespace qi;
        agg_pair %= quoted_string_parser<it,skipper> > ':' > int_[_pass = (qi::_1 >= AVG && qi::_1 <= SUM)];
        aggregates_parser_ = '{' > agg_pair % ',' > '}';
        BOOST_SPIRIT_DEBUG_NODE(aggregates_parser_);
    }
  private:    
    qi::rule<it, sql_faggregate(), skipper> faggr;
    qi::rule<it, column_and_aggregate(), skipper> agg_pair;
    qi::rule<it, column_and_aggregate_container(), skipper> aggregates_parser_;
};

Since when is a vector containing 3 element pairs an "empty vector"? Also, why do you cripple the code so much in your question? I think the code was a **lot** more readable when I gave it to you. Also, presenting the question in readable form is a polite effort. Lastly, adding ~12 more lines could have made the code compilable. Why remove them? It's almost as if you don't want people to help. — sehe, Jun 29 '13 at 22:59
Also, had you tried eliminating probable causes? Just commenting `[spirit::qi::_pass = (spirit::qi::_1 >=AVG && spirit::qi::_1<=SUM)]` made the problem disappear. This might have lead you to the relevant documentation. — sehe, Jun 29 '13 at 23:07
Don't use `using namespace boost::spirit`. Instead do `namespace qi = boost::spirit::qi`. This will prevent a lot of problems with ADL/ambiguous identifiers. `using namespace parsers;` should go _inside_ `doParse` (if at all). Finally, don't use `boost::bind` inside semantic actions. `phoenix::bind` is far superior as it understands Spirit's _lazy_ placeholders. — sehe, Jun 29 '13 at 23:14
In case you wanted to know, [here's how to do the attribute propagation manually](http://ideone.com/0vjeKE). The code review was free of charge :/ Cheers — sehe, Jun 29 '13 at 23:16
(Note that [that manual approach](http://ideone.com/0vjeKE) kept your ~~messy~~"creative" code organization: it shows you **exactly** how you _might_ do things when you don't want to do a proper grammar struct. I don't _recommend_ doing this, obviously.) — sehe, Jun 29 '13 at 23:28

score 5 · Accepted Answer · edited May 23 '17 at 10:25

5

Like I said in the answer right where I suggested this semantic action for validation:

enter image description here

When a semantic action is present, automatic attribute propagation does not normally occur. Using %= forces automatic attribute propagation (we want this because the semantic action doesn't assign the attribute value(s), it just validates them).

Again, a fully working demonstration, incorporating your rules:

#include <boost/fusion/adapted/std_pair.hpp>
#include <boost/spirit/include/qi.hpp>
#include <boost/spirit/include/phoenix.hpp>

namespace qi    = boost::spirit::qi;
namespace phx   = boost::phoenix;

typedef std::string column_name_t;

enum sql_faggregate
{
    AVG,
    // ....
    SUM,
};

typedef std::pair<column_name_t, sql_faggregate> column_and_aggregate;
typedef std::vector<column_and_aggregate> column_and_aggregate_container;

template <typename It, typename Skipper = qi::space_type>
    struct parser : qi::grammar<It, column_and_aggregate_container(), Skipper>
{
    parser() : parser::base_type(aggregates_parser)
    {
        using namespace qi;
        quoted_string     = lexeme['"' >> +~char_('"') >> '"'];
        agg_pair         %= quoted_string > ':' // A rule validation technic is used below.
                            > int_[_pass = (_1 >=AVG && _1<=SUM)];

        aggregates_parser = '{' > agg_pair % ',' > '}';

        BOOST_SPIRIT_DEBUG_NODE(aggregates_parser);
    }

  private:
    qi::rule<It, std::string(), qi::space_type>           quoted_string;
    qi::rule<It, sql_faggregate(), qi::space_type>        faggr;
    qi::rule<It, column_and_aggregate(), qi::space_type>           agg_pair;
    qi::rule<It, column_and_aggregate_container(), qi::space_type> aggregates_parser;
};

bool doParse(const std::string& input)
{
    typedef std::string::const_iterator It;
    auto f(begin(input)), l(end(input));

    parser<It, qi::space_type> p;
    column_and_aggregate_container data;

    try
    {
        bool ok = qi::phrase_parse(f,l,p,qi::space,data);
        if (ok)   
        {
            std::cout << "parse success\n";
            for (auto& pair : data)
                std::cout << "result: '" << pair.first << "' : " << (int) pair.second << "\n";
        }
        else      std::cerr << "parse failed: '" << std::string(f,l) << "'\n";

        if (f!=l) std::cerr << "trailing unparsed: '" << std::string(f,l) << "'\n";
        return ok;
    } catch(const qi::expectation_failure<It>& e)
    {
        std::string frag(e.first, e.last);
        std::cerr << e.what() << "'" << frag << "'\n";
    }

    return false;
}

int main()
{
    bool ok = doParse("{ \"column 1\" : 1, \"column 2\" : 0, \"column 3\" : 1 }");
    return ok? 0 : 255;
}

Prints

parse success
result: 'column 1' : 1
result: 'column 2' : 0
result: 'column 3' : 1

as expected

edited May 23 '17 at 10:25

Community

1
1

answered Jun 29 '13 at 22:43

sehe

374,641
47
450
633

What if I don't need gramatics structure, but only one rule? – alexander.sivak Jun 29 '13 at 23:23
Then you make only one rule. Can you tell me what the problem is with either? – sehe Jun 29 '13 at 23:23
The way I see it you use the namespace to contain the rules, but this (a) hardcodes the iterator and skipper (b) makes it hard for you to statically initialize them (e.g. using `%=`). (c) makes it hard to compose grammars (d) makes it harder to debug grammars. The way I see it, you could fix ***all of these*** by using the grammar struct and (optionally) marking the `parser` instance `static const`. – sehe Jun 29 '13 at 23:26
But how would u use validation in rules in my example without grammar? – alexander.sivak Jun 29 '13 at 23:28
Please. **Do** try to read the answers before you ask redundant questions. `operator%=` was in the previous answer. The answer to your last, repeated (!) question was posted [10 minutes before you even commented for the first time](http://stackoverflow.com/questions/17385077/boostspirit-parser-returns-empty-vector/17385379#comment25238512_17385077). – sehe Jun 29 '13 at 23:29
1

No. It works everywhere. However, it will require `%=` assignment, which can not coincide with rule _declaration_. Therefore you need an initialization step. Nothing prevents you from initializing a global rule variable somewhere. However, all of this has nothing to do with Spirit (it's the same for variables of type `int`) ***and*** it comes with all the caveats of using global variables in the first place. – sehe Jun 29 '13 at 23:36
I have severa such subparsers which parse JSON-like formatted sequence of SQL queries. There're for example filter parser, aggregate parser(in the example above) ans so on. Is it better to make grammar for each of them? – alexander.sivak Jun 29 '13 at 23:39
I'd say: either create parser structs for them **or** declare the rules locally to the parsing code (again, consider `static const`). Cheers. – sehe Jun 29 '13 at 23:42
I declared the rule in the same namespace and then assigned it an expression using %=. It did not compile – alexander.sivak Jun 29 '13 at 23:54
Cough. Like I said, it's no different from integers, and _you need an initialization step_. See this illustration: http://ideone.com/oWfYPZ (can you see how a grammar struct might not be all that bad? http://ideone.com/rNlQj7) – sehe Jun 30 '13 at 00:15
The code in the first post is renewed, the vs2012 compiler yields error C2275: 'quoted_string_parser' : illegal use of this type as an expression. How to use grammar in other grammar correctly? – alexander.sivak Jun 30 '13 at 15:38

boost::spirit parser returns empty vector

1 Answers1

Linked