Attributes of sequence and list operator in boost.spirit qi?

Question

I want to parse something like

"{xxxx}
{xxxx}"

which is separated by eol into a vector<vector<wchar_t>> : ({xxxx},{xxxx}) so that "{" and "}" stays with internal characters together. My code is:

#define BOOST_SPIRIT_UNICODE

#include <iostream>
#include<boost/spirit/include/qi.hpp>
#include<string>
#include<vector>

using namespace std;
namespace sw=boost::spirit::standard_wide;
namespace qi= boost::spirit::qi;
using boost::spirit::standard_wide::char_;

int main()
{
    wstring s = L"{\"id\":23,\"text\":\"sf\nsf\"}\n{\"id\":23,\"text\":\"sfsf\"}";
    qi::rule<wstring::iterator, vector<vector<wchar_t>>(), sw::blank_type> ru;
    ru = (qi::char_(L"{") >> *(char_-char_(L"}")) >> char_(L"}")) % qi::eol;
    vector<vector<wchar_t>> result;
    qi::phrase_parse(s.begin(), s.end(), ru, sw::blank, result);

    for (auto& v : result) {
        //cout << "Size of string: " << v.size() << endl;
        for (auto& s : v) {
            wcout << s;
        };
        cout << endl;
    };
    std::cout << "Size of result"<<result.size()<<endl ;
}

However ouput is:

{
"id":23,"text":"sf
sf"
}
{
"id":23,"text":"sfsf"
}
Size of result6

It looks like that "{" becomes a single element of type vector<wchar_t> for the outer vector.

Then consider the rule:

ru = (qi::char_(L"{") >> *(char_-char_(L"}")) >> char_(L"}")) % qi::eol;

According to the documentation, *(char_-char_(L"}")) should be vector<A>. And because a: A, b: vector<A> --> (a >> b): vector<A>, then I think that (qi::char_(L"{") >> *(char_-char_(L"}")) >> char_(L"}")) should be vector<wchar_t>. This is contracdicted to the result.

Where I'm wrong?

Answer 1

And because a: A, b: vector --> (a >> b): vector, then I think that (qi::char_(L"{") >> *(char_-char_(L"}")) >> char_(L"}")) should be vector. This is contracdicted to the result.

Indeed that's not what happens. Applying a modernized trick from Detecting the parameter types in a Spirit semantic action

struct sense_f {
    template <typename T> void operator()(T&&) const {
        std::cout << boost::core::demangle(typeid(T).name()) << "\n";
    }
};
static const boost::phoenix::function<sense_f> sense;

We can print the actual attribute type:

ru = (char_(L'{') >> *(char_ - char_(L'}')) >> char_(L'}')) [sense(qi::_0)] % qi::eol;

Which will print Live On Coliru:

boost::fusion::vector<wchar_t, std::vector<wchar_t, std::allocator<wchar_t> >, wchar_t>

Simple Solution

Assuming that you don't need to capture the {}, you can just make them literals instead of char_:

ru = (L'{' >> *(char_ - L'}') >> L'}') [sense(qi::_0)] % qi::eol;

Which will print Live On Coliru:

boost::fusion::vector<std::vector<wchar_t, std::allocator<wchar_t> >&>

Indeed, if you also make it propagate the attribute:

ru %= (L'{' >> *(char_ - L'}') >> L'}') [sense(qi::_0)] % qi::eol;

The program prints:

boost::fusion::vector<std::vector<wchar_t, std::allocator<wchar_t> >&>
boost::fusion::vector<std::vector<wchar_t, std::allocator<wchar_t> >&>
"\"id\":23,\"text\":\"sf
sf\""
"\"id\":23,\"text\":\"sfsf\""

Note that there is attribute compatibility between std::vector<wchar_t> and std::wstring which is why I used the latter.

Bonus

If you DO want to include {} and any intermediate whitespace, use qi::raw:

ru %= qi::raw [L'{' >> *(char_ - L'}') >> L'}'] [sense(qi::_0)] % qi::eol;

Now it prints:

boost::fusion::vector<boost::iterator_range<__gnu_cxx::__normal_iterator<wchar_t const*, std::__cxx11::basic_string<wchar_t, std::char_traits<wchar_t>, std::allocator<wchar_t> > > >&>
boost::fusion::vector<boost::iterator_range<__gnu_cxx::__normal_iterator<wchar_t const*, std::__cxx11::basic_string<wchar_t, std::char_traits<wchar_t>, std::allocator<wchar_t> > > >&>
"{\"id\":23,\"text\":\"sf
sf\"}"
"{\"id\":23,\"text\":\"sfsf\"}"

As you can see even iterator_range<It> has attribute compatibility with std::wstring because the input is also a sequence of wchar_t.

Of course, take the sense action off unless you want that output.

Full Listing

The final result using the qi::raw approach:

Live On Coliru

#define BOOST_SPIRIT_UNICODE

#include <boost/spirit/include/qi.hpp>
#include <iostream>
#include <iomanip>
#include <string>
#include <vector>

namespace sw = boost::spirit::standard_wide;
namespace qi = boost::spirit::qi;
using sw::char_;

int main() {
    std::wstring s = LR"({"id":23,"text":"sf
sf"}
{"id":23,"text":"sfsf"})";

    using Data = std::vector<std::wstring>;
    using It = std::wstring::const_iterator;

    qi::rule<It, Data(), sw::blank_type> ru
        = qi::raw [L'{' >> *(char_ - L'}') >> L'}'] % qi::eol;

    Data result;
    It f = s.begin(), l = s.end();

    if (qi::phrase_parse(f, l, ru, sw::blank, result)) {
        for (auto& s : result) {
            std::wcout << std::quoted(s) << std::endl;
        };
    } else {
        std::wcout << "Parse failed\n";
    }

    if (f!=l) {
        std::wcout << L"Remaining unparsed: " << std::quoted(std::wstring(f,l)) << std::endl;
    }
}