Improvements in repeat directive with variable factor for X3

Question

I need to parse a sequence of elements where there is a first number telling how many elements must be parsed next.

As a simplification of what I need: [3 10 20 30] should be parsed as showed next:

- Number of elements: 3
- Vector of elements: {10, 20, 30}

Being used to Qi and its "repeat directive + phoenix" syntax I tried the same for X3, failing in compiling. I have being looking in web for the same problem, and i find next link in stack overflow: Boost Spirit X3 cannot compile repeat directive with variable factor

I am shocked of how something so elegantly solved in Qi can be so ugly and cumberson in X3 (personal opinion, please nobody got offended). Of course I get the reasons about phoonix abandoned due to c++14 replacing it.

But I wonder is there is any further improvement in X3 about this subject due to this post is from 2015. I have been looking but nothing found. Any advise?

NOTE- No code included due that is same case/code that the posted link.

Thanks.

score 3 · Accepted Answer · answered Jan 24 '19 at 16:26

Usually it means that there was no PR for that feature (or it was but has some issues). The repeat also has design problems. For example you can parse {10 20 30} with it, but not {10, 20, 30} (requires a kind of list parser).

I cannot agree with that Qi has an elegant way of doing it because you have to use a rule with local variable or pass a reference to an external value. The natural way seems to be repeat(len_parser)[item_parser], but it has additional design issues with skippers (or skippers has design issues that limits complex directives flexibility).

Fortunately the Spirit X3 is much simpler in writing own parser combinators.

#include <boost/spirit/home/x3.hpp>

namespace x3e {

namespace x3 = boost::spirit::x3;

template <typename LenParser, typename Subject>
struct vlrepeat_directive : x3::unary_parser<Subject, vlrepeat_directive<LenParser, Subject>>
{
    using base_type = x3::unary_parser<Subject, vlrepeat_directive<LenParser, Subject>>;
    static bool const handles_container = true;

    vlrepeat_directive(LenParser const& lp_, Subject const& subject)
        : base_type(subject), lp(lp_) {}

    template<typename Iterator, typename Context, typename RContext, typename Attribute>
    bool parse(Iterator& first, Iterator const& last
      , Context const& context, RContext& rcontext, Attribute& attr) const
    {
        static_assert(x3::traits::has_attribute<LenParser, Context>::value, "must syntesize an attribute");

        Iterator iter = first;
        typename x3::traits::attribute_of<LenParser, Context>::type len;
        if (!lp.parse(iter, last, context, rcontext, len))
            return false;

        for (; len; --len) {
            if (!x3::detail::parse_into_container(
                    this->subject, iter, last, context, rcontext, attr))
                return false;
        }

        first = iter;
        return true;
    }

    LenParser lp;
};

template <typename LenParser>
struct vlrepeat_gen
{
    template <typename Subject>
    vlrepeat_directive<LenParser, typename x3::extension::as_parser<Subject>::value_type>
    operator[](Subject const& p) const
    {
        return { lp, x3::as_parser(p) };
    }

    LenParser lp;
};

template <typename Parser>
vlrepeat_gen<Parser> vlrepeat(Parser const& p)
{
    static_assert(x3::traits::is_parser<Parser>::value, "have to be a parser");
    return { p };
}

}

template <typename LenParser, typename Subject, typename Context>
struct boost::spirit::x3::traits::attribute_of<x3e::vlrepeat_directive<LenParser, Subject>, Context>
    : build_container<typename attribute_of<Subject, Context>::type> {};

And use it:

#include <iostream>
#include <vector>

int main()
{
    namespace x3 = boost::spirit::x3;

    auto s = "5: 1 2 3 4 5", e = s + std::strlen(s);
    std::vector<int> v;
    if (phrase_parse(s, e, x3e::vlrepeat(x3::uint_ >> ':')[x3::int_], x3::space, v)) {
        std::cout << "Result:\n";
        for (auto x : v)
            std::cout << x << '\n';
    }
    else
        std::cout << "Failed!\n";
}

Output:

Result:
1
2
3
4
5

https://wandbox.org/permlink/K572K0BMEqA8lMJm

(it has a call to detail::parse_into_container which is not a public API)

Nice answer, thank you very much. I think this could be included as part of X3 instead of boiler plate that code every time is need. Have you thought in pulling a proposal into Boost? — Pablo, Jan 25 '19 at 09:17
Also, I think there is still a case where the old way (repeat+phoenix with qi or the one provided in the link for X3) can be useful. Sometimes there are formats where the number of repeats is located far, far away from the repeated data . — Pablo, Jan 25 '19 at 09:20
As I said there are issues that will not let the parser get into Spirit (you cannot just disable pre-skipping of the first element, having different skippers for length and items parsers will lead to a very ugly thing), it also needs to be integrated into already existing `repeat`. That's strange that length is far from the data, but there is an uncontroversial example: length+SoA (multiple arrays of the same size). Something like `with()[(len = uint_) >> repeat(len)[int_] >> repeat(len)[int_]]` should be possible to implement. — Nikita Kniazev, Jan 25 '19 at 12:45

Improvements in repeat directive with variable factor for X3

1 Answers1