2

i was working on an interpreter for a language with a friend, and we started with a decision I'm guessing wasn't that wise: we made all the elements for execution first (practically a tree made of different classes); but now looking at boost examples i get a lot confused about how to merge the two. I know what to start from (the grammar), i know what to reach (instantiated classes owning each other), i don't know how to reach it.

We started with expressions without variables, hence we looked at spirit calculator examples; but i don't understand when to instantiate elements.

Example of expression items:

namespace exp
{
class op
    {
    private:
    public:
        virtual double exec(function_scope &fs);

    };

class operand : public op
    {
    private:
        double value;

    public:
        operand(double value);
        double exec(function_scope &fs);
    };

class op_bin : public op
    {
    private:
    public:
        op * ll;
        op* rr;
        op_bin(op* ll, op* rr);
        ~op_bin();
    };

namespace bin
    {
    class sum : public op_bin
        {
        public:
            sum(op* ll, op* rr);
            double exec(function_scope &fs);
        };
    }
}

Ignore the exec function, it's used at runtime.

For example the code 5 + (2 + 1) should result in a final equivalent of:

new exp::bin::sum(new exp::operand(5), new exp::bin::sum(new exp::operand(2), new exp::operand(1))

Once i understand how to do that I've practically done.

Nikita Kniazev
  • 3,728
  • 2
  • 16
  • 30
Barnack
  • 921
  • 1
  • 8
  • 20
  • 1
    So many `new`s, so few smart pointers... – Quentin Nov 15 '18 at 13:06
  • this is not a question about dealing with pointers @Quentin – Barnack Nov 15 '18 at 13:07
  • 1
    Hence a comment and not an answer :) – Quentin Nov 15 '18 at 13:09
  • @Quentin i'd gladly ask a question about when smart pointers are not needed and why are they used anyway but it'd be blocked as too broad and opinion based – Barnack Nov 15 '18 at 13:12
  • 1
    They hook into RAII to automatically perform the bookkeeping you need to do manually otherwise. That's it. – Quentin Nov 15 '18 at 13:20
  • sure, but this is a tree structure that will end up having a single root; deleting the root will delete everything through all destructors destroying what's beneath them; it's essencially a dfs visit of the whole tree. Hence i don't see the need for smart pointers. – Barnack Nov 15 '18 at 13:26
  • 2
    There's never a *need*, but that's exactly what `std::unique_ptr` gives you out of the box without you writing any destructor or move constructor (and remembering to delete the copy constructor). It's just convenient. – Quentin Nov 15 '18 at 13:33
  • 1
    There is nothing bad with `new` calls (especially in the example as it simplifies code reading), but storing raw pointers in classes is a smell, I agree. – Nikita Kniazev Nov 16 '18 at 01:01
  • 2
    Heh. I'd flip that logic around. It's okay to store raw pointers (as long as they're non-owning). However, using `new` or `delete` indicates that ownership is not properly protected and hence invites error. Regardless, here's a link to a question that does explain what's the problem with dynamically allocated attribute types: https://stackoverflow.com/questions/37911950/semantic-actions-runs-multiple-times-in-boostspirit-parsing/37912787#37912787 – sehe Nov 16 '18 at 01:03
  • 2
    You can store references, with them you know that the object exists (no need for null check or asserts), the only downside I know is a lack of default assignment operator for those classes. – Nikita Kniazev Nov 16 '18 at 01:15

1 Answers1

6

Well, I was going to write what's wrong with your question, but instead I went to prove myself that it is not that hard to make what you want.

Few keypoints:

  • I slightly modified, renamed and extended your ast to make it work and to actually show something.
  • Spirit rules for some reason make copy of an attribute (I think it is a bug), so I workarounded this issue for unique_ptr with a trait. (fixed in 1.70)
  • I am not sure if x3::omit is actually required there (you can remove all except the last and it will compile), but it looks like it is an another bug in Spirit.
  • make_node looks unreliable and may broke in surprising ways, you can split it into separate unary/binary node creators if you wish.
  • At some point you will want to use stateful allocator for your ast nodes creation, it should be very simple by injecting allocator into the parser context. I am leaving it for you as an exercise.

The parser:

#include <boost/spirit/home/x3.hpp>
#include <memory>
#include <iostream>

namespace ast
{

class expression
{
protected:
    expression() = default;
public:
    virtual ~expression() = default;
    expression(expression&& other) = delete;
    expression& operator=(expression&& other) = delete;

    virtual void print(std::ostream&) const = 0;

    friend std::ostream& operator<<(std::ostream& os, expression const& node)
    {
        node.print(os);
        return os;
    }
};

class operand : public expression
{
    double value_;

public:
    constexpr operand(double value) : value_{value} {}
    void print(std::ostream& os) const override { os << value_; }
};

class op_bin : public expression
{
protected:
    std::unique_ptr<expression> left_, right_;

public:
    op_bin(std::unique_ptr<expression> left, std::unique_ptr<expression> right)
      : left_{ std::move(left) }, right_{ std::move(right) }
    {}

    op_bin(expression * left, expression * right)
        : left_{ left }, right_{ right }
    {}
};

class plus : public op_bin
{
public:
    using op_bin::op_bin;
    void print(std::ostream& os) const override
    { os << '(' << *left_ << " + " << *right_ << ')'; }
};

class minus : public op_bin
{
public:
    using op_bin::op_bin;
    void print(std::ostream& os) const override
    { os << '(' << *left_ << " - " << *right_ << ')'; }
};

class mul : public op_bin
{
public:
    using op_bin::op_bin;
    void print(std::ostream& os) const override
    { os << '(' << *left_ << " * " << *right_ << ')'; }
};

class div : public op_bin
{
public:
    using op_bin::op_bin;
    void print(std::ostream& os) const override
    { os << '(' << *left_ << " / " << *right_ << ')'; }
};

} // namespace ast

namespace grammar
{

namespace x3 = boost::spirit::x3;

template <typename T>
struct make_node_
{
    template <typename Context>
    void operator()(Context const& ctx) const
    {
        if constexpr (std::is_convertible_v<decltype(x3::_attr(ctx)), T>) {
            x3::_val(ctx) = std::make_unique<T>(std::move(x3::_attr(ctx)));
        }
        else {
            x3::_val(ctx) = std::make_unique<T>(std::move(x3::_val(ctx)), std::move(x3::_attr(ctx)));
        }
    }
};

template <typename T>
constexpr make_node_<T> make_node{};

using x3::double_;
using x3::char_;

x3::rule<class expression_r, std::unique_ptr<ast::expression>, true> const expression;
x3::rule<class prec1_r, std::unique_ptr<ast::expression>, true> const prec1;
x3::rule<class prec0_r, std::unique_ptr<ast::expression>, true> const prec0;

auto const expression_def =
    prec1
    >> *(   x3::omit[('+' > prec1)[make_node<ast::plus>]]
        |   x3::omit[('-' > prec1)[make_node<ast::minus>]]
        )
    ;

auto const prec1_def =
    prec0
    >> *(   x3::omit[('*' > prec0)[make_node<ast::mul>]]
        |   x3::omit[('/' > prec0)[make_node<ast::div>]]
        )
    ;

auto const prec0_def =
        x3::omit[double_[make_node<ast::operand>]]
    |   '(' > expression > ')'
    ;

BOOST_SPIRIT_DEFINE(
    expression
  , prec1
  , prec0
);

} // namespace grammar

#if BOOST_VERSION < 107000
namespace boost::spirit::x3::traits {

template <typename Attribute>
struct make_attribute<std::unique_ptr<Attribute>, std::unique_ptr<Attribute>>
  : make_attribute_base<std::unique_ptr<Attribute>>
{
    typedef std::unique_ptr<Attribute>& type;
    typedef std::unique_ptr<Attribute>& value_type;
};

} // namespace boost::spirit::x3::traits
#endif

int main()
{
    namespace x3 = boost::spirit::x3;

    std::string s = "1 + 2 * (3 - 4) / 5";
    std::unique_ptr<ast::expression> expr;
    if (auto iter = s.cbegin(); !phrase_parse(iter, s.cend(), grammar::expression, x3::space, expr)) {
        std::cout << "parsing failed";
    }
    else {
        if (iter != s.cend())
            std::cout << "partially parsed\n";
        std::cout << *expr << '\n';
    }
}

Output:

(1 + ((2 * (3 - 4)) / 5))
Nikita Kniazev
  • 3,728
  • 2
  • 16
  • 30
  • 1
    Well done. For Qi, see my older take on polymorphic AST creation here: https://stackoverflow.com/questions/23299722/how-can-i-use-polymorphic-attributes-with-boostspiritqi-parsers/23301519#23301519 – sehe Nov 16 '18 at 01:02
  • I remember I saw a polymorphic parser earlier, not sure if it was your answer, hope my answer is a nice addition to your collection. – Nikita Kniazev Nov 16 '18 at 01:08
  • I certainly looks like it to me – sehe Nov 16 '18 at 01:11
  • Thanks for the sample; i already had extended ast, just gave the minimum for a working example; although would you mind explain me what that make_node/make_node_ is and how it works? – Barnack Nov 16 '18 at 16:43
  • 1
    I cannot explain it better than the docs https://www.boost.org/doc/libs/develop/libs/spirit/doc/x3/html/spirit_x3/tutorials/semantic_actions.html – Nikita Kniazev Nov 16 '18 at 18:13
  • Thanks for the link, i was actually more focused on q3's syntax and didn't check how it worked on x3. – Barnack Nov 18 '18 at 15:47
  • Ok i got the semantic action syntax, i got that \_attr returns the attribute of the current parser, documentation example is pretty clear. I honestly prefer to avoid introducing (hence turning everything in the entire ast to) smart pointers (which i've not even experience using); and from your code i understood everything (i'd say everything x3 related) but i'm utterly lost when it comes to the actual content of make\_node\_'s () operator. Can you please turn the example to use op\_bin's pointers contructor rather than the unique pointers one? Thanks in advance. – Barnack Nov 18 '18 at 16:11
  • @Nikita Kniazev is it not possible because of some boost limitations or you just missed the notification? – Barnack Nov 21 '18 at 15:39
  • 1
    @Barnack It is a good opportunity for you to start using smart pointers. You can replace `unique_ptr` in `op_bin` with a raw pointer, it will require a trivial change to `make_node_` (because it itself is trivial), but changing parsers to use raw pointers will make them leaking memory on backtracking (that you may introduce by adding complexity to the grammar). – Nikita Kniazev Nov 23 '18 at 00:52