x3 linker error with separate TU

Question

As my parser grows I separated some rules into different translation units (TU) and the linker problem rises. After weeks of several try&errors without success I reduced my ~50 rules to the (hopefully) minimal example presented here. I've read the related linking errors while separate parser using boost spirit x3 and checked, that I use at the context typedef iso8859_1::space_type and that I invoke iso8859_1::space later on. Also I don't let the compiler deduce the iterator_type.

The linker error I made more fancy by replacing some things:

In function `bool x3::rule<parser::string_literal_class, ast::string_literal, false>::parse<std::string::const_iterator, x3::context<x3::error_handler_tag, std::reference_wrapper<x3::error_handler<std::string::const_iterator > > const, x3::context<x3::skipper_tag, x3::char_class<char_encoding::iso8859_1, x3::space_tag> const, x3::unused_type> >, x3::variant<ast::string_literal> >(std::string::const_iterator&, std::string::const_iterator const&, x3::context<x3::error_handler_tag, std::reference_wrapper<x3::error_handler<std::string::const_iterator > > const, x3::context<x3::skipper_tag, x3::char_class<char_encoding::iso8859_1, x3::space_tag> const, x3::unused_type> > const&, x3::unused_type, x3::variant<ast::string_literal>&) const':

main.cpp: .text._ZNK5boost6spirit2x34ruleIN6parser20string_literal_classEN3ast14string_literalELb0EE5parseIN9__gnu_cxx17__normal_iteratorIPKcNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEEENS1_7contextINS1_17error_handler_tagEKSt17reference_wrapperINS1_13error_handlerISJ_EEENSK_INS1_11skipper_tagEKNS1_10char_classINS0_13char_encoding9iso8859_1ENS1_9space_tagEEENS1_11unused_typeEEEEENS1_7variantIJS6_EEEEEbRT_RKS13_RKT0_SY_RT1_[_ZNK5boost6spirit2x34ruleIN6parser20string_literal_classEN3ast14string_literalELb0EE5parseIN9__gnu_cxx17__normal_iteratorIPKcNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEEENS1_7contextINS1_17error_handler_tagEKSt17reference_wrapperINS1_13error_handlerISJ_EEENSK_INS1_11skipper_tagEKNS1_10char_classINS0_13char_encoding9iso8859_1ENS1_9space_tagEEENS1_11unused_typeEEEEENS1_7variantIJS6_EEEEEbRT_RKS13_RKT0_SY_RT1_]+0x37): undefined reference to `bool parser::parse_rule<std::string::const_iterator, x3::context<x3::error_handler_tag, std::reference_wrapper<x3::error_handler<std::string::const_iterator > > const, x3::context<x3::skipper_tag, x3::char_class<char_encoding::iso8859_1, x3::space_tag> const, x3::unused_type> >, x3::variant<ast::string_literal> >(x3::rule<parser::string_literal_class, ast::string_literal, false>, std::string::const_iterator&, std::string::const_iterator const&, x3::context<x3::error_handler_tag, std::reference_wrapper<x3::error_handler<std::string::const_iterator > > const, x3::context<x3::skipper_tag, x3::char_class<char_encoding::iso8859_1, x3::space_tag> const, x3::unused_type> > const&, x3::variant<ast::string_literal>&)'

I'm irritated by the __gnu_cxx17 iterator (from libstdc++??). I did try g++7.1.0 with the same result.

Attached the files to reproduce the linker error using a CMake file (with option LINKER_ERROR to get or even not (monolitic build without TU)).

ast.hpp

#ifndef AST_HPP_
#define AST_HPP_

#include <boost/spirit/home/x3/support/ast/variant.hpp>
#include <string>

#include <boost/fusion/include/adapt_struct.hpp>

namespace ast {
    namespace x3 = boost::spirit::x3;

    struct string_literal {
        std::string literal;
    };

    using primary = x3::variant<string_literal>;
    using factor = x3::variant<primary>;
}

BOOST_FUSION_ADAPT_STRUCT(ast::string_literal,
    literal
)

#endif // AST_HPP_

grammar_def.hpp

#include "ast.hpp"
#include <boost/spirit/home/x3.hpp>
#if defined(LINKER_ERROR)
#include "literal.hpp"
#endif

namespace parser {

   namespace x3 = boost::spirit::x3;

    struct factor_class;
    struct primary_class;
#if !defined(LINKER_ERROR)
    struct string_literal_class;
#endif

    typedef x3::rule<factor_class, ast::factor> factor_type;
    typedef x3::rule<primary_class, ast::primary> primary_type;
#if !defined(LINKER_ERROR)
    typedef x3::rule<string_literal_class, ast::string_literal> string_literal_type;
#endif

    factor_type const factor{ "factor" };
    primary_type const primary{ "primary" };
#if !defined(LINKER_ERROR)
    string_literal_type const string_literal{ "string_literal" };
#endif

#if defined(LINKER_ERROR)
    namespace {
        auto const& string_literal = parser_api::string_literal();
    }
#endif

    auto const factor_def = primary;

    auto const primary_def = string_literal;

#if !defined(LINKER_ERROR)
    auto const string_literal_def =
        x3::lexeme ['"' >> *x3::char_ >> '"' ]
        ;
#endif

    BOOST_SPIRIT_DEFINE(factor, primary)
#if !defined(LINKER_ERROR)
    BOOST_SPIRIT_DEFINE(
            string_literal
    )
#endif
}

literal_def.hpp

#ifndef PARSER_LITERAL_DEF_HPP_
#define PARSER_LITERAL_DEF_HPP_

#include "literal.hpp"

namespace parser {

    namespace x3 = boost::spirit::x3;

    string_literal_type const string_literal { "string_literal" };

    auto const string_literal_def =
        x3::lexeme ['"' >> *x3::char_ >> '"' ];

    BOOST_SPIRIT_DEFINE(
        string_literal
    )
}

namespace parser_api {
    parser::string_literal_type const& string_literal()
    {
        return parser::string_literal;
    }
}

#endif /* PARSER_LITERAL_DEF_HPP_ */

literal.cpp

#include "literal_def.hpp"
#include "parser_config.hpp"

namespace parser {
    namespace x3 = boost::spirit::x3;

    BOOST_SPIRIT_INSTANTIATE(
        string_literal_type,
        iterator_type,
        context_type
    );
}

literal.hpp

#ifndef PARSER_LITERAL_HPP_
#define PARSER_LITERAL_HPP_

#include "ast.hpp"
#include <boost/spirit/home/x3.hpp>

namespace parser {

    namespace x3 = boost::spirit::x3;

    struct string_literal_class;
    typedef x3::rule<string_literal_class, ast::string_literal> string_literal_type;

    BOOST_SPIRIT_DECLARE(string_literal_type);
}

namespace parser_api {
    parser::string_literal_type const& string_literal();
}

#endif /* PARSER_LITERAL_HPP_ */

main.cpp

#include "grammar_def.hpp"
#include "parser_config.hpp"
#include <iostream>

int main()
{
    namespace x3 = boost::spirit::x3;

    ast::factor  attr;

    std::string input{ "a * b" };
    parser::iterator_type iter = input.begin();
    parser::iterator_type const end = input.end();

    parser::error_handler_type error_handler(iter, end, std::cerr);

    auto const parser =
        x3::with<x3::error_handler_tag>(std::ref(error_handler))
        [
              parser::factor
        ];

    bool success = x3::phrase_parse(iter, end, parser, x3::iso8859_1::space, attr);

    return 0;
}

parser_config.hpp

#ifndef PARSER_CONFIG_HPP_
#define PARSER_CONFIG_HPP_

#include <boost/spirit/home/x3.hpp>
#include <boost/spirit/home/x3/support/utility/error_reporting.hpp>

namespace parser {
    namespace x3 = boost::spirit::x3;

    template <typename Iterator>
    using error_handler = x3::error_handler<Iterator>;
    using error_handler_tag = x3::error_handler_tag;

    typedef std::string::const_iterator                 iterator_type;
    typedef error_handler<iterator_type>                error_handler_type;
    typedef x3::phrase_parse_context<
            x3::iso8859_1::space_type>::type            phrase_context_type;
    typedef x3::with_context<
        error_handler_tag
        , std::reference_wrapper<error_handler_type> const
        , phrase_context_type
    >::type                                             context_type;
}

#endif /* CONFIG_HPP_ */

and finally the CMakeLists.txt

cmake_minimum_required(VERSION 3.6)

project(x3_linker_error LANGUAGES CXX)
find_package(Boost REQUIRED COMPONENTS system)

set(CMAKE_CXX_STANDARD 14)
set(CMAKE_CXX_STANDARD_REQUIRED ON)
set(CMAKE_CXX_EXTENSIONS OFF)
# lib
add_library(${PROJECT_NAME}_x3 STATIC
    literal.cpp
)
# exe
add_executable(${PROJECT_NAME}
    main.cpp
)
target_link_libraries(${PROJECT_NAME}
    PRIVATE
    ${PROJECT_NAME}_x3
    ${Boost_SYSTEM_LIBRARY}
)

set(CMAKE_VERBOSE_MAKEFILE ON)
option(LINKER_ERROR "Enforce the linker error" ON)
if(LINKER_ERROR)
message("## Build to show linker errors")
target_compile_definitions(${PROJECT_NAME}_x3 PRIVATE LINKER_ERROR)
target_compile_definitions(${PROJECT_NAME} PRIVATE LINKER_ERROR)
endif()

The code is also at wandbox.

Setting parser_api::string_literal() into the instance file literal.cpp doesn't change the result too (as expected). Hopefully it isn't a silly fault on my side but after weeks I'm at the end...

Addendum

I also tested the tip of Joel de Guzman to insert int x = context_type{}; to literal.cpp and main.cpp to get the the context used - its both the same:

x3_linker_error/literal.cpp
'boost::spirit::x3::context<boost::spirit::x3::error_handler_tag, const std::reference_wrapper<boost::spirit::x3::error_handler<__gnu_cxx::__normal_iterator<const char*, std::__cxx11::basic_string<char> > > >, boost::spirit::x3::context<boost::spirit::x3::skipper_tag, const boost::spirit::x3::char_class<boost::spirit::char_encoding::iso8859_1, boost::spirit::x3::space_tag>, boost::spirit::x3::unused_type> >::context(<brace-enclosed initializer list>)'

x3_linker_error/main.cpp
'boost::spirit::x3::context<boost::spirit::x3::error_handler_tag, const std::reference_wrapper<boost::spirit::x3::error_handler<__gnu_cxx::__normal_iterator<const char*, std::__cxx11::basic_string<char> > > >, boost::spirit::x3::context<boost::spirit::x3::skipper_tag, const boost::spirit::x3::char_class<boost::spirit::char_encoding::iso8859_1, boost::spirit::x3::space_tag>, boost::spirit::x3::unused_type> >::context(<brace-enclosed initializer list>)'

Those are not the exact errors you get, are they? They don't look like compiler output, and, specifically there's no explicit error message. Also, why do you have the LINKER_ERROR option - should that be the only code you post here? :-( — einpoklum, Jul 10 '17 at 15:34

Olx · Accepted Answer · 2017-06-06T12:54:51.630

With help of Larry Evans the problem is solved. For details please follow the spirit-general mailing list.

The code fragments as such is correct. It seems that the problem did rise up in another context as Linking error when changing > to >> some times before and discussed by Seth.

The solution is to use/clone Larry's get_rhs branch of spirit.x3

git clone --branch get_rhs https://github.com/cppljevans/spirit.git destination

and compile with defined BOOST_ERROR_CODE_HEADER_ONLY. Spirit.X3 as of Boost 1.64 doesn't compile. I've adapted the CMake file to reflect the changes (and cloned into cppljevans) and added further defines seen by Larry's compile log:

cmake_minimum_required(VERSION 3.6)

project(x3_linker_error LANGUAGES CXX)
find_package(Boost REQUIRED COMPONENTS system)

set(CMAKE_CXX_STANDARD 14)
set(CMAKE_CXX_STANDARD_REQUIRED ON)
set(CMAKE_CXX_EXTENSIONS OFF)

# lib
add_library(${PROJECT_NAME}_x3 STATIC
    literal.cpp
)
target_include_directories(${PROJECT_NAME}_x3
    PUBLIC ${CMAKE_CURRENT_SOURCE_DIR}/cppljevans/include
)
target_compile_definitions(${PROJECT_NAME}_x3 PRIVATE 
    BOOST_SPIRIT_GET_RHS_CRTP=1 BOOST_SPIRIT_ATTR_XFORM_IN_RULE=1
)

# exe
add_executable(${PROJECT_NAME}
    main.cpp
)
target_include_directories(${PROJECT_NAME}
    PUBLIC ${CMAKE_CURRENT_SOURCE_DIR}/cppljevans/include
)
target_compile_definitions(${PROJECT_NAME} PRIVATE 
    BOOST_SPIRIT_GET_RHS_CRTP=1 BOOST_SPIRIT_ATTR_XFORM_IN_RULE=1
    BOOST_ERROR_CODE_HEADER_ONLY 
)
target_link_libraries(${PROJECT_NAME}
    PRIVATE
    ${PROJECT_NAME}_x3
    ${Boost_SYSTEM_LIBRARY}
)

set(CMAKE_VERBOSE_MAKEFILE ON)

option(LINKER_ERROR "Enforce the linker error" ON)

if(LINKER_ERROR)
message("## Build to show linker errors")
target_compile_definitions(${PROJECT_NAME}_x3 PRIVATE LINKER_ERROR)
target_compile_definitions(${PROJECT_NAME} PRIVATE LINKER_ERROR)
endif()

What is the cause of the problem? And how does using the branch solve it? It looks like an unsupported branch with _many_ undocumented changes with a very low chance of being taken in upstream. I've asked about this on the mailing list and got no response from Larry. — sehe, Jun 06 '17 at 20:21
(By the way, thanks for sharing back on SO, that's valuable regardless of any issues with the problem or the solution!) — sehe, Jun 06 '17 at 20:25
I agree, the concrete cause of the problem would be interesting. I didn't explain the cause as Larry did a **guess** of the reason and I wasn't able to understand the internals he mentioned unfortunately. The cause seems to me to be inherent by design of x3 and hence will raise up from time to time again in different flavours. The real problem with the solution is the unoffical branch, maintained by Larry as far I understood. — Olx, Jun 08 '17 at 07:57

x3 linker error with separate TU

1 Answers1

Linked