9

I have a template class with a specialization, defined in another file. Therefore it is possible to generate two versions of the same class: one time by replacing the template parameter and one time using the specialization. My current understanding is, that this can lead to two instanciations of the same type to have different sizes in memory, resulting in segmentation faults.

I created a minimal example and the following code is to illustrate the question:

Create a template class:

// - templateexample.h ---------------
#ifndef TEMPLATEEXAMPLE_H
#define TEMPLATEEXAMPLE_H
template<typename T> class Example
{
    public:
        Example(){}
        int doWork() {return 42;}
};
#endif
// -----------------------------------

Template specialization in another file:

// - templatespecialization.h --------
#ifndef TEMPLATESPECIALIZATION_H
#define TEMPLATESPECIALIZATION_H    
#include "templateexample.h"
template<> class Example<int>
{
    public:
        Example() : a(0), b(1), c(2), d(3) {} 
        int doWork() {return a+b+c+d;}

    private:
        int a; //<== the specialized object will be larger in memory
        int b;
        int c;
        int d;
};
#endif
// --------------------------------

Have a class which includes only the template class definition, but should include the specialization.

// - a.h --------------------------
#ifndef A_H
#define A_H   
#include "templateexample.h"
class A
{
    public:
        Example<int> returnSmallExample();
};
#endif

// - a.cpp ------------------------
#include "a.h"
Example<int> A::returnSmallExample() {return Example<int>();}
// --------------------------------

The main class now knows two versions of Example<int> the one from A and the one from the templatespecialization.h.

// - main.cpp ---------------------
#include <iostream>
#include "a.h"
#include "templatespecialization.h"

int main()
{
    A a;
    Example<int> test = a.returnSmallExample();
    std::cout<<test.doWork()<<std::endl;
}
// --------------------------------

Please note, this problem will only occur when compiling class A separately, this example from ideone outputs 6, whereas using separate files can result in a segmentation fauls, or output 42 (https://ideone.com/3RTzlC). On my machine the example compiles successfully and outputs 2013265920:

proof

In the production version of the above example everything is build into a shared library which is used by main.

Question 1: Why doesn't the linker detect this problem? This should be easy to spot by comparing the size of objects.

Question 2: is there a way to examine the object files or the shared library to detect multiple implementations of the same type like in the example above?


Edit: please note: the code above is a minimal example to explain the problem. The reason for the situation is that the template class is from one library and I cannot edit the files from this library. Finally the whole thing is used all over the place in the executable and now I need to find out if the problem above occurs.


Edit: code above can be compiled like this:

#!/bin/bash 
g++ -g -c a.cpp 
g++ -g -c main.cpp 
g++ -o test a.o main.o 
iammilind
  • 68,093
  • 33
  • 169
  • 336
Beginner
  • 5,277
  • 6
  • 34
  • 71
  • 2
    Your code is fundamentally badly structured. Any time you instantiate a template, *all* specializations must be visible. Don't put some partial specializations into a separate header file. Keep everything in one header. – Kerrek SB Jan 08 '15 at 09:49
  • @KerrekSB this is not under my control. The template header is from a library which I cannot edit. – Beginner Jan 08 '15 at 10:06
  • 7
    But you can write a header of your own that includes the template and then defines the specialization. Use that consistently throughout your program, and never directly include the library's header. – Wyzard Jan 08 '15 at 14:28
  • @Wyzard yes, this would solve the issue. The question is if I can find out if the problem actually exists, using some tricks with nm or objdump for example? – Beginner Jan 08 '15 at 14:31

4 Answers4

7

You have different definition of the same template and its specializations in different translation units. This leads to One Definition Rule violation.

A fix would be to put the specialization in the same header file where the primary class template is defined.

Question 1: Why doesn't the linker detect this problem? This should be easy to spot by comparing the size of objects.

Different types may have the same size (e.g. double and int64_t), so, obviously, just comparing sizes of objects does not work.

Question 2: is there a way to examine the object files or the shared library to detect multiple implementations of the same type like in the example above?

You should use gold linker for linking your C++ applications, if you do not use it already. One nice feature of it is --detect-odr-violations command line switch which does exactly what you ask:

gold uses a heuristic to find potential ODR violations: if the same symbol is seen defined in two different input files, and the two symbols have different sizes, then gold looks at the debugging information in the input objects. If the debugging information suggests that the symbols were defined in different source files, gold reports a potential ODR violation. This approach has both false negatives and false positives. However, it is reasonably reliable at detecting problems when linking unoptimized code. It is much easier to find these problems at link time than to debug cases where the wrong symbol.

See Enforcing One Definition Rule for more details.

Maxim Egorushkin
  • 131,725
  • 17
  • 180
  • 271
  • Upvoted, Really cool feature if this exists! I had to alter my answer :-), because with gold linker, the facility is there of capturing ODR violations. – iammilind Jan 13 '15 at 13:00
  • @iammilind I have been using this switch for the last 4 years. Detected a few ODR violations for me. – Maxim Egorushkin Jan 13 '15 at 13:01
1

Question 1: Why doesn't the linker detect this problem? This should be easy to spot by comparing the size of objects.

Because this is not a linker's problem. In case of templates, the main declaration and all the other specializations (be it class or function) should be visible upfront.

Question 2: is there a way to examine the object files or the shared library to detect multiple implementations of the same type like in the example above?

At least I am not aware of any.

To further simplify this situation, look at a similar broken code:

// foo.h
inline void foo () {
#ifdef FOO
  return;
#else
  throw 0;
#endif
}

// foo1.cpp
#define FOO
#include"foo.h"
// ... uses foo() with "return"

// foo2.cpp
#include"foo.h"
// ... uses foo() with "throw"

It's possible that you get different results based on the way of compilation being used.

Update:
Having multiple body definitions for a same function is undefined behavior. That's the reason why you are getting an awkward output like 2013265920 and the same happens in my machine as well. The output should be either 42 or 6. I gave you above example because with inline functions you can create such race conditions.

With my limited knowledge on linking stage, the responsibility undertaken by a typical linker is limited only till rejecting more than 1 non-inline functions definitions with the same signature. e.g.

// Header.h is included in multiple .cpp files
void foo () {}  // rejected due to multiple definitions
inline void bar () {}  // ok because `inline` keyword is found

Beyond that it doesn't check if the function body is similar or not, because that is already parsed during earlier stage and linker doesn't parse the function body.
Having said above, now pay attention to this statement:

template functions are always inline by nature

Hence linker may not get a chance to reject them.

The safest way is to #include the read-only header into your specialized header and include that specialized header everywhere.

iammilind
  • 68,093
  • 33
  • 169
  • 336
  • @Beginner, updated. In brief, linker just checks if a function is `inline` or not to be able to get qualified to have definition in header. `template` functions are always `inline` and hence always approved by linker. – iammilind Jan 13 '15 at 12:45
  • what is this nonsense – Lightness Races in Orbit Jan 13 '15 at 13:13
  • 1
    read C++11 14.7.3/12 and/or provide a standard citation for this "template functions are always inline" that you keep saying – Lightness Races in Orbit Jan 13 '15 at 13:16
  • @LightnessRacesinOrbit, May you debunk this question: [how does templates work, are they always inlined?](http://stackoverflow.com/questions/5431362/how-does-templates-work-are-they-always-inlined). Only if a `template` function is fully specialized, it's not inline [Does it make any sense to use inline keyword with templates?](http://stackoverflow.com/questions/10535667/does-it-make-any-sense-to-use-inline-keyword-with-templates) – iammilind Jan 13 '15 at 13:19
  • @iammilind: Accepted answer to the first question says _"Templates will be inlined in the standard meaning of `inline`, which is more related to the One Definition Rule than to actual code inlining."_ Again, this is _not_ the same as saying that "`template` functions are always `inline`". Accepted answer to the second question says _"And no, not every function template is inline by default"_ which is the opposite of your point. So again I ask you to provide evidence for your claim. – Lightness Races in Orbit Jan 13 '15 at 13:22
  • @LightnessRacesinOrbit, `inline` has only 1 guaranteed effect that is to maintain ODR (it may work as a hint to compiler for *inlining* purpose). Hence be clear that we are discussing only ODR here. Now that clears your comment on the 1st Question link I posted. In 2nd question link, accepted answer does say that *not every ...*, for that I had already mentioned that fully specialized functions are not `inline` (again ODR) in my previous comment. Let's not drag the discussion. If you don't like then downvote the answer and carry on. – iammilind Jan 13 '15 at 13:30
  • What's the point in downvoting and not leaving a comment to instruct you what is wrong? People always complain about drive-by downvotes, but then they complain about constructive criticism. It's bizarre. – Lightness Races in Orbit Jan 13 '15 at 14:15
0

I don't know of a way to do this by analysis of the compiled binary, but you could build a graph of your program's #include relationships — there are tools that can do this, such as Doxygen — and use it to look for files that (directly or indirectly) include the library header but not the specialization header.

You'd need to examine each file to determine whether it actually uses the template in question, but at least you can narrow down the set of files that you have to examine.

Wyzard
  • 33,849
  • 3
  • 67
  • 87
  • The thing is, I would like to get away from tinkering with includes without even knowing if I solved the problem. If there was a way to get a better understanding of the resulting library, I could be sure I pinpointed the problem. – Beginner Jan 08 '15 at 14:42
  • You don't have to change anything to examine the program's current include graph. And if you "tinker with includes" such that nothing includes the library header except via the specialization header, then you *do* know that you've solved the problem. – Wyzard Jan 08 '15 at 14:47
  • Its a good suggestion and I am doing it right now. But the number of includes is huge and there are many constructs like the one above (I know it's stupid) and the program crashes very rarely. I really would like to understand the resulting lib, to see if this is really the problem. – Beginner Jan 08 '15 at 14:51
0

I think you managed to deceive the compiler. But I should note that in my humble opinion you understand conception of templates in a wrong way, particularly you are trying to mix up template specialization with inheritance. I mean, template specialization must not add data members to class, the only aims are to define types for function parameters and class fields. If you want to change algorithms, i.e. rewrite code, or add new date members to class, you should define derived class. Concerning "several-step" separate compilation and template class in library, C++ reference said (http://www.cplusplus.com/doc/oldtutorial/templates/):

Because templates are compiled when required, this forces a restriction for multi-file projects: the implementation (definition) of a template class or function must be in the same file as its declaration. That means that we cannot separate the interface in a separate header file, and that we must include both interface and implementation in any file that uses the templates. Since no code is generated until a template is instantiated when required, compilers are prepared to allow the inclusion more than once of the same template file with both declarations and definitions in a project without generating linkage errors.

VolAnd
  • 6,367
  • 3
  • 25
  • 43