using extern template (C++11) to avoid instantiation

Question

Figure 1: function templates

TemplHeader.h

template<typename T>
void f();

TemplCpp.cpp

template<typename T>
void f(){
   //...
}    
//explicit instantation
template void f<T>();

Main.cpp

#include "TemplHeader.h"
extern template void f<T>(); //is this correct?
int main() {
    f<char>();
    return 0;
}

Is this the correct way to use extern template, or do I use this keyword only for class templates as in Figure 2?

Figure 2: class templates

TemplHeader.h

template<typename T>
class foo {
    T f();
};

TemplCpp.cpp

template<typename T>
void foo<T>::f() {
    //...
}
//explicit instantation
template class foo<int>;

Main.cpp

#include "TemplHeader.h"
extern template class foo<int>();
int main() {
    foo<int> test;
    return 0;
}

I know it is good to put all of this in one header file, but if we instantiate templates with the same parameters in multiple files, then we got multiple same definitions and the compiler will remove them all (except one) to avoid errors. How do I use extern template? Can we use it only for classes, or can we use it for functions too?

Also, Figure 1 and Figure 2 may be expanded to a solution where templates are in a single header file . In that case, we need to use the extern template keyword to avoid multiple same instantations. Is this only for classes or functions too?

Answer 1

You should only use extern template to force the compiler to not instantiate a template when you know that it will be instantiated somewhere else. It is used to reduce compile time and object file size.

For example:

// header.h

template<typename T>
void ReallyBigFunction()
{
    // Body
}

// source1.cpp

#include "header.h"
void something1()
{
    ReallyBigFunction<int>();
}

// source2.cpp

#include "header.h"
void something2()
{
    ReallyBigFunction<int>();
}

This will result in the following object files:

source1.o
    void something1()
    void ReallyBigFunction<int>()    // Compiled first time

source2.o
    void something2()
    void ReallyBigFunction<int>()    // Compiled second time

If both files are linked together, one void ReallyBigFunction<int>() will be discarded, resulting in wasted compile time and object file size.

To not waste compile time and object file size, there is an extern keyword which makes the compiler not compile a template function. You should use this if and only if you know it is used in the same binary somewhere else.

Changing source2.cpp to:

// source2.cpp

#include "header.h"
extern template void ReallyBigFunction<int>();
void something2()
{
    ReallyBigFunction<int>();
}

Will result in the following object files:

source1.o
    void something1()
    void ReallyBigFunction<int>() // compiled just one time

source2.o
    void something2()
    // No ReallyBigFunction<int> here because of the extern

When both of these will be linked together, the second object file will just use the symbol from the first object file. No need for discard and no wasted compile time and object file size.

This should only be used within a project, like in times when you use a template like vector<int> multiple times, you should use extern in all but one source file.

This also applies to classes and function as one, and even template member functions.

Answer 2

Wikipedia has the best description

In C++03, the compiler must instantiate a template whenever a fully specified template is encountered in a translation unit. If the template is instantiated with the same types in many translation units, this can dramatically increase compile times. There is no way to prevent this in C++03, so C++11 introduced extern template declarations, analogous to extern data declarations.

C++03 has this syntax to oblige the compiler to instantiate a template:

  template class std::vector<MyClass>;

C++11 now provides this syntax:

  extern template class std::vector<MyClass>;

which tells the compiler not to instantiate the template in this translation unit.

_{^{The warning: nonstandard extension used...}}

Microsoft VC++ used to have a non-standard version of this feature for some years already (in C++03). The compiler warns about that to prevent portability issues with code that needed to compile on different compilers as well.

Look at the sample in the linked page to see that it works roughly the same way. You can expect the message to go away with future versions of MSVC, except of course when using other non-standard compiler extensions at the same time.

Answer 3

extern template is only needed if the template declaration is complete

This was hinted at in other answers, but I don't think enough emphasis was given to it.

What this means is that in the OP's examples, the extern template has no effect because the template definitions on the headers were incomplete:

• void f();: just declaration, no body
• class foo: declares method f() but has no definition

So I would recommend just removing the extern template definition in that particular case: you only need to add them if the classes are completely defined.

For example:

TemplHeader.h

template<typename T>
void f();

TemplCpp.cpp

template<typename T>
void f(){}

// Explicit instantiation for char.
template void f<char>();

Main.cpp

#include "TemplHeader.h"

// Commented out from OP code, has no effect.
// extern template void f<T>(); //is this correct?

int main() {
    f<char>();
    return 0;
}

compile and view symbols with nm:

g++ -std=c++11 -Wall -Wextra -pedantic -c -o TemplCpp.o TemplCpp.cpp
g++ -std=c++11 -Wall -Wextra -pedantic -c -o Main.o Main.cpp
g++ -std=c++11 -Wall -Wextra -pedantic -o Main.out Main.o TemplCpp.o
echo TemplCpp.o
nm -C TemplCpp.o | grep f
echo Main.o
nm -C Main.o | grep f

output:

TemplCpp.o
0000000000000000 W void f<char>()
Main.o
                 U void f<char>()

and then from man nm we see that U means undefined, so the definition did stay only on TemplCpp as desired.

All this boils down to the tradeoff of complete header declarations:

• upsides:
  • allows external code to use our template with new types
  • we have the option of not adding explicit instantiations if we are fine with object bloat
• downsides:
  • when developing that class, header implementation changes will lead smart build systems to rebuild all includers, which could be many many files
  • if we want to avoid object file bloat, we need not only to do explicit instantiations (same as with incomplete header declarations) but also add extern template on every includer, which programmers will likely forget to do

Further examples of those are shown at: Explicit template instantiation - when is it used?

Since compilation time is so critical in large projects, I would highly recommend incomplete template declarations, unless external parties absolutely need to reuse your code with their own complex custom classes.

And in that case, I would first try to use polymorphism to avoid the build time problem, and only use templates if noticeable performance gains can be made.

Tested in Ubuntu 18.04.

Answer 4

The known problem with the templates is code bloating, which is consequence of generating the class definition in each and every module which invokes the class template specialization. To prevent this, starting with C++0x, one could use the keyword extern in front of the class template specialization

#include <MyClass>
extern template class CMyClass<int>;

The explicit instantion of the template class should happen only in a single translation unit, preferable the one with template definition (MyClass.cpp)

template class CMyClass<int>;
template class CMyClass<float>;

Answer 5

If you have used extern for functions before, exactly same philosophy is followed for templates. if not, going though extern for simple functions may help. Also, you may want to put the extern(s) in header file and include the header when you need it.