Building a C++ template library with make

Question

I am having difficulties with re-writting my C++ library into template form; main problem concerns re-designing the Makefile.

In the previous state, when it was non-template I had:

a header file lib.h in which (protected by include guards) we have the declarations of classes and overloaded operators. This file does not include any other libraries at all.
an implementation file lib.cpp in which, on top of the file, I include many headers from the standard library (cmath, iostream, etc) as well as I include the header file for this custom library: #include "lib.h"
a Makefile with commands for build/install the library:

CC = g++
CFLAGS = -O2
SRC = lib.cpp
HDR = $(SRC:.cpp=.h)
OBJ = $(SRC:.cpp=.o)
LIB = $(OBJ:.o=.a)

.PHONY: all install clean uninstall

# =========================================================
# Build
# =========================================================

all: $(LIB)

# Create an object file
$(OBJ):
    $(CC) $(CFLAGS) -c -o $@ $(SRC)

# Create a static library file
$(LIB): $(OBJ)
    ar rcs $@ $<

# =========================================================
# Install
# =========================================================

install: ~/lib/lib/$(LIB) ~/lib/include/$(HDR)

# Create top-level directory for the libraries
~/lib:
    mkdir -p $@;

# Create top-level directory for the static library files
~/lib/lib:
    mkdir -p $@;

# Create top-level directory for the headers
~/lib/include:
    mkdir -p $@;

# Copy the library file into the right directory
~/lib/lib/$(LIB): $(LIB) ~/lib/lib
    cp $< $@

# Copy the header file into the right directory
~/lib/include/$(HDR): $(HDR) ~/lib/include
    cp $< $@

following the installation my CI (GH Actions) would compile a small test program, which includes my library header (#include <lib.h>) with the following command:

g++ -O0 -Wall --std=c++14 test.cpp -I ~/lib/include/ ~/lib/lib/lib.a -o test

This setup worked fine.

Problems emerge now, when I want to re-write my class to be template.

Following the information in this post I have added an #include "lib.cpp" at the end of the header file for my library (inside the include guard, ofc). With that change I needed to adjust the compilation process and do not provide my implementation file in the command line (as it is already included in the header and I have to avoid re-definition errors). That works fine. The core of the problem is now in the Makefile, in the command to build an object file. As I try to compile the implementation file of my library, it includes the header and the header includes the implementation again... I red about the problem in this port and they suggest to remove the header inclusion from the implementation file. So I did that, I commented the #include "lib.h" and tried to run:

g++ -O2 -c -o lib.o lib.cpp

And I end up with a lot of error: use of undeclared identifier errors...

How can I build the library properly with make? My constraint is that the library stays in two separate files: header and implementation. What I want in the end is to be able to #include <lib.h> in my further programs.

Is it even possible to create an archived object file? What about shared object library (.so)

With a typical template library, everything is in the headers. There's nothing to build or link. See also: [header-only library](https://en.wikipedia.org/wiki/Header-only). Why again do you insist on having two separate files? — Igor Tandetnik, Jan 09 '21 at 15:27
It's not possible to distribute template code as object files, unless you use explicit template instantiation (which is a significant limitation on the usefulness of your library). Attempts to fake it by including cpp files in header files bring no benefit. You should do what everyone else does and make a header only library. — john, Jan 09 '21 at 15:30
The compilation of my library might take too much time, a header-only version would require to recompile the whole thing each time anything changes downstream, I would like to avoid that. — maciek, Jan 09 '21 at 15:39
That's the nature of template libraries - the code cannot be compiled until a template is instantiated with the particular set of template arguments.. You can't have both templates and separate compilation; pick one. Including `.cpp` file into `.h` file doesn't give you separate compilation - whenever the `.cpp` file changes, everything that includes it (indirectly via the `.h` file) would need to be recompiled. That the file has `.cpp` extension and not `.h` doesn't change this fact - file extensions are merely a naming convention for convenience of humans, they don't possess magic properties — Igor Tandetnik, Jan 09 '21 at 16:09
That's the nature of *templates*, period. You cannot compile templates to object code, only their instantiations. And it is the code using the library that determines what instantiations are needed. And yes, this is among the reasons that C++ compilation is often more costly than, say, C compilation. — John Bollinger, Jan 09 '21 at 16:54

score 1 · Accepted Answer · answered Jan 09 '21 at 19:05

I have added an #include "lib.cpp" at the end of the header file for my library (inside the include guard, ofc).

I can't say I think much of the advice you followed there. Code that is intended to be used as or in a header should be named appropriately (.h, .hpp, or whatever convention your project follows), and code that is suitable for direct compilation pretty much never belongs in a header. Perhaps your conversion involves changing everything of the latter kind into to former kind, so maybe you want to rename lib.cpp to lib_impl.h, or something similar, and skip trying to compile it at all. Maybe.

Note well, however, that if you name and structure the implementation code as a header, then it needs its own include guards. Note also that it must not, then, contain any external, non-template functions. If it did, then no two separate translation units contributing to the same program could both include the header, as that would result in duplicate function definitions, include guards notwithstanding. And if there are no external, non-template functions (and no external object definitions) then it is pointless to try to compile to an object file, as there would be no accessible entry point to any function within.

How can I build the library properly with make?

This isn't really about make in particular. That just provides automation. The issue is with the structure of the library itself and your expectations.

My constraint is that the library stays in two separate files: header and implementation.

This constraint makes sense only if the implementation contains any instantiated templates for external objects or functions, or any external non-template functions or objects. These are the things that would contribute to a buildable object file that you could directly or indirectly link to applications. If converting to a template library means that there are no longer any such entities in your library then the constraint is arbitrary, as the converted result is a header-only library. Nevertheless, you can split up your headers any way you like, as long as each resulting one is structured as a header, with only contents appropriate for a header, and with its own inclusion guards.

On the other hand, if your converted implementation code does contain any of those things then they must not be #included into any header, as discussed above.

What I want in the end is to be able to #include <lib.h> in my further programs.

If the converted implementation code is suitable for use as or in a header, then you are already there, but it would be much better style to rename your lib.cpp as a header. If you want to be able to include that header directly into code other than the main library header then it needs its own inclusion guards -- that will take care of the duplicate declarations. Do note, however, that those errors arise from the fact that you have a circular dependency, and that's a strong sign that you ought to refactor. Such a refactoring would involve moving enough code from lib.cpp to lib.h that the circular dependency can be removed (one of the files would no longer #include the other).

Whatever implementation code is not suitable for use in a header obviously must not be included into the header. If any such code remains in the converted library then perhaps you keep that in lib.cpp, and move the rest into lib.h

Is it even possible to create an archived object file? What about shared object library (.so)

Templates cannot be compiled. They are templates for code that can be compiled: their instantiations. If your conversion leaves nothing else then no, you cannot usefully create an object file or a shared library. And you don't need to do, for this is a header-only library.

Building a C++ template library with make

1 Answers1