How do I make sure erroneous code does not build in a CI process?

Question

I am writing some C++ library, and I already use some CI (Continuous Integration) process, to make sure the features work. The build is makefile-based, so the CI script holds a make test line. That target will run the automated tests (say, using Catch), and build passes only if it succeeds.

The good practices mandates that a wrong usage of the library by user code should preferably trigger a compile error, instead of a run-time error. So I included lots of static checking.

However, I want to make sure in my automated testing process that this is really the case (that user code using the library erroneously will fail to build).

So I put up a folder holding several .cpp files, each demonstrating an incorrect usage of the library, thus each of them should fail to build.

My problem is that if I add a new target in my makefile to build these, it will return after first failure.

What I want to do is make sure that all these files fail to build, and return 0 only in that case.

How can I write a target that performs this ?

Some (not working) code, that will return after first failure.

SRC_FILES := $(wildcard *.cpp)
OBJ_FILES := $(patsubst %.cpp, %.o, $(SRC_FILES))

all: $(OBJ_FILES)
    @echo "done"

%.o: %.cpp
    $(CXX) -o $@ -c $<

If I basically ignore the failures (prepending line with -), I won't be able to check that all of the files are incorrect.

Kinda related: https://stackoverflow.com/questions/2188376/ but that question asks for a null return code if running code fails, not the build. — kebs, Sep 26 '19 at 17:40

score 2 · Accepted Answer · edited Sep 27 '19 at 09:06

2

The only way to do it if you want to use the same pattern rule, is through recursive make and passing the -k flag (--keep-going), like this:

TEST_SRCS := bad1.cpp bad2.cpp bad3.cpp

.PHONY: test_compile
test_compile:
          $(MAKE) -k $(TEST_SRCS:%.cpp=%.o)

Another option is to compile them all in a loop in a single recipe:

test_compile:
         ok=true; \
         for src in $(TEST_SRCS); do \
           $(CXX) -o $${src%.cpp}.o -c $$src || ok=false; \
         done; \
         $$ok

edited Sep 27 '19 at 09:06

kebs

6,387
4
41
70

answered Sep 26 '19 at 19:23

MadScientist

92,819
9
109
136

Thanks for taking a look, I'll check and try that ASAP. – kebs Sep 26 '19 at 20:50

bobbogo · Answer 2 · 2019-09-27T15:51:50.610

srcs := fail1.cpp fail2.cpp fail3.cpp
failed := ${srcs:.cpp=.failed}

PHONY: test
test: ${failed} ; : $@ Success

${failed}: %.failed: %.cpp
    ! gcc blah blah $< -o ${@:.failed=.o}
    touch $@

What happens here is that when you want to build test, make first tries to build fail1.failed.

If the compile succeeds, make fails with an error.

OTOH if the compile fails, the output file fail1.failed is created.

BTW, make knows that fail1.failed depends on fail1.cpp. If the .failed is newer than the .cpp, the test will not even run. Nice.

Same for fail2.failed and fail3.failed. Parallel safe, and if you get your dependencies right, only the tests that need to run get run. The whole point of make really.

Thanks for answer, interesting approach, I'll try. – kebs Sep 27 '19 at 16:14 — kebs, Sep 27 '19 at 16:14

score 0 · Answer 3 · answered Sep 30 '19 at 15:45

Accepting answer form @MadScientist but for the record, seems this can be done also this way:

SRC_FILES := $(wildcard *.cpp)
OBJ_FILES := $(patsubst %.cpp, %.o, $(SRC_FILES))

all: $(OBJ_FILES)
    @echo "done"

%.o: %.cpp
    ! $(CXX) -o $@ -c $<

Just by reversing the return value from the compile shell command: if a single file succeeds, then make will immediately exit with an "error" return value.

How do I make sure erroneous code does not build in a CI process?

3 Answers3