7

Consider this Makfile: 

all: 
    test 1 -eq 2 | cat
    echo 'done'

It will be executed with no error.

I've heard of set -o pipefail that I may use like this:

all: 
    set -o pipefail;     \
    test 1 -eq 2 | cat;  \
    echo 'done'

Apart that it does not work, this writing is very painful.

Another solution would be to use temporary files. I would like to avoid it.

What other solution can I use?

nowox
  • 25,978
  • 39
  • 143
  • 293
  • How does that second snippet not work? It should work just fine (other than `cat` being pointless there obviously). Or did you mean that it doesn't prevent `done` from being echoed out? It doesn't do that because nothing tells the shell to stop executing. You only have a single line here not multiples so make can't terminate for the `test` failure (even though it propagates past the `| cat`). Drop the `; \` between `cat` and `echo` and it'll do what you expect. – Etan Reisner Nov 25 '15 at 20:40
  • I don't get any errors with my `GNU Make 4.1` – nowox Nov 25 '15 at 20:44
  • With your example code you **shouldn't** get any errors. The `test` failure just gets ignored. Nothing looks at it. You are expecting make to see it and fail the recipe but that only happens for the return code of the *entire* command (normally a single line but in your case the concatenated set of three lines). – Etan Reisner Nov 25 '15 at 21:00
  • In my use case I am using a home made cksum script that I pipe into `dd -of=$@ -seek=$$(( $$(stat -c %s $@) - 4 ))`. If the cksum fail, dd should also fail and I should never execute the rest of the commands. – nowox Nov 25 '15 at 21:10
  • Then if you want to use a continued line in a makefile you need to check for failure and exit yourself because make won't do that for you. `set -o pipefail; \ cksum ... | dd || exit 1; \ echo done` – Etan Reisner Nov 25 '15 at 21:12
  • 1
    Or use two lines `set -o pipefail; \ cksum ... | dd ... || exit 1` and `echo done` *without* merging those together. – Etan Reisner Nov 25 '15 at 21:15
  • Thanks, I've just realized it while reading your previous comment. However the syntax is very annoying. I am looking for a more definitive solution where i tell make to always enable `set -o pipefail` for all pipes. – nowox Nov 25 '15 at 21:15
  • You enable it *for a given shell session*. So you need to make sure you do that for *each* session you have. Each line is a new session. If you don't want that you can use `.ONESHELL` in make 4.1 (but realize that will prevent make from **ever** bailing a recipe out for a recipe-internal non-zero return code so you get to check your commands yourself at all times.\ – Etan Reisner Nov 25 '15 at 21:19
  • And this isn't a syntax problem this is a semantics issue. You don't fully grasp how make works here. – Etan Reisner Nov 25 '15 at 21:19
  • Let us [continue this discussion in chat](http://chat.stackoverflow.com/rooms/96190/discussion-between-nowox-and-etan-reisner). – nowox Nov 25 '15 at 21:34

3 Answers3

18

You can force pipefail on as part of make's SHELL invocation From equivalent of pipefail in GNU make?

SHELL=/bin/bash -o pipefail
Community
  • 1
  • 1
John
  • 181
  • 1
  • 3
  • 1
    Thanks. This should probably be the accepted answer here. Short, concise and solves it generally for all shell invocations in the `Makefile`. – Per Lundberg Sep 30 '20 at 19:11
3

For anything more complicated than single commands I generally prefer using a script. That way you control the interpreter completely (via the shebang line), and you can put more complicated commands together rather than trying to shoe-horn it into effectively a single line. For example:

Makefile:

all:
    ./my.sh

my.sh:

#!/usr/bin/env bash
set -o errexit -o pipefail
test 1 -eq 2 | cat
echo 'done'

That said, the exit code of a Makefile command block like the one you have is the exit code of the last command since you separate the commands with ;. You can use && to execute only until you get an error (equivalent to errexit), like this:

set -o pipefail && test 1 -eq 2 | cat && echo 'done'
l0b0
  • 55,365
  • 30
  • 138
  • 223
  • This could be a solution, but then you will have dozens of scripts related to your Makefile. Then, I am wondering how do you organize it. – nowox Nov 25 '15 at 20:20
  • Like any other complex project, you'll want to make that complexity manageable. And a lot of scripts doing one thing each is generally a lot more manageable than a single huge makefile. – l0b0 Nov 25 '15 at 20:23
  • Do you know good *GitHub* projects that I can take inspiration from? – nowox Nov 25 '15 at 20:26
  • I've got a few [makefile includes](https://github.com/l0b0/make-includes) which I use in various other projects, but I'd just try to keep it as simple as possible. And make sure you use a real language rather than shell at every possible opportunity. The code may be longer, but you'll save yourself so much grief in terms of maintainability and testability. – l0b0 Nov 25 '15 at 20:32
  • That's my very problem. Today I rewrote a script written in Perl that split files by adding checksum at the end of each one for a shell script embedded into the target. I gave a try in Python, but it is too slow to work with (about 5-10 times slower than Perl which is sometime slower than pure shell script. – nowox Nov 25 '15 at 20:36
  • You might want to ask about that script on [codereview.SE](https://codereview.stackexchange.com/). It shouldn't be any appreciably slower in any major language than any other, and certainly not in Python. – l0b0 Nov 25 '15 at 20:44
  • Let us [continue this discussion in chat](http://chat.stackoverflow.com/rooms/96188/discussion-between-nowox-and-l0b0). – nowox Nov 25 '15 at 20:45
1

Setting the SHELL variable in a Makefile works, however Make normally uses /bin/sh as shell, so minor differences creep in.

I grew fond of adding colors to the makeoutput, but these now come out wrong when changing from sh to bash shell.

Here is a makefile

SHELL=/bin/bash #-o pipefail 

LYELLOW ="\\033[1;33m"
NC      ="\\033[0m"

demo1:
    @ set +o pipefail && echo $(LYELLOW)"I will fail"$(NC) && test 1 -eq 2 | cat
    @ echo done
    
demo2:
    @ set -o pipefail && echo $(LYELLOW)"I will fail"$(NC) && test 1 -eq 2 | cat
    @ echo done

demo3:
    @ set -o pipefail && echo -e $(LYELLOW)"I will fail"$(NC) && test 1 -eq 2 | cat
    @ echo done

That by running demo1 and demo2 produces the output:

$ make demo1
\033[1;33mI will fail\033[0m
done

$make demo2
\033[1;33mI will fail\033[0m
make: *** [Makefile:11: demo2] Error 1

that is all very well with using the (outcommented in the Makefile above) global pipefail ala SHELL=/bin/bash -o pipefail or just by adding perline pipefails ala

set -o pipefail && "some commands>"

But colors are now output correctly anymore (the backslashes now comes out directly to the terminal, colors also being hard to show here, anyway).

The fix for this it to add an -e on the echo statements using the bash shell as in the demo3 target, somehow not needed when running sh.

Carsten Eie Frigaard
  • 33
  • 4