Timing of Makefile "include" statements for auto-generated files

Question

In this Makefile...

all:    piped.mk
ifeq ($(PIPED),1)
    @echo Output of make is piped
else
    @echo Output of make is NOT piped
endif

piped.mk:
    [ -t 1 ] && PIPED=0 || PIPED=1 ; echo "PIPED=$${PIPED}" > piped.mk

.PHONY: piped.mk all

include piped.mk

...I would expect the following:

• The first rule, all, says it depends on the file piped.mk.
• The file piped.mk is generated by asking the shell whether the terminal's stdout is a TTY or not
• The file piped.mk is included at the end; in theory therefore this should trigger makes "remaking" logic (see section 3.5 in the make manual)
• To force the creation of piped.mk (since it is not supposed to depend on files, but on the way make was invoked), it is also marked as a .PHONY target - one that should be remade no matter what.

If these assumptions are correct, I don't understand this result:

$ make
[ -t 1 ] && PIPED=0 || PIPED=1 ; echo "PIPED=${PIPED}" > piped.mk
Output of make is NOT piped

$ cat piped.mk 
PIPED=0

$ make | more
[ -t 1 ] && PIPED=0 || PIPED=1 ; echo "PIPED=${PIPED}" > piped.mk
Output of make is NOT piped

$ cat piped.mk 
PIPED=1

It appears that file piped.mk is always remade, as intended - and it indeed contains the intended information (whether the standard output is piped or not). The Makefile however, seems to ignore the value of PIPED - it always reports "NOT piped"; as if piped.mk is not re-included after its regeneration...

UPDATE

Traditionally, the prototypical example of automatically generated Makefile parts is the generation of the C file dependencies - in which case all examples I've seen end up using include at the bottom of the Makefile. However, maybe this approach only works for the generation of dependencies, and does not work for the assignment of Makefile variables...

To check if this was the reason, I attempted to move the include up to the top...

$ cat Makefile
include piped.mk

all: ...

...which caused a nasty "delay" effect:

$ rm -f piped.mk
$ make | more
[ -t 1 ] && PIPED=0 || PIPED=1 ; echo "PIPED=${PIPED}" > piped.mk
Output of make is NOT piped

$ make | more
[ -t 1 ] && PIPED=0 || PIPED=1 ; echo "PIPED=${PIPED}" > piped.mk
Output of make is piped

Basically, the inclusion does happen - but it happens BEFORE the piped.mk generation. It therefore uses the "old" value of the definition of PIPED, not the new one that was just placed inside piped.mk upon execution. Maybe the re-inclusion of piped.mk is only allowed to update dependency rules and not variable definitions?

So, my question:

Is it possible to auto-generate a part of a Makefile that contains variable definitions, and include it within the same Makefile invocation?

That is, without a separate $(MAKE) invocation - which is the only way I have found so far to make it work:

all:
        @[ -t 1 ] && PIPED=0 || PIPED=1 ; \
            $(MAKE) binary "PIPED=$${PIPED}"

binary:
ifeq ($(PIPED),1)
        @echo Output of make is piped
else
        @echo Output of make is NOT piped
endif

.PHONY: all binary

Thanks for any suggestions.

Answer 1

I am skeptical about the logic in your assignment, but you can do it this way:

PIPED = $(shell [ -t 1 ] && PIPED=0 || PIPED=1 ; echo "$${PIPED}")

Ad for why the reinvocation approach didn't work, it isn't that reinvocation doesn't reevaluate variables, it's that you weren't reinvoking Make at all; rebuilding an included file doesn't trigger reinvocation if that file is PHONY.

Answer 2

First of all, many thanks to @Beta for his feedback.

@Beta recommended using the $(shell ...) construct to execute the [-t 1] check; but unfortunately this can't work in this case. The intent is to detect whether the Makefile is processed in a piped execution...

$ make | less     # page/search interactively in the build log
$ make | grep ... # Bulk search for stuff in the build log
$ make > log.txt  # Or use the log.txt from within e.g. a LaTEX doc...

...or not. Think e.g. of avoiding the printing of color codes in the output (which would mess up re-using the output as-is from within a LaTEX document, or confuse your pager, grep, etc). This construct...

PIPED:=$(shell [ -t 1 ] ... )

...when embedded in the Makefile doesn't work; because stdout is already redirected inside make, to capture the output and assign it to a variable.

@Beta additionally gave another suggestion, which was key to solving this - it was about my use of .PHONY:

...rebuilding an included file doesn't trigger reinvocation if that file is PHONY.

Indeed, this explained why I would see the re-execution of the rule that created piped.mk, but the file would not be re-included; and therefore the PIPED variable stayed the same.

To avoid that, I removed the .PHONY spec altogether, and added an action at the end of the all rule, to remove the piped.mk (thus forcing it to be re-created next time make is invoked):

all:    piped.mk
ifeq ($(PIPED),1)
        @echo Output of make is piped because PIPED is ${PIPED}
else
        @echo Output of make is NOT piped because PIPED is ${PIPED}
endif
        @rm -f piped.mk

piped.mk:
        [ -t 1 ] && PIPED=0 || PIPED=1 ; echo "PIPED=$${PIPED}" > piped.mk

-include piped.mk

Using this, I saw the following result:

$ make
[ -t 1 ] && PIPED=0 || PIPED=1 ; echo "PIPED=${PIPED}" > piped.mk
Output of make is NOT piped because PIPED is 0

Good. And for piping...

$ make | more
[ -t 1 ] && PIPED=0 || PIPED=1 ; echo "PIPED=${PIPED}" > piped.mk
Output of make is NOT piped because PIPED is 1

Ooops! Clearly piped.mk was re-included, and PIPED was re-evaluated (since the action reported because PIPED is 1) - but the rule that was executed, amazingly, was the wrong one in the ifeq!

ifeq ($(PIPED),1)
        @echo Output of make is piped because PIPED is ${PIPED}
else
        @echo Output of make is NOT piped because PIPED is ${PIPED}
endif

Note: I seriously doubt anyone out there would expect that a Makefile with these 5 lines could ever print Output of make is NOT piped because PIPED is 1. In fact, I'd consider this to be a bug in make.

My final idea was that maybe make DOES NOT re-evaluate the check conditions that are BEFORE the point where the re-inclusion took place - and I moved the include on top:

-include piped.mk

all:    piped.mk
ifeq ($(PIPED),1)
    @echo Output of make is piped because PIPED is ${PIPED}
else
    @echo Output of make is NOT piped because PIPED is ${PIPED}
endif
    @rm -f piped.mk

piped.mk:
    [ -t 1 ] && PIPED=0 || PIPED=1 ; echo "PIPED=$${PIPED}" > piped.mk

This, finally, worked:

$ make
[ -t 1 ] && PIPED=0 || PIPED=1 ; echo "PIPED=${PIPED}" > piped.mk
Output of make is NOT piped because PIPED is 0

$ make | more
[ -t 1 ] && PIPED=0 || PIPED=1 ; echo "PIPED=${PIPED}" > piped.mk
Output of make is piped because PIPED is 1

So, to summarize:

• Never place included files in .PHONY rules - if you want them re-evaluated every time, just make sure to remove them in the end of your processing.
• If your generated files contain dependencies - like the '.d' files described in the make manual - it is OK to include them at the end. If, however, they contain variable assignments, make sure to include them BEFORE you use/check said variables.

Again, many thanks to @Beta for pointing me in the right direction.

UPDATE:

And a few months later, I realized that since the variable was re-evaluated (after removing it from the .PHONY), I could also avoid using ifeq and instead decide in the shell itself:

all:
    @[ ${PIPED} -eq 1 ] && echo Piped. || echo Not piped.
    @rm -f piped.mk

piped.mk:
    @[ -t 1 ] && PIPED=0 || PIPED=1 ; echo "PIPED=$${PIPED}" > piped.mk

include piped.mk