0

I'm working on the issue described below. When researching it, the GNU Make manual, Section 6.11, says:

Variable values in make are usually global; that is, they are the same regardless of where they are evaluated ... One exception to that is automatic variables (see Automatic Variables).

The other exception is target-specific variable values. This feature allows you to define different values for the same variable, based on the target that make is currently building. As with automatic variables, these values are only available within the context of a target’s recipe (and in other target-specific assignments).

Set a target-specific variable value like this:

target … : variable-assignment

Multiple target values create a target-specific variable value for each member of the target list individually.

...

There is one more special feature of target-specific variables: when you define a target-specific variable that variable value is also in effect for all prerequisites of this target, and all their prerequisites, etc. (unless those prerequisites override that variable with their own target-specific variable value). So, for example, a statement like this:

prog : CFLAGS = -g
prog : prog.o foo.o bar.o

...

In the context of the GNU Make example above, I need something like:

Intel machine with SSE4.2:

crc-simd.o: CRC_FLAG = -msse4.2
crc-simd.o:
    $(CXX) $(CXXFLAGS) $(CRC_FLAG) -c $<

ARMv8a machine with CRC32:

crc-simd.o: CRC_FLAG = -march=armv8-a+crc
crc-simd.o:
    $(CXX) $(CXXFLAGS) $(CRC_FLAG) -c $<

I think Define make variable at rule execution time may be related, but the details are still foggy for me. I'm not sure if it is the keystone or just another alternative.

Is a target-specific variable the proper tool to conditionally set the CRC_FLAG variable? If it is, then how do we use it to conditionally set a variable?

If not, then is there a way to assign a value to CRC_FLAG only when crc-simd.o needs to be built? If there is, then how do we do it?

Here's what our GNUmakefile contains. Though ARMv8a is shown, x86/x32/x64 is similar. And we do similar for SSE4.2, NEON, AES, SHA, CLMUL, AVX and BMI.

After the test compile, CRC_FLAG will be empty on MIPS, take -msse4.2 when SSE4.2 is available, and -march=armv8-a+crc on ARMv8a. The problem is, it makes recipes like make clean take too much time to run. Even building one dirty object file incurs the full wrath of all the compiles. The lag is noticeable.

TEMPDIR ?= /tmp
EGREP ?= egrep

IS_ARMV8 ?= $(shell uname -m | $(EGREP) -i -c 'aarch32|aarch64')
...

ifeq ($(IS_ARMV8),1)
  HAS_CRC := $(shell $(CXX) $(CXXFLAGS) -march=armv8-a+crc -o $(TEMPDIR)/t.o -c crc-simd.cpp; echo $$?)
  ifeq ($(HAS_CRC),0)
    CRC_FLAG := -march=armv8-a+crc
  endif
endif
...

# SSE4.2 or ARMv8a available
crc-simd.o : crc-simd.cpp
    $(CXX) $(strip $(CXXFLAGS) $(CRC_FLAG) -c) $<
jww
  • 97,681
  • 90
  • 411
  • 885
  • You seem to have answered your original question, now it sounds like your question should be "how do I stop `make clean` from rebuilding everything", but you haven't provided a complete example. – user657267 Jul 29 '17 at 21:36
  • Thanks @user657267. *"how do I stop make clean from rebuilding everything"* - there's actually more rules that need to avoid building everything. Hence the reason I was trying to be surgical about populating `CRC_FLAG`. Sorry about the confusion. – jww Jul 29 '17 at 21:48
  • I don't know what you mean by _incurs the full wrath of all the compiles_. All what compiles? You mean, the `HAS_CRC := $(shell $(CXX) ...)` compiles? I don't see why all of them would run; wouldn't only the one compile for this particular value of `IS_...` run? – MadScientist Jul 29 '17 at 23:11
  • @MadScientist - If I need to build `foo.o` and `bar.o`, they will cause the `HAS_CRC` gear to operate because I don't know how to say, *"Only evaluate the `HAS_CRC` and only set the value for `CRC_FLAG` variable when building `crc-simd.o`"*. – jww Jul 29 '17 at 23:17
  • OK, but it only runs once per invocation of make, and only one of them runs, correct? So, it's basically one extra compilation per invocation of make, right? I get that this could be annoying but I'm trying to understand your comments about _incurs the full wrath of all compiles_. – MadScientist Jul 29 '17 at 23:20
  • @MadScientist - Regarding the wrath of the compiler, we don't use Autotools, so we configure on the fly with GNUmakefile. `CRC_FLAG` invokes the compiler to see if `-msse4.2` or `-march=armv8-a+crc` is consumed by the compiler. Now, multiply that by 12 to 24 for other features. And do it on a CubieTruck or BeagleBoard. It takes make 5 minutes to start building the stuff on CubieTruck or BeagleBoard. The 5 minute penalty is even incurred by `foo.o` and `bar.o`, which don't use any of the features. – jww Jul 29 '17 at 23:35
  • OK but I assume you're not testing for sse4.2 on ARM and not testing for armv8-a+crc on Intel, right? You only test for the options that could exist based on the `IS_...` option obtained from `uname -m`? It also seems like you could probably avoid lots of invocations of `uname -m`; why not just run it one time then do the rest of the testing in the makefile? It would be a lot faster. – MadScientist Jul 29 '17 at 23:46

1 Answers1

1

If you only need this flag for a single .o file like crc-simd.o, then why use := in the assignment of CRC_FLAG? If you use = instead then it won't be expanded until it's used and if you only use it once it will only be expanded once.

Something like:

ifeq ($(IS_ARMV8),1)
  CRC_FLAG = $(shell $(CXX) $(CXXFLAGS) -march=armv8-a+crc -o $(TEMPDIR)/t.o -c crc-simd.cpp 2>/dev/null && printf %s -march=armv8-a+crc)
endif

# SSE4.2 or ARMv8a available
crc-simd.o : crc-simd.cpp
        $(CXX) $(strip $(CXXFLAGS) $(CRC_FLAG) -c) $<

This still compiles the file twice; it seems like there should be a faster way to determine if that flag is supported but anyway.

MadScientist
  • 92,819
  • 9
  • 109
  • 136
  • *" it seems like there should be a faster way to determine if that flag is supported but anyway..."* - Lol, yeah I know what you mean. The problem is mostly ARM targets. GCC and Clang are pretty f'd up when it comes to a native build. They act like they are ARMv6 on Aarch64, except they generate code for Aarch64. Otherwise, GCC advertises nothing. See, for example, [Why is __ARM_FEATURE_CRC32 not being defined by the compiler?](https://stackoverflow.com/q/37066261/608639) – jww Jul 29 '17 at 23:40
  • So, the output of `gcc -march=armv8-a --target-help` doesn't show the `crc` extension when it's available and not when it isn't available? – MadScientist Jul 29 '17 at 23:41
  • Thanks @MadScientist. I did not see anything interesting. No hits like crc, aes, sha, pmull, asimd. Here's the Pastebin, if interested: [`gcc -march=armv8-a --target-help`](https://pastebin.com/0tgWisfu). – jww Jul 30 '17 at 00:00
  • I believe you answered this question: *"If not, then is there a way to assign a value to CRC_FLAG only when crc-simd.o needs to be built? If there is, then how do we do it?"* Is that correct (you answered it)? – jww Jul 30 '17 at 00:39
  • That is the question I was answering, yes. If you use a recursive variable assignment then the value won't be expanded when the variable is defined; instead it's expanded when the variable is used. If the variable is only used one time in the recipe for the `crc-simd.o` target then it will be expanded only when that target is built. – MadScientist Jul 30 '17 at 16:52
  • Ack, thanks. Your suggestion about avoiding the compiles worked really well. I'm checking the preprocessor for defines in most cases; see [`GNUmakefile`](https://github.com/noloader/cryptopp/blob/master/GNUmakefile#L309) (ARMv8) and [`GNUmakefile`](https://github.com/noloader/cryptopp/blob/master/GNUmakefile#L201) (Intel). – jww Jul 30 '17 at 16:56
  • You probably want to jot a comment to yourself (or some future person) about why this is a recursive variable assignment. The rule of thumb for variables containing `$(shell ...)` functions is always use simple variables (`:=`) so documenting why that is not done here would be wise. Cheers! – MadScientist Jul 30 '17 at 18:31