Autotools: GCC's Makefile.in has a bug (7.3.0 and earlier). How to re-program automake to find 'ar' and 'objdump'

Question

GCC's cross compiling autotools is supposed to be flexible, but I've isolated a bug that's been breaking cross compiler builds that ought to work.

Note: Some systems will "poison" default compiler tool names to prevent using wrong tools by default. On my system, x86_64-pc-gnu-linux-ar will execute but "ar" is not found.

I need to build cross compiler toolchains with custom names. gcc's configure script supports this with --program-prefix or --program-transform-name. However, when using a custom name, all compile time tools have to be explicitly named on the configure line. gcc configure is not intelligent enough to find tools it has just built with a name change. (too stupid).

The GCC manual states how to explicitly name tools:

configure AR=x86_foo_b_ar AR_FOR_TARGET=ARMv6m_foo_b_ar ...

However, it doesn't work right. Autoools sometimes ignores the supplied names and the build fails. In particular, it ignores 'AR' and 'OBJDUMP' variables.

Apparently the toplevel gcc configure was created at a later date than lower level configures.

Makefile.in without Makefile.am in GCC?

Makefile.am does not exist in some subdirectories, but it does exist in newer subdirectories.

This causes inconsistencies in variable passing from the top-level makefile.

Internally, the top level "configure" script has variables AR_FOR_HOST (alias for AR), AR_FOR_BUILD, and AR_FOR_TARGET. These variables are used to re-define "AR" when entering sub-directories to force a generic make script to compile for a particular target.

I've even gone so far as to define the internal variables correctly as well as "AR" and "OBJDUMP" on the configure command line. ( Shouldn't be needed ).

gcc-7.3.0/configure --host=x86_64-pc-linux-gnu --program-prefix=armv6m-softfloat-eabi-newlib- AR_FOR_BUILD=/usr/bin/x86_64-pc-linux-gnu-ar AR=/usr/bin/x86_64-pc-linux-gnu-ar AR_FOR_HOST=/usr/bin/x86_64-pc-linux-gnu-ar AR_FOR_TARGET=/usr/libexec/gcc/armv6m-softfloat-eabi-newlib/ar AS_FOR_BUILD=/usr/bin/x86_64-pc-linux-gnu-as AS=/usr/bin/x86_64-pc-linux-gnu-as AS_FOR_HOST=/usr/bin/x86_64-pc-linux-gnu-as AS_FOR_TARGET=/usr/libexec/gcc/armv6m-softfloat-eabi-newlib/as DLLTOOL_FOR_TARGET=/usr/libexec/gcc/armv6m-softfloat-eabi-newlib/dlltool LD_FOR_BUILD=/usr/bin/x86_64-pc-linux-gnu-ld LD=/usr/bin/x86_64-pc-linux-gnu-ld LD_FOR_HOST=/usr/bin/x86_64-pc-linux-gnu-ld LD_FOR_TARGET=/usr/libexec/gcc/armv6m-softfloat-eabi-newlib/ld LIPO_FOR_TARGET=/usr/libexec/gcc/armv6m-softfloat-eabi-newlib/lipo NM_FOR_BUILD=/usr/bin/x86_64-pc-linux-gnu-nm NM=/usr/bin/x86_64-pc-linux-gnu-nm NM_FOR_HOST=/usr/bin/x86_64-pc-linux-gnu-nm NM_FOR_TARGET=/usr/libexec/gcc/armv6m-softfloat-eabi-newlib/nm OBJCOPY_FOR_BUILD=/usr/bin/x86_64-pc-linux-gnu-objcopy OBJCOPY=/usr/bin/x86_64-pc-linux-gnu-objcopy OBJCOPY_FOR_HOST=/usr/bin/x86_64-pc-linux-gnu-objcopy OBJCOPY_FOR_TARGET=/usr/libexec/gcc/armv6m-softfloat-eabi-newlib/objcopy OBJDUMP_FOR_BUILD=/usr/bin/x86_64-pc-linux-gnu-objdump OBJDUMP=/usr/bin/x86_64-pc-linux-gnu-objdump OBJDUMP_FOR_HOST=/usr/bin/x86_64-pc-linux-gnu-objdump OBJDUMP_FOR_TARGET=/usr/libexec/gcc/armv6m-softfloat-eabi-newlib/objdump RANLIB_FOR_BUILD=/usr/bin/x86_64-pc-linux-gnu-ranlib RANLIB=/usr/bin/x86_64-pc-linux-gnu-ranlib RANLIB_FOR_HOST=/usr/bin/x86_64-pc-linux-gnu-ranlib RANLIB_FOR_TARGET=/usr/libexec/gcc/armv6m-softfloat-eabi-newlib/ranlib READELF_FOR_BUILD=/usr/bin/x86_64-pc-linux-gnu-readelf READELF=/usr/bin/x86_64-pc-linux-gnu-readelf READELF_FOR_HOST=/usr/bin/x86_64-pc-linux-gnu-readelf READELF_FOR_TARGET=/usr/libexec/gcc/armv6m-softfloat-eabi-newlib/readelf STRIP_FOR_BUILD=/usr/bin/x86_64-pc-linux-gnu-strip STRIP=/usr/bin/x86_64-pc-linux-gnu-strip STRIP_FOR_HOST=/usr/bin/x86_64-pc-linux-gnu-strip STRIP_FOR_TARGET=/usr/libexec/gcc/armv6m-softfloat-eabi-newlib/strip CC_FOR_TARGET=/usr/libexec/gcc/armv6m-softfloat-eabi-newlib/cc CXX_FOR_TARGET=/usr/libexec/gcc/armv6m-softfloat-eabi-newlib/cxx WINDRES_FOR_TARGET=/usr/libexec/gcc/armv6m-softfloat-eabi-newlib/windres WINDMC_FOR_TARGET=/usr/libexec/gcc/armv6m-softfloat-eabi-newlib/windmc --target=armv6m-softfloat-eabi --build=x86_64-pc-linux-gnu --prefix=/usr --bindir=/usr/x86_64-pc-linux-gnu/armv6m-softfloat-eabi-newlib/gcc-bin/7.3.0 --includedir=/usr/lib/gcc/armv6m-softfloat-eabi-newlib/7.3.0/include --datadir=/usr/share/gcc-data/armv6m-softfloat-eabi-newlib/7.3.0 --mandir=/usr/share/gcc-data/armv6m-softfloat-eabi-newlib/7.3.0/man --infodir=/usr/share/gcc-data/armv6m-softfloat-eabi-newlib/7.3.0/info --with-gxx-include-dir=/usr/lib/gcc/armv6m-softfloat-eabi-newlib/7.3.0/include/g++-v7 --with-python-dir=/share/gcc-data/armv6m-softfloat-eabi-newlib/7.3.0/python --enable-languages=c --enable-obsolete --enable-secureplt --disable-werror --with-system-zlib --enable-nls --without-included-gettext --enable-checking=release --with-bugurl=https://bugs.gentoo.org/ --with-pkgversion=Gentoo 7.3.0-r3 p1.4 --disable-esp --enable-poison-system-directories --disable-bootstrap --with-newlib --enable-multilib --disable-altivec --disable-fixed-point --with-float=soft --disable-libgcj --disable-libgomp --disable-libmudflap --disable-libssp --disable-libcilkrts --disable-libmpx --disable-vtable-verify --disable-libvtv --disable-libquadmath --enable-lto --without-isl --disable-libsanitizer --enable-default-pie --enable-default-ssp

I'm wanting gcc to both make and use tools that start with the prefix: armv6m-softfloat-eabi-newlib- (Arm cortex m0 chipset is what I am using)

But "make" still fails when attempting to execute "ar" in the .../libcpp directory. The reason is that .../libcpp/Makefile.in is not updated by automake. It's a hand crafted file. On line 28 of the old .../libcpp/Makefile.in it says "AR = ar"

So, the AR variable is hardcoded to "ar" But, "ar" doesn't exist on my system. I've tried editing .../libcpp/Makefile.in with "AR = dummyname" , and the build crashes with "can't fine dummyname" instead of can't find "ar". So, the bug is on line 28.

All other variables in the .../libcpp/Makefile.in are of the form:

CC = @CC@
INSTALL = @INSTALL@

etc.. On a positive note: The compiler used by .../libcpp IS the fully qulaified name I gave to gcc-7.3.0/configure. That success made me think I could fix the bug by editing the makefile to read:

AR = @AR@

But the build fails with "Can't find AR@"

I'm not familiar enough with autotools to hand edit the Makefile.in and fix the bug. What's the @variable@ name format do?

Does the configure.ac in the subdirectory have to define "AR" in some way for @AR@ to be linked to the value in the toplevel directory?

I've tried a few other tests while building different gcc versions. Re-running autoconfig, automake, is hell because GCC uses AC_PREREQ() macro.

For example, I have autotools 2.69 installed ... but gcc 7.3.0 fails and complains that I must use autotools 2.64, ONLY. eg: AC_PREREQ(2.64)

So, fixing the bug via autotools doesn't seem practical. I'm hoping to simply patch the .../libcpp/Makefile.in, since that file is exactly the same in so many versions of gcc.

Questions: Why is "ar" hard-coded ? Is this a serious legacy issue? and what is a minimal patch that won't interfere with other configurations of GCC?

Is it better to modify the shell or the Makefile; eg: like the top level configure shell script could define a bash function that would be inherited by make as "if" it were a program.

if [ -z ${AR##*-*} ] ; then
    ar() { $AR } 
fi

Edit: A quick-fix patch for gcc-7.3.0 This is not a "correct" fix, but just a work-around.

I've found three places where the sub-directories ignore variables passed in from the toplevel configure. .../libcpp/Makefile.in on line 29 .../gcc/configure just before line 29531 .../libcc1/configure just before 14574

The second and third errors are from a defective macro in configure.ac. I haven't traced it back because I can't run autoconfig anyway.

I added a line to the configure(s), to see if passing the default OBJDUMP override variable would allow gcc to compile. It does. I'm not sure I've chosen the right override variable for all cases of gcc compile switches, but at least it proves where the bug is.

Patch file follows:

--- gcc-old/libcpp/Makefile.in                                                  
+++ gcc-new/libcpp/Makefile.in                                                  
@@ -28,3 +28,3 @@                                                               
 INSTALL = @INSTALL@                                                            
-AR = ar                                                                        
+AR ?= ar                                                                       
 ARFLAGS = cru                                                                  
--- gcc-old/gcc/configure                                                       
+++ gcc-new/gcc/configure                                                       
@@ -29531,4 +29531,6 @@                                                         
      ;;                                                                        
    esac                                                                        

+   if [ -n $OBJDUMP ]; then export_sym_check="$OBJDUMP -T"; fi                 
+                                                                               
    if test x"$enable_plugin" = x"yes"; then                                    
--- gcc-old/libcc1/configure                                                    
+++ gcc-new/libcc1/configure                                                    
@@ -14574,4 +14574,6 @@                                                         
      ;;                                                                        
    esac                                                                        

+   if [ -n $OBJDUMP ]; then export_sym_check="$OBJDUMP -T"; fi                 
+                                                                               
    if test x"$enable_plugin" = x"yes"; then

Have you considered filing a bug report against `GCC`? That might not get you a solution *today*, but it will be helpful to others. And if there is a known or intended workaround, then you just might get that today, after all. — John Bollinger, Jun 14 '18 at 13:17
I have considered it, but I'm not sure how to do it yet. I don't own an account or anything at the repository, so I expect there are several steps in order to be able to report a bug. I figure the best approach is to become educated, first, and make sure I've totally identified the issue correctly. Then if it's not resolvable, file a report. — Andrew of Scappoose, Jun 14 '18 at 14:56
The GCC manual contains information about this. Start here: https://gcc.gnu.org/onlinedocs/gcc-7.3.0/gcc/Service.html#Service — John Bollinger, Jun 14 '18 at 15:01
See https://gcc.gnu.org/bugs/ -- read, create account, file bug. — Jonathan Wakely, Jun 14 '18 at 18:54

John Bollinger · Answer 1 · 2018-06-14T14:59:01.923

TL;DR: there are a lot of things you could try, but the very first would be to specify AR on the command line when you run make:

make AR=x86_foo_b_ar

That shouldn't be necessary when you've already specified the same to configure, but if it doesn't work then that suggests a problem one or more levels up from the Makefile.in you're looking at. Variable definitions specified on the make command line override definitions in makefiles.

"make" still fails when attempting to execute "ar" from the .../libcpp directory. The reason is that .../libcpp/Makefile.in is not updated by automake. It's a hand crafted file.

To be clear, since understanding the system you are trying to use is immensely helpful in troubleshooting it, automake does not run at configuration or build time. It is used by the package maintainer to build one or more Makefile.in files to be included in source distributions, such as the one you obtained. Of course, this is not the only way to create Makefile.in files, and the configure script does not care how you create them (or other input files).

I'm not familiar enough with autotools to hand edit the Makefile.in and fix the bug. What's the @variable@ name format do?

Does the configure.ac in the subdirectory have to define "AR" in some way for @AR@ to be linked to the value in the toplevel directory?

The @variable@ construction is used for values that are expected to be substituted by the configure script when it builds a corresponding output file. For that to take place, there needs to be at least a corresponding AC_SUBST([variable]) or its equivalent in the configure.ac (sometimes named configure.in, instead). Normally, that's preceded somewhere in configure.ac by code assigning an appropriate value to shell variable variable.

If you modify configure.ac then you need to rebuild the configure script, and in that case it's probably safest to rebuild the whole build system, as a package maintainer would do. There may be a script provided for that purpose in the package (autogen.sh is a common name for such scripts), but the default mechanism is to run the Autotools program autoreconf in the top-level directory of the project source tree.

I've tried a few other tests while building different gcc versions. Re-running autoconfig, automake, is hell because GCC uses AC_PREREQ() macro.

For example, I have autotools 2.69 installed ... but gcc 7.3.0 fails and complains that I must use autotools 2.64, ONLY. eg: AC_PREREQ(2.64)

That description is not consistent with the documentation of AC_PREREQ, nor with my experience with that macro. AC_PREREQ tests for the specified Autoconf version or newer. It does not demand an exact Autoconf version. There may be something else in the build system that does so, but it's not AC_PREREQ.

In any case, one alternative would be to obtain and install Autoconf 2.64. You may even be able to install it alongside your existing version. Some systems even provide pre-built packages for exactly that purpose.

So, fixing the bug via autotools doesn't seem practical. I'm hoping to simply patch the .../libcpp/Makefile.in, since that file is exactly the same in so many versions of gcc.

Patching a Makefile.in does not require afterward re-running the autotools, so it's at least conceivable that that would work. Even for Makefile.in files that were generated by Automake. You could consider having a look at how AR is defined in some of the Automake-generated Makefile.in files in the project (supposing there are any) for an idea of how it should look.

Why is "ar" hard-coded ? Is this a serious legacy issue?

I can only speculate. As a threshold matter, I'm inclined to suppose that in that Makefile, the archiver of the build system is the one wanted (not that of the intended host system, nor a cross-ar for host-target). It is reasonable in that case for AR = ar to be provided as a default, because that can be overridden via a declaration of that variable on the command-line.

That you are in fact not getting the AR you specify to configure looks like a bug to me -- probably a regression introduced at some point when some of the higher-level bits of the build system were updated. I have no trouble imagining such an issue slipping by, as a system configuration such as yours, in which the system's own archiver goes only by a non-standard name, is very uncommon.

and what is a minimal patch that won't interfere with other configurations of GCC?

The first thing to try is to pass the AR definition on the top-level make command line:

make AR=x86_foo_b_ar

Such definitions will be passed on to recursively-invoked sub-makes, and definitions on the command line (but not, by default, from the environment) override definitions in Makefiles.

Is it better to modify the shell or the Makefile; eg: like the top level configure shell script could define a bash function that would be inherited by make as "if" it were a program.

The top-level configure script could be modified to define a shell function and export it to child processes, but not to its parent or siblings. This is nothing specific to configure; the shell just doesn't work that way. Whatever changes you make, if any, would be best made in Makefile.in files before running configure, or in the generated Makefiles afterward.

I'm running this from a modified version of Gentoo; It automatically runs make after running configure... But, if I wait for failure, and then manually re-run make from the TLD with make AR=... the exact same failure occurs. Obviously, the top level make is not invoking sub level makes with the AR= override. BTW: I can edit .../libcpp/Makefile.in and set "AR ?= ar", and the build will correctly build libcpp directory. The make will continue building gcc until it enters another directory, and tries to run RANLIB with a wrong name. I'm tracing out where RANLIB is defined... — Andrew of Scappoose, Jun 14 '18 at 15:01
I'm sorry; I have no more advice to offer on this topic, other than to reiterate my suggestion to file a bug report, or, alternatively, to contact the GCC team directly. I commented on your question with a link to the GCC manual's description of how to do that. — John Bollinger, Jun 14 '18 at 15:09
I'll take the suggestion; I'm just working through your explanation/speculation. After looking carefully at the top level Makefile, I'm sure that make defines multiple sets of "Export" lists. One list is BUILD_EXPORTS, another is HOST_EXPORTS, etc. Each of them defines AR, CC, ... differently. BUILD_EXPORTS defines AR="$(AR_FOR_BUILD)"; export AR; ... but HOST_EXPORTS defines AR="$(AR)"; export AR; ... So, a single over-ride for AR would break a gcc build, because some subdirectories are building host binaries, others are building build machine binaries. — Andrew of Scappoose, Jun 14 '18 at 19:34

Autotools: GCC's Makefile.in has a bug (7.3.0 and earlier). How to re-program automake to find 'ar' and 'objdump'

1 Answers1