12

I am trying to build boost library as dylib on MacOS. I need to build it for both the Intel architecture and the upcoming Apple Silicon (arm64) architecture.

I downloaded boost and ran the following commands:

./bootstrap.sh
./b2 -address-model=64 architecture=combined -a

lipo -archs always shows produced dylibs architecture is x86_64.

I have Xcode12 beta and MacOS Catalina 10.15.7,

I can build a sample universal library if I create a project in Xcode and set archs arm64 x86_64 in build settings.

Running command ./b2 cxxflags="-arch arm64 -arch x86_64" fails with following errors:

    "clang++" -x c++ -fvisibility-inlines-hidden -m64 -O3 -Wall -fvisibility=hidden -Wno-inline -arch arm64 -arch x86_64 -ftemplate-depth-255 -fvisibility=hidden -fvisibility-inlines-hidden -DBOOST_ALL_NO_LIB=1 -DNDEBUG -I"." -c -o "bin.v2/libs/serialization/build/clang-darwin-12.0/release/link-static/threading-multi/visibility-hidden/polymorphic_xml_iarchive.o" "libs/serialization/src/polymorphic_xml_iarchive.cpp"

...failed clang-darwin.compile.c++ bin.v2/libs/serialization/build/clang-darwin-12.0/release/link-static/threading-multi/visibility-hidden/polymorphic_xml_iarchive.o...
clang-darwin.compile.c++ bin.v2/libs/serialization/build/clang-darwin-12.0/release/link-static/threading-multi/visibility-hidden/polymorphic_xml_oarchive.o
In file included from libs/serialization/src/polymorphic_xml_oarchive.cpp:16:
In file included from ./boost/serialization/config.hpp:18:
In file included from ./boost/config.hpp:57:
In file included from ./boost/config/platform/macos.hpp:28:
In file included from ./boost/config/detail/posix_features.hpp:18:
In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/unistd.h:71:
In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/_types.h:27:
In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/sys/_types.h:32:
/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/sys/cdefs.h:807:2: error: Unsupported architecture
#error Unsupported architecture
 ^
In file included from libs/serialization/src/polymorphic_xml_oarchive.cpp:16:
In file included from ./boost/serialization/config.hpp:18:
In file included from ./boost/config.hpp:57:
In file included from ./boost/config/platform/macos.hpp:28:
In file included from ./boost/config/detail/posix_features.hpp:18:
In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/unistd.h:71:
In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/_types.h:27:
In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/sys/_types.h:33:
/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/machine/_types.h:34:2: error: architecture not supported
#error architecture not supported
 ^
In file included from libs/serialization/src/polymorphic_xml_oarchive.cpp:16:
In file included from ./boost/serialization/config.hpp:18:
In file included from ./boost/config.hpp:57:
In file included from ./boost/config/platform/macos.hpp:28:
In file included from ./boost/config/detail/posix_features.hpp:18:
In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/unistd.h:71:
In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/_types.h:27:
/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/sys/_types.h:55:9: error: unknown type name '__int64_t'
typedef __int64_t       __darwin_blkcnt_t;      /* total blocks */
        ^
/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/sys/_types.h:56:9: error: unknown type name '__int32_t'; did you mean '__int128_t'?
typedef __int32_t       __darwin_blksize_t;     /* preferred block size */
        ^
note: '__int128_t' declared here
/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/sys/_types.h:57:9: error: unknown type name '__int32_t'; did you mean '__int128_t'?
typedef __int32_t       __darwin_dev_t;         /* dev_t */
        ^
note: '__int128_t' declared here
/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/sys/_types.h:60:9: error: unknown type name '__uint32_t'; did you mean '__uint128_t'?
typedef __uint32_t      __darwin_gid_t;         /* [???] process and group IDs */
        ^
note: '__uint128_t' declared here
/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/sys/_types.h:61:9: error: unknown type name '__uint32_t'; did you mean '__uint128_t'?
typedef __uint32_t      __darwin_id_t;          /* [XSI] pid_t, uid_t, or gid_t*/
        ^
Anil8753
  • 2,663
  • 4
  • 29
  • 40

7 Answers7

7

I'm on Monterey with an M1 (XCode 13.1) and failed to get any of the other answers to work, but I combined stuff from Navan and Petr along with some other bits to get the libboost_coroutine to work on x86_64. I arrived at the following script which builds all the boost libraries (why not, it doesn't take long on the M1):

#!/bin/sh

rm -rf arm64 x86_64 universal stage bin.v2
rm -f b2 project-config*
./bootstrap.sh cxxflags="-arch x86_64 -arch arm64" cflags="-arch x86_64 -arch arm64" linkflags="-arch x86_64 -arch arm64"
./b2 toolset=clang-darwin target-os=darwin architecture=arm abi=aapcs cxxflags="-arch arm64" cflags="-arch arm64" linkflags="-arch arm64" -a
mkdir -p arm64 && cp stage/lib/*.dylib arm64
./b2 toolset=clang-darwin target-os=darwin architecture=x86 cxxflags="-arch x86_64" cflags="-arch x86_64" linkflags="-arch x86_64" abi=sysv binary-format=mach-o -a
mkdir x86_64 && cp stage/lib/*.dylib x86_64
mkdir universal
for dylib in arm64/*; do 
  lipo -create -arch arm64 $dylib -arch x86_64 x86_64/$(basename $dylib) -output universal/$(basename $dylib); 
done
for dylib in universal/*; do
  lipo $dylib -info;
done

This script prints out the lipo info for each dylib in the universal directory, so you can see quickly that every one has both x86_64 and arm64 inside. I had the following result, maybe it's useful for you to see if your output matches:

Architectures in the fat file: universal/libboost_atomic.dylib are: x86_64 arm64 
Architectures in the fat file: universal/libboost_chrono.dylib are: x86_64 arm64 
Architectures in the fat file: universal/libboost_container.dylib are: x86_64 arm64 
Architectures in the fat file: universal/libboost_context.dylib are: x86_64 arm64 
Architectures in the fat file: universal/libboost_contract.dylib are: x86_64 arm64 
Architectures in the fat file: universal/libboost_coroutine.dylib are: x86_64 arm64 
Architectures in the fat file: universal/libboost_date_time.dylib are: x86_64 arm64 
Architectures in the fat file: universal/libboost_filesystem.dylib are: x86_64 arm64 
Architectures in the fat file: universal/libboost_graph.dylib are: x86_64 arm64 
Architectures in the fat file: universal/libboost_iostreams.dylib are: x86_64 arm64 
Architectures in the fat file: universal/libboost_locale.dylib are: x86_64 arm64 
Architectures in the fat file: universal/libboost_log.dylib are: x86_64 arm64 
Architectures in the fat file: universal/libboost_log_setup.dylib are: x86_64 arm64 
Architectures in the fat file: universal/libboost_numpy27.dylib are: x86_64 arm64 
Architectures in the fat file: universal/libboost_prg_exec_monitor.dylib are: x86_64 arm64 
Architectures in the fat file: universal/libboost_program_options.dylib are: x86_64 arm64 
Architectures in the fat file: universal/libboost_python27.dylib are: x86_64 arm64 
Architectures in the fat file: universal/libboost_random.dylib are: x86_64 arm64 
Architectures in the fat file: universal/libboost_regex.dylib are: x86_64 arm64 
Architectures in the fat file: universal/libboost_serialization.dylib are: x86_64 arm64 
Architectures in the fat file: universal/libboost_stacktrace_addr2line.dylib are: x86_64 arm64 
Architectures in the fat file: universal/libboost_stacktrace_basic.dylib are: x86_64 arm64 
Architectures in the fat file: universal/libboost_stacktrace_noop.dylib are: x86_64 arm64 
Architectures in the fat file: universal/libboost_system.dylib are: x86_64 arm64 
Architectures in the fat file: universal/libboost_thread.dylib are: x86_64 arm64 
Architectures in the fat file: universal/libboost_timer.dylib are: x86_64 arm64 
Architectures in the fat file: universal/libboost_type_erasure.dylib are: x86_64 arm64 
Architectures in the fat file: universal/libboost_unit_test_framework.dylib are: x86_64 arm64 
Architectures in the fat file: universal/libboost_wave.dylib are: x86_64 arm64 
Architectures in the fat file: universal/libboost_wserialization.dylib are: x86_64 arm64
Jesse Pangburn
  • 655
  • 1
  • 7
  • 16
  • 1
    Thanks a lot ! It worked for me too ! I was facing a linker issue (the same as reported here : https://github.com/boostorg/coroutine/issues/60) about `_jump_fcontext`, `_make_fcontext`, referenced from boost::coroutines::detail::coroutine_context when trying to compile boost 1.81.0 on my M1 laptop. – Saitama Feb 07 '23 at 13:44
  • 1
    @kamui Thanks for reporting the different error that it fixed for you! Hopefully it'll help someone with that same error find this solution. – Jesse Pangburn Feb 08 '23 at 16:37
  • @JessePangburn this worked well for dynamic libs (.dylib) but is it possible to create static libraries (.a)? I tried adding `link=static` to the `.b2` commands, but that did not seem to do it. Any suggestions? – ariestav Jul 20 '23 at 18:33
  • @JessePangburn please disregard my last comment. The static libs were creates in `/stage` but the script was globbing `*.dylib` files so they were not being copied into the `/universal` directory. Many thanks for your script! – ariestav Jul 20 '23 at 18:41
  • @ariestav Glad to hear it worked for you! – Jesse Pangburn Jul 22 '23 at 02:13
5

The following works for me on Big Sur using Xcode 12.2. On Catalina 10.15.7 (19H15) I get the same error messages as OP.

./bootstrap.sh --with-libraries=regex,date_time 
./b2 architecture=combined cxxflags="-arch x86_64 -arch arm64"
bfx
  • 897
  • 10
  • 16
  • What version of Boost were you building here? – jamieguinan Jan 08 '21 at 18:25
  • 1
    @jamieguinan [Boost Version 1.74.0](https://www.boost.org/users/history/version_1_74_0.html). Version 1.75.0 came out only a few days after I wrote this answer and I haven't tried building that, yet. – bfx Jan 11 '21 at 10:36
  • 1
    Building 1.75.0 with Xcode 12.4 under Big Sur (on intel) I get the issue mentioned in the response above by Navan Chauhan with this approach - the static libs are universal but the dylibs are only x86_64. – Corbell Mar 01 '21 at 21:28
  • 1
    I tried this for 1.77.0 with Xcode 13.1 under Monterey on M1, but the dylibs were arm64 only. – Jesse Pangburn Nov 11 '21 at 05:42
  • Doesn't work. On Intel it compiles for Intel and on M1 it compiles for M1 but never both. – Lothar Jun 08 '22 at 01:37
3

After spending considerable amount of time here:

https://github.com/bfgroup/b2/issues/105

I was able to compile boost on arm64. The command line is:

./b2 architecture=arm address-model=64 asmflags=--target=arm64-apple-darwin21.2.0 cflags=--target=arm64-apple-darwin21.2.0 cxxflags=--target=arm64-apple-darwin21.2.0 linkflags=--target=arm64-apple-darwin21.2.0 -s NO_LZMA=1 -s NO_ZSTD=1 abi=aapcs

the LZMA and ZSTD flags are there since I did not have an universal binary for those libs on my machine.

Clusty
  • 117
  • 7
1

I was facing the same issue and came across this answer, which was for creating universal binaries for i386 and x86_64.

To summarise the answer, you need to run ./b2 twice ( with different toolsets as clang always builds for x86_64, even if you pass the appropriate CXXFlags

This is the script I used to first individually generate the libraries I required and then combined them using lipo

#!/bin/sh

rm -rf arm64 x86_64 universal
./bootstrap.sh --with-toolset=clang --with-libraries=thread,system,filesystem,program_options,serialization 
./b2  cxxflags="-arch arm64" toolset=darwin -a
mkdir -p arm64 && cp stage/lib/*.dylib arm64
./b2 toolset=clang cxxflags="-arch arm64 -arch x86_64" -a
mkdir x86_64 && cp stage/lib/*.dylib x86_64
mkdir universal
for dylib in arm64/*; do 
  lipo -create -arch arm64 $dylib -arch x86_64 x86_64/$(basename $dylib) -output universal/$(basename $dylib); 
done
Navan Chauhan
  • 397
  • 6
  • 14
  • Doesnt work on Monterey, despite "-arm64" and correct config output, this is still producing x86_64 targets – Lothar Jun 08 '22 at 01:27
1

I followed the answer by Navan Chauhan which seems to be the way to go on macOS 10.15.7 with Xcode 12.3.

My only issue was that the build system was adding additional -arch armv4t clang option (or something similar) which fails the build. Using the architecture=combine cannot work as well as it is able to manage Intel and PowerPC only. What does work for me instead is:

  1. bootstrap.sh --with-toolset=clang-darwin and cxxflags, cflags and linkflags set to -arch x86_64 -arch arm64 plus additional options.
  2. x86_64 build: b2 toolset=clang-darwin target-os=darwin architecture=x86 stage and cxxflags, cflags and linkflags set to -arch x86_64 only (plus additional options).
  3. arm64 build: b2 toolset=clang-darwin target-os=darwin architecture=arm abi=aapcs stage and cxxflags, cflags and linkflags set to -arch arm64 only (plus additional options).
  4. Merge libraries with lipo…

I've been forced to set the ABI also as the automatic-guess-something is not able to recognize Apple Silicon goal therefor some assembly stuff is not compiled – hence missing symbols later on during the build. Finally the armv4t issue is solved by the clang-darwin toolset.

The only caveat of this at the moment is that both builds go into the same architectural/configuration directories which can be solved by custom user-config.jam or by immediate call to install just after each stage which is what I did.

Works on:

  • Host platform: Intel
  • macOS 10.15.7
  • Xcode 12.3 (with Command Line Tools installed)
  • Boost 1.75.0 (dependent compression and ICU libraries already compiled as universal)
Petr Hadraba
  • 103
  • 1
  • 4
0

You don't need to use lipo, you can pass -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk to cxxflags and it will build a universal library out of the box.

lorcan
  • 19
  • 1
  • 2
  • `./bootstrap.sh --with-libraries=thread,system,filesystem,program_options,serialization && /b2 toolset=darwin -a cxxflags="-arch arm64 -arch x86_64 -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk"` does not build a universal library. Checking using `lipo` shows the architecture to be `x86_64` only – Navan Chauhan Nov 14 '20 at 07:14
  • Here is the full command I used: `# build_boost.sh ./bootsrap.sh with="--with-exception --with-filesystem --with-test" cxxflags="-fvisibility=hidden -fvisibility-inlines-hidden -std=c++17 -stdlib=libc++ -arch x86_64 -arch arm64 -mmacosx-version-min=10.11 -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk" cflags="-mmacosx-version-min=10.11" linkflags="-std=c++17 -stdlib=libc++ -arch x86_64 -arch arm64 -mmacosx-version-min=10.11"` – lorcan Nov 14 '20 at 15:56
  • Then `./b2 -q -a -j8 architecture=combined toolset=darwin link=static runtime-link=static threading=multi \ cflags="$cflags" cxxflags="$cxxflags" mflags="$cflags" mmflags="$cflags" linkflags="$linkflags" \ --stagedir=stage variant=release $with` Verify with `lipo -archs libboost_filesystem.a x86_64 arm64` Make sure you used the latest XCode 12.2 because 12.1 didn't have the arm bits. – lorcan Nov 14 '20 at 15:57
  • 2
    I am not able to reproduce this with your commands, what boost version are you building for? I am trying on `1.74.0` – Navan Chauhan Nov 14 '20 at 17:27
  • Building boost 1.74.0, macOS Catalina, XCode 12.2 + command line tools The key is using `architecture=combined` along with `-isysroot`. What errors are you getting ? – lorcan Nov 14 '20 at 22:09
  • 1
    I am building all the libs on Apple Silicon; thread will not build: error: at libs/thread/build/Jamfile.v2:273 error: 64-bit PPC compilation is not supported when targeting OSX 10.6 or later I guess when architecture=combined, it somehow falls back to PPC64. When I only build the libs I really need, it seems to work, but when linking my program, it complains about missing _jump_fcontext/_make_fcontext/_ontop_fcontext – Dix Nov 28 '20 at 11:27
  • Providing `linkflags` did the trick for me. – milembar Nov 11 '21 at 01:19
-1

Checked for macOS M1 building boost 1.82.0, successfully build fat binaries with both architecture included:

./bootstrap.sh
./b2 -a address-model=64 architecture=arm+x86

To install systemwide:

sudo ./b2 -a address-model=64 architecture=arm+x86 install

Checked that it linked successfully. Feel free to make sure with:

lipo -info /usr/local/lib/libboost_chrono.dylib
Architectures in the fat file: /usr/local/lib/libboost_chrono.dylib are: x86_64 arm64

Hope it helps.

Sergei Krivonos
  • 4,217
  • 3
  • 39
  • 54