I’m trying to bootstrap a complete llvm/clang/libc++/lld toolchain on a standard Debian glibc/x86-64 system. (Ultimately, I want to do the same on a more unusual musl-based Linux distribution, but need to make the bootstrap work on a standard supported system before I attempt anything harder!)
Using a Debian/unstable chroot with up-to-date GNU toolchain, python3, and cmake, in a ‘build’ subdir of a checkout of release/12.x of llvm-project.git, I run:
cmake -G "Unix Makefiles" \
-DLLVM_ENABLE_PROJECTS="compiler-rt;libunwind;libcxx;libcxxabi;lld;clang" \
-DLLVM_TARGETS_TO_BUILD=Native \
-DCLANG_ENABLE_BOOTSTRAP=ON \
-DCMAKE_BUILD_TYPE=Release \
-DCLANG_DEFAULT_CXX_STDLIB=libc++ \
-DCLANG_DEFAULT_RTLIB=compiler-rt \
-DLIBCXX_USE_COMPILER_RT=YES \
-DLIBCXXABI_USE_COMPILER_RT=YES \
-DLIBCXXABI_USE_LLVM_UNWINDER=YES \
-DBOOTSTRAP_CMAKE_BUILD_TYPE=Release \
-DBOOTSTRAP_CLANG_DEFAULT_CXX_STDLIB=libc++ \
-DBOOTSTRAP_CLANG_DEFAULT_RTLIB=compiler-rt \
-DBOOTSTRAP_LIBCXX_USE_COMPILER_RT=YES \
-DBOOTSTRAP_LIBCXXABI_USE_COMPILER_RT=YES \
-DBOOTSTRAP_LIBCXXABI_USE_LLVM_UNWINDER=YES \
-DBOOTSTRAP_LLVM_USE_LINKER=lld \
../llvm
make -j16
This gives me libunwind, libstdc++, etc. correctly-built and independent of libgcc_s and libatomic:
# ldd lib/libc++.so.1
linux-vdso.so.1 (0x00007ffc31e5f000)
libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007fde7be18000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fde7bc53000)
libc++abi.so.1 => /root/llvm-project/build/lib/libc++abi.so.1 (0x00007fde7bc19000)
libunwind.so.1 => /root/llvm-project/build/lib/libunwind.so.1 (0x00007fde7bc02000)
/lib64/ld-linux-x86-64.so.2 (0x00007fde7bf1f000)
libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007fde7bbfc000)
# ldd lib/libunwind.so.1
linux-vdso.so.1 (0x00007ffd91d7f000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f47c124f000)
libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f47c1249000)
libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f47c1227000)
/lib64/ld-linux-x86-64.so.2 (0x00007f47c1431000)
However, the compiler and other binaries have been linked against GNU libstdc++ and libgcc_s:
# ldd bin/clang
linux-vdso.so.1 (0x00007ffc5e5d8000)
libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f5ff97a5000)
libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f5ff979f000)
libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007f5ff95d2000)
libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f5ff948e000)
libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f5ff9474000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f5ff92af000)
/lib64/ld-linux-x86-64.so.2 (0x00007f5fff181000)
To test the new compiler, I used the same bootstrap process again after a ‘make install’ to /usr/local: I re-ran the same cmake command in a clean ‘build2’ subdir with extra flags -DCMAKE_C_COMPILER=clang and -DCMAKE_CXX_COMPILER=clang++.
This time, the binaries were linked against libc++ correctly… but the libraries are now incorrectly linked to libgcc_s and libatomic - even though they weren’t at the end of the previous build!
# ldd bin/clang
linux-vdso.so.1 (0x00007ffee47f8000)
libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007fd3740fa000)
librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007fd3740ef000)
libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007fd3740e9000)
libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007fd373fa5000)
libc++.so.1 => /root/llvm-project/build2/bin/../lib/libc++.so.1 (0x00007fd373ed1000)
libunwind.so.1 => /root/llvm-project/build2/bin/../lib/libunwind.so.1 (0x00007fd373ec1000)
libc++abi.so.1 => /root/llvm-project/build2/bin/../lib/libc++abi.so.1 (0x00007fd373e7e000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fd373cb9000)
/lib64/ld-linux-x86-64.so.2 (0x00007fd374122000)
libatomic.so.1 => /usr/lib/x86_64-linux-gnu/libatomic.so.1 (0x00007fd373caf000)
libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007fd373c95000)
# ldd lib/libc++.so.1
linux-vdso.so.1 (0x00007fff264bb000)
libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007fef1179c000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fef115d7000)
libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007fef11493000)
librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007fef11488000)
libatomic.so.1 => /usr/lib/x86_64-linux-gnu/libatomic.so.1 (0x00007fef1147e000)
libc++abi.so.1 => /root/llvm-project/build2/lib/libc++abi.so.1 (0x00007fef1143d000)
libunwind.so.1 => /root/llvm-project/build2/lib/libunwind.so.1 (0x00007fef1142b000)
/lib64/ld-linux-x86-64.so.2 (0x00007fef11898000)
libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007fef11411000)
libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007fef1140b000)
# ldd lib/libunwind.so.1
linux-vdso.so.1 (0x00007ffc0adf5000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f3fd53c4000)
libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f3fd53aa000)
libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f3fd53a4000)
libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f3fd5382000)
/lib64/ld-linux-x86-64.so.2 (0x00007f3fd559f000)
Adding an additional -DCMAKE_CXX_FLAGS=-stdlib=libc++ -DCMAKE_EXE_LINKER_FLAGS=-stdlib=libc++ doesn’t change this either.
I expect I’m making an obvious beginner mistake, but I’m stumped! What am I doing wrong here? I aim to end up with a standalone toolchain, using libc++, llvm libunwind, etc., with compiler-rt providing builtins instead of libgcc_s/libatomic. (My sanity check is then that it can build itself correctly.)
PS The behaviour is identical with both llvm-project 11.0.1 and release/12.x. I’m using Debian unstable because buster/stable has binutils 2.31.1 with a bug that breaks the build: bug 42994.