compilation of rustc 1.16.0 on x86_64 seems not to pick up the LLVM NVPTX library but still uses it - linking failure

[daym]

When I try to build rustc 1.16.0 on x86_64, I get

yntax_ext-63283ac5" "-L" "/tmp/guix-build-rustc-1.16.0.drv-0/rustc-1.16.0-src/x86_64-unknown-linux-gnu/stage0/lib/rustlib/x86_64-unknown-linux-gnu/lib" "-l" "proc_macro-63283ac5" "-L" "/tmp/guix-build-rustc-1.16.0.drv-0/rustc-1.16.0-src/x86_64-unknown-linux-gnu/stage0/lib/rustlib/x86_64-unknown-linux-gnu/lib" "-l" "rustc_typeck-63283ac5" "-L" "/tmp/guix-build-rustc-1.16.0.drv-0/rustc-1.16.0-src/x86_64-unknown-linux-gnu/stage0/lib/rustlib/x86_64-unknown-linux-gnu/lib" "-l" "rustc_const_eval-63283ac5" "-L" "/tmp/guix-build-rustc-1.16.0.drv-0/rustc-1.16.0-src/x86_64-unknown-linux-gnu/stage0/lib/rustlib/x86_64-unknown-linux-gnu/lib" "-l" "rustc_platform_intrinsics-63283ac5" "-L" "/tmp/guix-build-rustc-1.16.0.drv-0/rustc-1.16.0-src/x86_64-unknown-linux-gnu/stage0/lib/rustlib/x86_64-unknown-linux-gnu/lib" "-l" "rustc-63283ac5" "-L" "/tmp/guix-build-rustc-1.16.0.drv-0/rustc-1.16.0-src/x86_64-unknown-linux-gnu/stage0/lib/rustlib/x86_64-unknown-linux-gnu/lib" "-l" "rustc_const_math-63283ac5" "-L" "/tmp/guix-build-rustc-1.16.0.drv-0/rustc-1.16.0-src/x86_64-unknown-linux-gnu/stage0/lib/rustlib/x86_64-unknown-linux-gnu/lib" "-l" "rustc_back-63283ac5" "-L" "/tmp/guix-build-rustc-1.16.0.drv-0/rustc-1.16.0-src/x86_64-unknown-linux-gnu/stage0/lib/rustlib/x86_64-unknown-linux-gnu/lib" "-l" "graphviz-63283ac5" "-L" "/tmp/guix-build-rustc-1.16.0.drv-0/rustc-1.16.0-src/x86_64-unknown-linux-gnu/stage0/lib/rustlib/x86_64-unknown-linux-gnu/lib" "-l" "getopts-63283ac5" "-L" "/tmp/guix-build-rustc-1.16.0.drv-0/rustc-1.16.0-src/x86_64-unknown-linux-gnu/stage0/lib/rustlib/x86_64-unknown-linux-gnu/lib" "-l" "fmt_macros-63283ac5" "-L" "/tmp/guix-build-rustc-1.16.0.drv-0/rustc-1.16.0-src/x86_64-unknown-linux-gnu/stage0/lib/rustlib/x86_64-unknown-linux-gnu/lib" "-l" "arena-63283ac5" "-L" "/tmp/guix-build-rustc-1.16.0.drv-0/rustc-1.16.0-src/x86_64-unknown-linux-gnu/stage0/lib/rustlib/x86_64-unknown-linux-gnu/lib" "-l" "syntax-63283ac5" "-L" "/tmp/guix-build-rustc-1.16.0.drv-0/rustc-1.16.0-src/x86_64-unknown-linux-gnu/stage0/lib/rustlib/x86_64-unknown-linux-gnu/lib" "-l" "rustc_errors-63283ac5" "-L" "/tmp/guix-build-rustc-1.16.0.drv-0/rustc-1.16.0-src/x86_64-unknown-linux-gnu/stage0/lib/rustlib/x86_64-unknown-linux-gnu/lib" "-l" "syntax_pos-63283ac5" "-L" "/tmp/guix-build-rustc-1.16.0.drv-0/rustc-1.16.0-src/x86_64-unknown-linux-gnu/stage0/lib/rustlib/x86_64-unknown-linux-gnu/lib" "-l" "term-63283ac5" "-L" "/tmp/guix-build-rustc-1.16.0.drv-0/rustc-1.16.0-src/x86_64-unknown-linux-gnu/stage0/lib/rustlib/x86_64-unknown-linux-gnu/lib" "-l" "flate-63283ac5" "-L" "/tmp/guix-build-rustc-1.16.0.drv-0/rustc-1.16.0-src/x86_64-unknown-linux-gnu/stage0/lib/rustlib/x86_64-unknown-linux-gnu/lib" "-l" "rustc_data_structures-63283ac5" "-L" "/tmp/guix-build-rustc-1.16.0.drv-0/rustc-1.16.0-src/x86_64-unknown-linux-gnu/stage0/lib/rustlib/x86_64-unknown-linux-gnu/lib" "-l" "serialize-63283ac5" "-L" "/tmp/guix-build-rustc-1.16.0.drv-0/rustc-1.16.0-src/x86_64-unknown-linux-gnu/stage0/lib/rustlib/x86_64-unknown-linux-gnu/lib" "-l" "log-63283ac5" "-L" "/tmp/guix-build-rustc-1.16.0.drv-0/rustc-1.16.0-src/x86_64-unknown-linux-gnu/stage0/lib/rustlib/x86_64-unknown-linux-gnu/lib" "-l" "rustc_llvm-63283ac5" "-L" "/tmp/guix-build-rustc-1.16.0.drv-0/rustc-1.16.0-src/x86_64-unknown-linux-gnu/stage0/lib/rustlib/x86_64-unknown-linux-gnu/lib" "-l" "std-63283ac5" "/tmp/guix-build-rustc-1.16.0.drv-0/rustc-1.16.0-src/x86_64-unknown-linux-gnu/stage0/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcompiler_builtins-63283ac5.rlib" "-l" "LLVMInterpreter" "-l" "LLVMMCJIT" "-l" "LLVMExecutionEngine" "-l" "LLVMRuntimeDyld" "-l" "LLVMipo" "-l" "LLVMVectorize" "-l" "LLVMLinker" "-l" "LLVMIRReader" "-l" "LLVMAsmParser" "-l" "LLVMSparcDisassembler" "-l" "LLVMSparcCodeGen" "-l" "LLVMSparcAsmParser" "-l" "LLVMSparcDesc" "-l" "LLVMSparcInfo" "-l" "LLVMSparcAsmPrinter" "-l" "LLVMMSP430CodeGen" "-l" "LLVMMSP430Desc" "-l" "LLVMMSP430Info" "-l" "LLVMMSP430AsmPrinter" "-l" "LLVMSystemZDisassembler" "-l" "LLVMSystemZCodeGen" "-l" "LLVMSystemZAsmParser" "-l" "LLVMSystemZDesc" "-l" "LLVMSystemZInfo" "-l" "LLVMSystemZAsmPrinter" "-l" "LLVMPowerPCDisassembler" "-l" "LLVMPowerPCCodeGen" "-l" "LLVMPowerPCAsmParser" "-l" "LLVMPowerPCDesc" "-l" "LLVMPowerPCInfo" "-l" "LLVMPowerPCAsmPrinter" "-l" "LLVMMipsDisassembler" "-l" "LLVMMipsCodeGen" "-l" "LLVMMipsAsmParser" "-l" "LLVMMipsDesc" "-l" "LLVMMipsInfo" "-l" "LLVMMipsAsmPrinter" "-l" "LLVMAArch64Disassembler" "-l" "LLVMAArch64CodeGen" "-l" "LLVMAArch64AsmParser" "-l" "LLVMAArch64Desc" "-l" "LLVMAArch64Info" "-l" "LLVMAArch64AsmPrinter" "-l" "LLVMAArch64Utils" "-l" "LLVMARMDisassembler" "-l" "LLVMARMCodeGen" "-l" "LLVMARMAsmParser" "-l" "LLVMARMDesc" "-l" "LLVMARMInfo" "-l" "LLVMARMAsmPrinter" "-l" "LLVMX86Disassembler" "-l" "LLVMX86AsmParser" "-l" "LLVMX86CodeGen" "-l" "LLVMSelectionDAG" "-l" "LLVMAsmPrinter" "-l" "LLVMCodeGen" "-l" "LLVMTarget" "-l" "LLVMScalarOpts" "-l" "LLVMInstCombine" "-l" "LLVMInstrumentation" "-l" "LLVMProfileData" "-l" "LLVMTransformUtils" "-l" "LLVMBitWriter" "-l" "LLVMAnalysis" "-l" "LLVMX86Desc" "-l" "LLVMObject" "-l" "LLVMMCParser" "-l" "LLVMBitReader" "-l" "LLVMMCDisassembler" "-l" "LLVMX86Info" "-l" "LLVMX86AsmPrinter" "-l" "LLVMMC" "-l" "LLVMX86Utils" "-l" "LLVMCore" "-l" "LLVMSupport" "-l" "rt" "-l" "dl" "-l" "pthread" "-l" "z" "-l" "m" "-l" "stdc++" "-l" "dl" "-l" "pthread" "-l" "gcc_s" "-l" "c" "-l" "m" "-l" "rt" "-l" "util" "-Wl,-rpath,$ORIGIN/../lib" "-Wl,-rpath,/usr/local/lib/rustlib/x86_64-unknown-linux-gnu/lib" "-Wl,--enable-new-dtags"
  = note: /tmp/guix-build-rustc-1.16.0.drv-0/rustc-1.16.0-src/x86_64-unknown-linux-gnu/stage0/lib/rustlib/x86_64-unknown-linux-gnu/lib/librustc_llvm-63283ac5.so: undefined reference to `LLVMInitializeNVPTXTargetMC'
/tmp/guix-build-rustc-1.16.0.drv-0/rustc-1.16.0-src/x86_64-unknown-linux-gnu/stage0/lib/rustlib/x86_64-unknown-linux-gnu/lib/librustc_llvm-63283ac5.so: undefined reference to `LLVMInitializeNVPTXAsmPrinter'
/tmp/guix-build-rustc-1.16.0.drv-0/rustc-1.16.0-src/x86_64-unknown-linux-gnu/stage0/lib/rustlib/x86_64-unknown-linux-gnu/lib/librustc_llvm-63283ac5.so: undefined reference to `LLVMInitializeNVPTXTargetInfo'
/tmp/guix-build-rustc-1.16.0.drv-0/rustc-1.16.0-src/x86_64-unknown-linux-gnu/stage0/lib/rustlib/x86_64-unknown-linux-gnu/lib/librustc_llvm-63283ac5.so: undefined reference to `LLVMInitializeNVPTXTarget'
collect2: error: ld returned 1 exit status

When doing the same on rustc 1.15.0 it builds and works fine.

The "configure" flags are:

                    (flags (list
                            (string-append "--prefix=" out)
                            (string-append "--datadir=" out "/share")
                            (string-append "--infodir=" out "/share/info")
                            (string-append "--default-linker=" gcc "/bin/gcc")
                            (string-append "--default-ar=" binutils "/bin/ar")
                            (string-append "--python=" python "/bin/python2")
                            (string-append "--local-rust-root=" rustc)
                            (string-append "--llvm-root=" llvm)
                            (string-append "--jemalloc-root=" jemalloc "/lib")
                            "--release-channel=stable"
                            "--enable-rpath"
                            "--enable-local-rust"
                            "--disable-rustbuild" ; use Makefiles
                            "--disable-manage-submodules")))

llvm version is 3.8.1 (not using the bundled llvm).

[hanna-kruppe]

Your LLVM might have been built without the NVPTX backend (not all that odd, considering the relative obscurity of that backend), but we do have conditional compilation for when LLVM targets are missing. First, to ensure Rust gets the right inputs for that, can you please check:

• Do you have the LLVMNVPTX*a libraries?
• What does llvm-config --components print?
• What does llvm-config --libs print?

If that is all consistent, the build script might be malfunctioning — likely candidates are the detection of available components and the fiddling with the linker arguments.

[daym]

Do you have the LLVMNVPTX*a libraries?

$ ls -1 libLLVMNVPTX*
libLLVMNVPTXAsmPrinter.so
libLLVMNVPTXCodeGen.so
libLLVMNVPTXDesc.so
libLLVMNVPTXInfo.so

What does llvm-config --components print?

$ ...
$ ./llvm-config --components
aarch64 aarch64asmparser aarch64asmprinter aarch64codegen aarch64desc aarch64disassembler aarch64info aarch64utils all all-targets amdgpu amdgpuasmparser amdgpuasmprinter amdgpucodegen amdgpudesc amdgpuinfo amdgpuutils analysis arm armasmparser armasmprinter armcodegen armdesc armdisassembler arminfo asmparser asmprinter bitreader bitwriter bpf bpfasmprinter bpfcodegen bpfdesc bpfinfo codegen core cppbackend cppbackendcodegen cppbackendinfo debuginfocodeview debuginfodwarf debuginfopdb engine executionengine hexagon hexagonasmparser hexagoncodegen hexagondesc hexagondisassembler hexagoninfo instcombine instrumentation interpreter ipo irreader libdriver lineeditor linker lto mc mcdisassembler mcjit mcparser mips mipsasmparser mipsasmprinter mipscodegen mipsdesc mipsdisassembler mipsinfo mirparser msp430 msp430asmprinter msp430codegen msp430desc msp430info native nativecodegen nvptx nvptxasmprinter nvptxcodegen nvptxdesc nvptxinfo objcarcopts object option orcjit passes powerpc powerpcasmparser powerpcasmprinter powerpccodegen powerpcdesc powerpcdisassembler powerpcinfo profiledata runtimedyld scalaropts selectiondag sparc sparcasmparser sparcasmprinter sparccodegen sparcdesc sparcdisassembler sparcinfo support symbolize systemz systemzasmparser systemzasmprinter systemzcodegen systemzdesc systemzdisassembler systemzinfo tablegen target transformutils vectorize x86 x86asmparser x86asmprinter x86codegen x86desc x86disassembler x86info x86utils xcore xcoreasmprinter xcorecodegen xcoredesc xcoredisassembler xcoreinfo

What does llvm-config --libs print

$ ./llvm-config --libs
-lLLVMLTO -lLLVMObjCARCOpts -lLLVMSymbolize -lLLVMDebugInfoPDB -lLLVMDebugInfoDWARF -lLLVMXCoreDisassembler -lLLVMXCoreCodeGen -lLLVMXCoreDesc -lLLVMXCoreInfo -lLLVMXCoreAsmPrinter -lLLVMSystemZDisassembler -lLLVMSystemZCodeGen -lLLVMSystemZAsmParser -lLLVMSystemZDesc -lLLVMSystemZInfo -lLLVMSystemZAsmPrinter -lLLVMSparcDisassembler -lLLVMSparcCodeGen -lLLVMSparcAsmParser -lLLVMSparcDesc -lLLVMSparcInfo -lLLVMSparcAsmPrinter -lLLVMPowerPCDisassembler -lLLVMPowerPCCodeGen -lLLVMPowerPCAsmParser -lLLVMPowerPCDesc -lLLVMPowerPCInfo -lLLVMPowerPCAsmPrinter -lLLVMNVPTXCodeGen -lLLVMNVPTXDesc -lLLVMNVPTXInfo -lLLVMNVPTXAsmPrinter -lLLVMMSP430CodeGen -lLLVMMSP430Desc -lLLVMMSP430Info -lLLVMMSP430AsmPrinter -lLLVMMipsDisassembler -lLLVMMipsCodeGen -lLLVMMipsAsmParser -lLLVMMipsDesc -lLLVMMipsInfo -lLLVMMipsAsmPrinter -lLLVMHexagonDisassembler -lLLVMHexagonCodeGen -lLLVMHexagonAsmParser -lLLVMHexagonDesc -lLLVMHexagonInfo -lLLVMCppBackendCodeGen -lLLVMCppBackendInfo -lLLVMBPFCodeGen -lLLVMBPFDesc -lLLVMBPFInfo -lLLVMBPFAsmPrinter -lLLVMARMDisassembler -lLLVMARMCodeGen -lLLVMARMAsmParser -lLLVMARMDesc -lLLVMARMInfo -lLLVMARMAsmPrinter -lLLVMAMDGPUCodeGen -lLLVMAMDGPUAsmParser -lLLVMAMDGPUDesc -lLLVMAMDGPUUtils -lLLVMAMDGPUInfo -lLLVMAMDGPUAsmPrinter -lLLVMAArch64Disassembler -lLLVMAArch64CodeGen -lLLVMAArch64AsmParser -lLLVMAArch64Desc -lLLVMAArch64Info -lLLVMAArch64AsmPrinter -lLLVMAArch64Utils -lLLVMMIRParser -lLLVMLibDriver -lLLVMOption -lLLVMTableGen -lLLVMLineEditor -lLLVMX86Disassembler -lLLVMX86AsmParser -lLLVMX86CodeGen -lLLVMSelectionDAG -lLLVMAsmPrinter -lLLVMX86Desc -lLLVMMCDisassembler -lLLVMX86Info -lLLVMX86AsmPrinter -lLLVMX86Utils -lLLVMMCJIT -lLLVMPasses -lLLVMipo -lLLVMVectorize -lLLVMLinker -lLLVMIRReader -lLLVMAsmParser -lLLVMDebugInfoCodeView -lLLVMInterpreter -lLLVMCodeGen -lLLVMScalarOpts -lLLVMInstCombine -lLLVMInstrumentation -lLLVMProfileData -lLLVMBitWriter -lLLVMOrcJIT -lLLVMTransformUtils -lLLVMExecutionEngine -lLLVMTarget -lLLVMAnalysis -lLLVMRuntimeDyld -lLLVMObject -lLLVMMCParser -lLLVMBitReader -lLLVMMC -lLLVMCore -lLLVMSupport

[hanna-kruppe]

So the build script correctly picks up that the NVPTX component is there, but doesn't link against the libraries despite them being in the --libs list. Odd.

[daym]

How do I debug the build script? It seems that stdout is already redirected...

Also, how do I invoke the build script?

[hanna-kruppe]

The build script is compiled and invoked by Cargo as part of the larger build process orchestrated by rustbuild. Invoking it manually will be quite challenging. First, rustbuild sets quite a few environment variables, override rustc, etc. — though --on-fail=sh, or some other shell in place of sh, should put you into a shell that has all the right environment variables set. Second, even with that out of the way I'm not sure if there's an easy way to compile & run the build script exactly as Cargo would.

Instead, you might try passing -vv to Cargo, as that will cause it to print the output of the build script. From a quick glance, rustbuild doesn't seem to support this, so you may have to hack it in (this looks good, but it's a blind guess).

[Ledest]

sed -i -r "/-DLLVM_TARGETS_TO_BUILD=/s/([^=])'/\1;NVPTX'/" configure
sed -i '/msp430/a("nvptx", "nvptx"),' src/tools/compiletest/src/util.rs
sed -i '/^LLVM_OPTIONAL_COMPONENTS=/s/ *$/ nvptx/' mk/main.mk

that help me build rust-1.16.0 with system llvm-3.9.1

[daym]

Note that rustbuild is really really broken for me (cargo-"error: An unknown error occurred") so I disabled rustbuild (as can be seen above).

@rkruppe:

Instead, you might try passing -vv to Cargo, as that will cause it to print the output of the build script. From a quick glance, rustbuild doesn't seem to support this, so you may have to hack it in (this looks good, but it's a blind guess).

I have changed that. And now?

--on-fail=sh, or some other shell in place of sh, should put you into a shell that has all the right environment variables set.

No. Passing "--on-fail=sh" to configure gives me "Option '--on-fail=sh' is not recognized".

@Ledest: Thank you!

But if you want I can try to figure out what went wrong instead of just patching it out.

[hanna-kruppe]

OHH, you're not using rustbuild. Then all my advice so far was completely off, the build script didn't even run. Sorry, I should have seen that in the configure flags. The workaround @Ledest posted looks good then. As the makefiles will be gone in the next release (and have already been removed from the master branch over a month ago) I don't think there's much point in trying to hunt down what exactly caused this issue.

[ipetkov]

@daym Not sure if this issue is still relevant, as today Guix bootstraps rust starting with 1.19 (1.16 isn't built anywhere).