Get julia working on aarch64-darwin #317900

MilesCranmer · 2024-06-07T04:06:40Z

Description of changes

~~(General) Updates the julia JULIA_CPU_TARGET based on upstream binary flags from https://github.com/JuliaCI/julia-buildkite for both all systems~~ Moved to julia: update JULIA_CPU_TARGET from latest upstream settings #318350
(Darwin) Gets the julia package working on aarch64-darwin:
- (Many thanks to Julia community on https://discourse.julialang.org/t/building-julia-for-nix/115304?u=milescranmer for helping get this working)
- Enables USE_BINARYBUILDER=1 on darwin. I couldn't get the dependencies compiled from source yet, so I think this is a good solution for the time being, since the only other option is julia-bin which pre-packages the entire binary
- ~~Creates ./patches/1.10/0001-hardcode-macos-SDK.patch which hardcodes the macOS SDK to get around the issue of the Makefile calling sw_vers~~
- Adds a few dependencies which are only turned on for darwin

Things done

pkgs/development/compilers/julia/patches/1.10/0001-hardcode-macos-SDK.patch

thomasjm · 2024-06-07T05:59:02Z

This looks cool! Does julia.withPackages work?

MilesCranmer · 2024-06-07T08:14:27Z

This looks cool! Does julia.withPackages work?

I'm new to nix (I'm learning it to make building Julia easier :)). How should I try this syntax with this version of the build files?

NickCao · 2024-06-09T14:12:18Z

@ofborg build julia_19 julia_110

MilesCranmer · 2024-06-09T22:23:44Z

I finally figured out where codesign is; it is in darwin.sigtool.

MilesCranmer · 2024-06-09T22:23:53Z

@ofborg build julia_19 julia_110

MilesCranmer · 2024-06-09T22:39:00Z

What is the default make target that nix runs? I think it's trying to build the mac app or something...

The only place the current error could come from is contrib/mac/app/Makefile within the Julia source, which is for building the dmg file.

 529   │ app:
 530   │     $(MAKE) -C contrib/mac/app
 531   │     @mv contrib/mac/app/$(JULIA_BINARYDIST_FILENAME).dmg $(BUILDROOT)

NickCao · 2024-06-09T22:40:44Z

What is the default make target that nix runs?

Just the default target, as if running make with no arguments.

MilesCranmer · 2024-06-09T22:46:03Z

Thanks. Very odd. This is the current error it's stuck at:

nix run --impure --expr 'with import ./. {}; ((julia_19.withPackages) ["Plots"])'
error: builder for '/nix/store/8pwim6gghwada1n3y84bnm1lip1admc2-julia-1.9.4.drv' failed with exit code 2;
       last 25 log lines:
       > rm -f /nix/store/i7yl2z44xy4dsnwxshhnamixizhd82j0-julia-1.9.4/share/julia/stdlib/v`cut -d. -f1-2 < /private/tmp/nix-build-julia-1.9.4.drv-0/julia-1.9.4/VERSION`/*/build-configured
       > rm -f /nix/store/i7yl2z44xy4dsnwxshhnamixizhd82j0-julia-1.9.4/share/julia/stdlib/v`cut -d. -f1-2 < /private/tmp/nix-build-julia-1.9.4.drv-0/julia-1.9.4/VERSION`/*/build-compiled
       > rm -f /nix/store/i7yl2z44xy4dsnwxshhnamixizhd82j0-julia-1.9.4/share/julia/stdlib/v`cut -d. -f1-2 < /private/tmp/nix-build-julia-1.9.4.drv-0/julia-1.9.4/VERSION`/*/build-checked
       > # Copy in beautiful new man page
       > /private/tmp/nix-build-julia-1.9.4.drv-0/julia-1.9.4/contrib/install.sh 644 /private/tmp/nix-build-julia-1.9.4.drv-0/julia-1.9.4/usr/share/man/man1/julia.1 /nix/store/i7yl2z44xy4dsnwxshhnamixizhd82j0-julia-1.9.4/share/man/man1/
       > # Copy .desktop file
       > mkdir -p /nix/store/i7yl2z44xy4dsnwxshhnamixizhd82j0-julia-1.9.4/share/applications/
       > /private/tmp/nix-build-julia-1.9.4.drv-0/julia-1.9.4/contrib/install.sh 644 /private/tmp/nix-build-julia-1.9.4.drv-0/julia-1.9.4/contrib/julia.desktop /nix/store/i7yl2z44xy4dsnwxshhnamixizhd82j0-julia-1.9.4/share/applications/
       > # Install appdata file
       > mkdir -p /nix/store/i7yl2z44xy4dsnwxshhnamixizhd82j0-julia-1.9.4/share/appdata/
       > /private/tmp/nix-build-julia-1.9.4.drv-0/julia-1.9.4/contrib/install.sh 644 /private/tmp/nix-build-julia-1.9.4.drv-0/julia-1.9.4/contrib/julia.appdata.xml /nix/store/i7yl2z44xy4dsnwxshhnamixizhd82j0-julia-1.9.4/share/appdata/
       > # Update RPATH entries and JL_SYSTEM_IMAGE_PATH if ../lib/julia != ../lib/julia
       > # Set rpath for libjulia-internal, which is moving from `../lib` to `../lib/julia`.
       > install_name_tool -add_rpath @loader_path/../ /nix/store/i7yl2z44xy4dsnwxshhnamixizhd82j0-julia-1.9.4/lib/julia/libjulia-internal.dylib
       > install_name_tool -add_rpath @loader_path/../ /nix/store/i7yl2z44xy4dsnwxshhnamixizhd82j0-julia-1.9.4/lib/julia/libjulia-codegen.dylib
       > # Fix rpaths for dependencies. This should be fixed in BinaryBuilder later.
       > # Next, overwrite relative path to libjulia-internal in our loader if $(LOADER_BUILD_DEP_LIBS) != $(LOADER_INSTALL_DEP_LIBS)
       > MSYS2_ARG_CONV_EXCL='*' /private/tmp/nix-build-julia-1.9.4.drv-0/julia-1.9.4/usr/tools/stringreplace $(strings -t x - '/nix/store/i7yl2z44xy4dsnwxshhnamixizhd82j0-julia-1.9.4/lib/libjulia.1.9.dylib' | grep "libgcc_s.1.1.dylib:libopenlibm.dylib:@:@libjulia-internal.1.dylib:@libjulia-codegen.1.dylib:$" | awk '{print $1;}') "julia/libgcc_s.1.1.dylib:julia/libopenlibm.dylib:@:@julia/libjulia-internal.1.dylib:@julia/libjulia-codegen.1.dylib:" 255 "/nix/store/i7yl2z44xy4dsnwxshhnamixizhd82j0-julia-1.9.4/lib/libjulia.1.9.dylib"
       > # Codesign the libjulia we just modified
       > /private/tmp/nix-build-julia-1.9.4.drv-0/julia-1.9.4/contrib/codesign.sh "" "/nix/store/i7yl2z44xy4dsnwxshhnamixizhd82j0-julia-1.9.4/lib/libjulia.1.9.dylib"
       > Codesigning with identity -
       > Codesigning /nix/store/i7yl2z44xy4dsnwxshhnamixizhd82j0-julia-1.9.4/lib/libjulia.1.9.dylib...
       > The following arguments were not expected: --deep --timestamp -vvv --option=runtime
       > Run with --help for more information.
       > make: *** [Makefile:292: install] Error 109
       For full logs, run 'nix log /nix/store/8pwim6gghwada1n3y84bnm1lip1admc2-julia-1.9.4.drv'.
error: 1 dependencies of derivation '/nix/store/x4cwrwrpbvdd00zbncs75fgdw4w7a7qf-julia-package-closure.yml.drv' failed to build
error:
       … while calling the 'derivationStrict' builtin
         at <nix/derivation-internal.nix>:9:12:
            8|
            9|   strict = derivationStrict drvAttrs;
             |            ^
           10|

       … while evaluating derivation 'julia-1.9.4-env'
         whose name attribute is located at /Users/mcranmer/PermaDocuments/nixpkgs/pkgs/stdenv/generic/make-derivation.nix:331:7

       … while evaluating attribute 'artifactsNix' of derivation 'julia-1.9.4-env'
         at /Users/mcranmer/PermaDocuments/nixpkgs/pkgs/development/julia-modules/default.nix:181:11:
          180|   inherit minimalRegistry;
          181|   inherit artifactsNix;
             |           ^
          182|   inherit overridesJson;

       (stack trace truncated; use '--show-trace' to show the full trace)

       error: 1 dependencies of derivation '/nix/store/q6zxqwi04vi67qcsjj6yh6jwzcbakwqg-julia-sources.nix.drv' failed to build

For that specific error, the only place it could come from is (via https://github.com/search?q=repo:JuliaLang/julia%20/codesign.*option%3Druntime/&type=code) here:

https://github.com/JuliaLang/julia/blob/8a7d8f444ace6156fb73ce5a23ade9ff692fb678/contrib/mac/app/Makefile#L61

Which means it's trying to build the dmg file. Is that expected/desired? I would have thought it would just build the binary?

NickCao · 2024-06-09T22:54:36Z

What is the default make target that nix runs?

Just the default target, as if running make with no arguments.

The correct location is https://github.com/JuliaLang/julia/blob/v1.9.4/contrib/codesign.sh, github search only works with the master branch. Does this also affect 1.10?

MilesCranmer · 2024-06-09T22:56:48Z

Oops, indeed that looks like the right one. Forgot I was testing the build on 1.9.4.

Alright I'll see if I can add a patch..

MilesCranmer · 2024-06-09T22:59:04Z

@ofborg build julia_19 julia_110

MilesCranmer · 2024-06-09T23:21:28Z

Alright, now it gets all the way to building and installing which succeed. I can verify that Julia also downloads the whole of the package registry to the desired folder. However, the actual julia_19.withPackages fails due to a SIGKILL:

Resolving Julia packages with the following inputs
Julia: /nix/store/jh288dvjm6wwpchv1w424ylhzy6bq4iy-julia-1.9.4
Registry: /nix/store/gxv9wz6knd4a651ydnn98bilk76anl1d-source
/private/tmp/nix-build-julia-package-closure.yml.drv-0/.attr-0l2nkwhif96f51f4amnlf414lhl4rv9vh8iffyp431v6s28gsr90: line 124: 52453 Killed: 9               julia -e 'import Pkg
Pkg.Registry.add(Pkg.RegistrySpec(path="/nix/store/gxv9wz6knd4a651ydnn98bilk76anl1d-source"))
...

Is that an issue elsewhere or could it mean the codesigning is incorrect now (and so the OS kills it)? Not sure what else to try.

thomasjm · 2024-06-10T00:02:51Z

I've never seen a SIGKILL issue before. Can you confirm basic Julia functionality works before trying .withPackages?

MilesCranmer · 2024-06-10T03:16:04Z

It's weird, it goes through all the Julia tests in make test. I can see the LinearAlgebra tests running fine.

Although I will note I can't actually start the Julia binary stored in ./result/bin/julia from my regular terminal – I get the SIGKILL issue there too. I assumed it was due to not understanding nix and maybe from there being missing things on my LD_PRELOAD or something (since the tests within the build run fine). But now I'm not sure.

thomasjm · 2024-06-10T17:16:43Z

I don't have a Mac with Nix to test on, but some googling indicates that OS X may kill processes with improper code signing--it sounds like you might see something like EXC_BAD_ACCESS (SIGKILL (Code Signature Invalid)).

To debug further maybe you want to use something like dtruss. Maybe you can also try turning off "System Integrity Protection"?

samestep · 2025-05-16T17:16:00Z

@MilesCranmer what is the situation for this? I'm also interested in having aarch64-darwin-compatible Julia in Nixpkgs. I see the same behavior you described with ./result/bin/julia being immediately killed, though.

Judging from the lack of activity in this thread, I'm guessing you ended up giving up on Nix for Julia?

MilesCranmer · 2025-05-16T19:05:53Z

@samestep the state is as it currently sits in the PR. Feel free to take over! I think it's certainly doable to fix the issues. I simply no longer needed this package for my particular use-case and then other time pressures took priority. i.e., I won't have a chance to make progress on this for the foreseeable future.

And yes I think the issue about ./result/bin/julia being killed is simply due to code signing. I have seen the same issue with installing with juliaup install pr{number} since those aren't signed. I think I fixed it with codesign --force --sign on mac

MilesCranmer added 12 commits June 7, 2024 00:53

update JULIA_CPU_TARGET from latest upstream settings

3797017

get aarch64-darwin build working for julia

82b1c63

force warning on implicit function declaration

42c55c1

add patch for removing xcrun

0ece185

switch patch to 1.10.4

c49bad7

use binary builder and xcbuild on darwin

dd6fc73

skip more errors on darwin

c94b680

leave on warning

3996a2b

add back SDK version patch

f2e7a29

clear SSL_CA_ROOTS for julia

2ed2e9c

move JULIA_SSL_CA_ROOTS_PATH to shellHook

eb1bdd8

move JULIA_SSL_CA_ROOTS_PATH to preInstallCheck

bf1d005

ofborg bot added the 6.topic: darwin Running or building packages on Darwin label Jun 7, 2024

github-actions bot added the 6.topic: julia Julia is a high-level, high-performance dynamic language for technical computing. label Jun 7, 2024

NixOSInfra added the 12.first-time contribution This PR is the author's first one; please be gentle! label Jun 7, 2024

ofborg bot added the 8.has: package (new) This PR adds a new package label Jun 7, 2024

ofborg bot requested review from thomasjm and NickCao June 7, 2024 04:35

ofborg bot added 10.rebuild-darwin: 1-10 This PR causes between 1 and 10 packages to rebuild on Darwin. 10.rebuild-darwin: 1 This PR causes 1 package to rebuild on Darwin. 10.rebuild-linux: 1-10 This PR causes between 1 and 10 packages to rebuild on Linux. labels Jun 7, 2024

MilesCranmer added 4 commits June 7, 2024 06:02

only patchelf on linux

9de74f1

patch julia SSL cert in preInstall

17c1779

skip tests on darwin

23e0f90

compat with older julia

3232665

thomasjm reviewed Jun 7, 2024

View reviewed changes

pkgs/development/compilers/julia/patches/1.10/0001-hardcode-macos-SDK.patch Outdated Show resolved Hide resolved

ofborg bot requested a review from thomasjm June 7, 2024 06:15

use darwin tools to get sw_vers

f3cd3d1

add darwin.sigtool for codesigning

cb69395

patch options for codesign

2aab17f

wegank added the 2.status: merge conflict This PR has merge conflicts with the target branch label Sep 10, 2024

wegank added the 2.status: stale https://github.com/NixOS/nixpkgs/blob/master/.github/STALE-BOT.md label Jan 2, 2025

stale bot removed the 2.status: stale https://github.com/NixOS/nixpkgs/blob/master/.github/STALE-BOT.md label May 16, 2025

samestep mentioned this pull request May 21, 2025

Julia code is not autoformatted gradbench/gradbench#579

Open

nixpkgs-ci bot removed the 12.first-time contribution This PR is the author's first one; please be gentle! label Jun 25, 2025

Uh oh!

Get julia working on aarch64-darwin #317900

Are you sure you want to change the base?

Get julia working on aarch64-darwin #317900

Uh oh!

Conversation

MilesCranmer commented Jun 7, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description of changes

Things done

Uh oh!

Uh oh!

thomasjm commented Jun 7, 2024

Uh oh!

MilesCranmer commented Jun 7, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

NickCao commented Jun 9, 2024

Uh oh!

MilesCranmer commented Jun 9, 2024

Uh oh!

MilesCranmer commented Jun 9, 2024

Uh oh!

MilesCranmer commented Jun 9, 2024

Uh oh!

NickCao commented Jun 9, 2024

Uh oh!

MilesCranmer commented Jun 9, 2024

Uh oh!

NickCao commented Jun 9, 2024

Uh oh!

MilesCranmer commented Jun 9, 2024

Uh oh!

MilesCranmer commented Jun 9, 2024

Uh oh!

MilesCranmer commented Jun 9, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

thomasjm commented Jun 10, 2024

Uh oh!

MilesCranmer commented Jun 10, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

thomasjm commented Jun 10, 2024

Uh oh!

samestep commented May 16, 2025

Uh oh!

MilesCranmer commented May 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

MilesCranmer commented Jun 7, 2024 •

edited

Loading

MilesCranmer commented Jun 7, 2024 •

edited

Loading

MilesCranmer commented Jun 9, 2024 •

edited

Loading

MilesCranmer commented Jun 10, 2024 •

edited

Loading

MilesCranmer commented May 16, 2025 •

edited

Loading