download ca cert error on Linux on Julia 1.10.1

[jaakkor2]

On Julia 1.10.1, download regressed compared to Julia 1.10.0. This is on empty depot. Side effect is that registries cannot be updated (]up).

julia> download("https://julialang.org/")
ERROR: RequestError: Error reading ca cert file /etc/ssl/certs/ca-certificates.crt - mbedTLS: (-0x3E00) PK - Read/write of file failed while requesting https://julialang.org/
Stacktrace:
  [1] (::Downloads.var"#9#18"{…})(easy::Downloads.Curl.Easy)
    @ Downloads ~/julia/julia-1.10.1/share/julia/stdlib/v1.10/Downloads/src/Downloads.jl:413
  [2] with_handle(f::Downloads.var"#9#18"{…}, handle::Downloads.Curl.Easy)
    @ Downloads.Curl ~/julia/julia-1.10.1/share/julia/stdlib/v1.10/Downloads/src/Curl/Curl.jl:95
  [3] #8
    @ ~/julia/julia-1.10.1/share/julia/stdlib/v1.10/Downloads/src/Downloads.jl:354 [inlined]
  [4] arg_write(f::Downloads.var"#8#17"{…}, arg::IOStream)
    @ ArgTools ~/julia/julia-1.10.1/share/julia/stdlib/v1.10/ArgTools/src/ArgTools.jl:134
  [5] #7
    @ ~/julia/julia-1.10.1/share/julia/stdlib/v1.10/Downloads/src/Downloads.jl:353 [inlined]
  [6] arg_read
    @ ~/julia/julia-1.10.1/share/julia/stdlib/v1.10/ArgTools/src/ArgTools.jl:76 [inlined]
  [7] request(url::String; input::Nothing, output::IOStream, method::Nothing, headers::Vector{…}, timeout::Float64, progress::Nothing, verbose::Bool, debug::Nothing, throw::Bool, downloader::Nothing)
    @ Downloads ~/julia/julia-1.10.1/share/julia/stdlib/v1.10/Downloads/src/Downloads.jl:352
  [8] request
    @ ~/julia/julia-1.10.1/share/julia/stdlib/v1.10/Downloads/src/Downloads.jl:320 [inlined]
  [9] #3
    @ ~/julia/julia-1.10.1/share/julia/stdlib/v1.10/Downloads/src/Downloads.jl:258 [inlined]
 [10] arg_write(f::Downloads.var"#3#4"{Nothing, Vector{…}, Float64, Nothing, Bool, Nothing, Nothing, String}, arg::Nothing)
    @ ArgTools ~/julia/julia-1.10.1/share/julia/stdlib/v1.10/ArgTools/src/ArgTools.jl:123
 [11] #download#2
    @ ~/julia/julia-1.10.1/share/julia/stdlib/v1.10/Downloads/src/Downloads.jl:257 [inlined]
 [12] download(url::String, output::Nothing)
    @ Downloads ~/julia/julia-1.10.1/share/julia/stdlib/v1.10/Downloads/src/Downloads.jl:246
 [13] #invokelatest#2
    @ ./essentials.jl:892 [inlined]
 [14] invokelatest
    @ ./essentials.jl:889 [inlined]
 [15] do_download(url::String, path::Nothing)
    @ Base ./download.jl:24
 [16] download(url::String)
    @ Base ./download.jl:20
 [17] top-level scope
    @ REPL[2]:1
Some type information was truncated. Use `show(err)` to see complete types.

on

$ cat /etc/centos-release
CentOS Linux release 7.9.2009 (Core)

julia> versioninfo()
Julia Version 1.10.1
Commit 7790d6f0641 (2024-02-13 20:41 UTC)
Build Info:
  Official https://julialang.org/ release
Platform Info:
  OS: Linux (x86_64-linux-gnu)
  CPU: 64 × AMD EPYC 7371 16-Core Processor
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-15.0.7 (ORCJIT, znver1)
Threads: 8 default, 0 interactive, 4 GC (on 64 virtual cores)
Environment:
  JULIA_DEPOT_PATH = /home/jaakkor2/julia/depo1

There is no /etc/ssl/certs/ca-certificates.crt on this system.

[lbilli]

I get just a warning, but similar issue:

(@v1.10) pkg> up
┌ Warning: could not download https://pkg.julialang.org/registries
│   exception = RequestError: Error reading ca cert file /etc/ssl/certs/ca-certificates.crt - mbedTLS: (-0x3E00) PK - Read/write of file failed while requesting https://pkg.julialang.org/registries
└ @ Pkg.Registry ~/apps/julia-1.10.1/share/julia/stdlib/v1.10/Pkg/src/Registry/Registry.jl:69
    Updating registry at `~/.julia/registries/General.toml`
  No Changes to `~/.julia/environments/v1.10/Project.toml`
  No Changes to `~/.julia/environments/v1.10/Manifest.toml`

Still, it's a regression compared to v1.10

julia> versioninfo()
Julia Version 1.10.1
Commit 7790d6f0641 (2024-02-13 20:41 UTC)
Build Info:
  Official https://julialang.org/ release
Platform Info:
  OS: Linux (x86_64-linux-gnu)
  CPU: 4 × Intel(R) Core(TM) i3-7100 CPU @ 3.90GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-15.0.7 (ORCJIT, skylake)
Threads: 1 default, 0 interactive, 1 GC (on 4 virtual cores)

[jaakkor2]

@atbug wrote on Slack workaround

export JULIA_SSL_CA_ROOTS_PATH=/etc/ssl/certs/ca-bundle.crt

[KristofferC]

We did bump the Downloads and NetworkOptions stdlibs for 1.10.1 so could have something to do with that.

[benlorenz]

This is probably the same as JuliaLang/NetworkOptions.jl#33 which got triggered by JuliaLang/Downloads.jl#226 in the backports for 1.10.

[KristofferC]

I thought that was fixed by JuliaLang/NetworkOptions.jl#31?

[benlorenz]

I thought that was fixed by JuliaLang/NetworkOptions.jl#31?

No, that fix was only for BUNDLED_KNOWN_HOSTS_FILE but this is SYSTEM_CA_ROOTS:

$ julia-1.10.0/bin/julia -e 'using NetworkOptions; println(NetworkOptions.SYSTEM_CA_ROOTS);'
Base.RefValue{String}(#undef)
$ julia-1.10.1/bin/julia -e 'using NetworkOptions; println(NetworkOptions.SYSTEM_CA_ROOTS);'
Base.RefValue{String}("/etc/ssl/certs/ca-certificates.crt")

(this is on opensuse where this file does not exist)

[KristofferC]

Okay.. that's unfortunate

[KristofferC]

JuliaLang/NetworkOptions.jl#34 should hopefully fix that.