Error: mktempdir cleanup - IOError: rm... resource busy or locked (EBUSY) on Windows

[IanButterworth]

This is seen quite frequently on Pkg's windows tests. I also had to add time delay to file write & rewrite benchmarks on Windows in SystemBenchmark to avoid writing to files that weren't closed yet.
Seems like the file isn't being reliably released during close?

For instance:

• Github actions https://github.com/JuliaLang/Pkg.jl/pull/2373/checks?check_run_id=1796514917#step:5:700
• Appveyor https://ci.appveyor.com/project/JuliaLang/pkg-jl/builds/37524143#L354

┌ Error: mktempdir cleanup
│   exception =
│    IOError: rm("C:\\Users\\runneradmin\\AppData\\Local\\Temp\\jl_rwp162\\HelloWorld"): resource busy or locked (EBUSY)
│    Stacktrace:
│      [1] uv_error
│        @ .\libuv.jl:97 [inlined]
│      [2] rm(path::String; force::Bool, recursive::Bool)
│        @ Base.Filesystem .\file.jl:289
│      [3] rm(path::String; force::Bool, recursive::Bool)
│        @ Base.Filesystem .\file.jl:282
│      [4] mktempdir(fn::Main.PkgTests.REPLTests.var"#26#36", parent::String; prefix::String)
│        @ Base.Filesystem .\file.jl:739
│      [5] mktempdir
│        @ .\file.jl:732 [inlined]
│      [6] #22
│        @ C:\hostedtoolcache\windows\julia\nightly\x64\share\julia\stdlib\v1.7\Pkg\test\repl.jl:174 [inlined]
│      [7] cd(f::Main.PkgTests.REPLTests.var"#22#32", dir::String)
│        @ Base.Filesystem .\file.jl:95
│      [8] #21
│        @ C:\hostedtoolcache\windows\julia\nightly\x64\share\julia\stdlib\v1.7\Pkg\test\repl.jl:149 [inlined]
│      [9] (::Main.PkgTests.Utils.var"#10#12"{Bool, Main.PkgTests.REPLTests.var"#21#31"})()
│        @ Main.PkgTests.Utils C:\hostedtoolcache\windows\julia\nightly\x64\share\julia\stdlib\v1.7\Pkg\test\utils.jl:107
│     [10] withenv(::Main.PkgTests.Utils.var"#10#12"{Bool, Main.PkgTests.REPLTests.var"#21#31"}, ::Pair{String, Nothing}, ::Vararg{Pair{String, Nothing}})
│        @ Base .\env.jl:161
│     [11] temp_pkg_dir(fn::Main.PkgTests.REPLTests.var"#21#31"; rm::Bool)
│        @ Main.PkgTests.Utils C:\hostedtoolcache\windows\julia\nightly\x64\share\julia\stdlib\v1.7\Pkg\test\utils.jl:98
│     [12] temp_pkg_dir(fn::Function)
│        @ Main.PkgTests.Utils C:\hostedtoolcache\windows\julia\nightly\x64\share\julia\stdlib\v1.7\Pkg\test\utils.jl:78
│     [13] top-level scope
│        @ C:\hostedtoolcache\windows\julia\nightly\x64\share\julia\stdlib\v1.7\Pkg\test\repl.jl:149
│     [14] include(mod::Module, _path::String)
│        @ Base .\Base.jl:386
│     [15] include(x::String)
│        @ Main.PkgTests C:\hostedtoolcache\windows\julia\nightly\x64\share\julia\stdlib\v1.7\Pkg\test\runtests.jl:3
│     [16] top-level scope
│        @ C:\hostedtoolcache\windows\julia\nightly\x64\share\julia\stdlib\v1.7\Pkg\test\runtests.jl:23
│     [17] include(fname::String)
│        @ Base.MainInclude .\client.jl:451
│     [18] top-level scope
│        @ none:6
│     [19] eval
│        @ .\boot.jl:369 [inlined]
│     [20] exec_options(opts::Base.JLOptions)
│        @ Base .\client.jl:268
│     [21] _start()
│        @ Base .\client.jl:492
└ @ Base.Filesystem file.jl:742

Related issues
#29658
#27982

[KristofferC]

Isn't that just a problem in Pkg that we keep a GitRepo open or something along those lines.

[IanButterworth]

If that's the case then perhaps we should set cleanup=false on the appropriate temp dir/file handlers

[KristofferC]

Or fix it so that we don't leave unclosed files behind.

[StefanKarpinski]

So the pattern I believe we're looking for is that something is created as a temporary and then opened and never closed. There are two ways this will get cleaned up and raise an error like this: if there's a do block on the mktempdir call it will get cleaned up when that ends; or if there's no do block, it will happen when the process terminates. I can't tell if these are happening at process termination or earlier.

[IanButterworth]

I don't know if this MWE is representative of the errors seen in Pkg tests, but:

julia> for i in 1:1000
           mktempdir() do tmp
               cp("test.png", joinpath(tmp,"test.png"))
               cp(joinpath(tmp,"test.png"), "test2.png",  force=true)
           end
       end
ERROR: IOError: unlink("C:\\Users\\ian\\test.png"): permission denied (EACCES)
Stacktrace:
 [1] uv_error
   @ .\libuv.jl:97 [inlined]
 [2] unlink(p::String)
   @ Base.Filesystem .\file.jl:940
 [3] rm(path::String; force::Bool, recursive::Bool)
   @ Base.Filesystem .\file.jl:272
 [4] checkfor_mv_cp_cptree(src::String, dst::String, txt::String; force::Bool)
   @ Base.Filesystem .\file.jl:313
 [5] cp(src::String, dst::String; force::Bool, follow_symlinks::Bool)
   @ Base.Filesystem .\file.jl:352
 [6] #51
   @ .\REPL[7]:4 [inlined]
 [7] mktempdir(fn::var"#51#52", parent::String; prefix::String)
   @ Base.Filesystem .\file.jl:732
 [8] mktempdir(fn::Function, parent::String)
   @ Base.Filesystem .\file.jl:730
 [9] top-level scope
   @ REPL[7]:2

[kagalenko-m-b]

The issue is still present in the latest release 1.9.4 and makes it impossible to install a number of important packages, such as Images.jl

[StefanKarpinski]

That doesn't really make sense... this is not an error that blocks installing anything, it's an error that shows up during process shutdown. If you're having a problem that prevents installation of Images, you should open an issue describing what you tried and what goes wrong.

[kagalenko-m-b]

this is not an error that blocks installing anything, it's an error that shows up during process shutdown.

More precisely, it breaks the precompilation of some crucial packages. I have reported this in a few places, including main julia repo and on Discourse, back in August. I have this issue on both Windows 7 machines I tried to install Images on.