proposal: testing: optionally preserve temp directories on failure

[neild]

This proposal is inspired by discussion in #71287.

Many tests use t.TempDir to create a temporary directory. These temporary directories are deleted after a test finishes executing.

It is sometimes useful to examine a test's temporary files when debugging a failure. This currently requires modifying the test to use a non-temporary directory (unless I'm missing something, which is possible).

I propose that we add a new test flag that disables deleting the temporary directories created for failing tests. This would apply to both directories created by t.TempDir and, when the -artifacts flag is not present, t.ArtifactDir. (Assuming #71287 is accepted.)

-keepfail
    Keep the temporary directories created by failing tests.
    The names of any retained temporary directories will be printed to the test log.

A feature along these lines was suggested in the initial proposal which added TB.TempDir, and rejected on the grounds that we already have too many test flags: #35998 (comment)

I'm sympathetic to the view that we have too many test flags, but TB.TempDir has proven to be extremely useful and adding one more flag to make it more so seems reasonable. Also, I fear that if we don't add a way to preserve temporary directories we'll just be encouraging users to switch to TB.ArtifactDir for temp files so as to have an easy way to retain temporary files on failure. That would be an unfortunate source of unnecessary churn.

[gabyhelp]

Related Issues

• testing: add TB.TempDir() string #35998 (closed)
• proposal: testing: store test artifacts #71287

_{(Emoji vote if this was helpful or unhelpful; more detailed feedback welcome in this discussion.)}

[seankhliao]

I feel it's similar to go build -work where go build's temporary directories are kept.
But maybe it's not a good idea to mix them?

[mrclmr]

If we are already in the context of tests would the name -keeptempdir more explanatory? -keepfail could sound a little bit too generic to associate it with temp directories in tests and could be harder to remember.