Remove dependency formats from crate metadata

[bjorn3]

It only really needs to be stored for dylibs and only the dependency format for said dylib and not for any other crate types also used. It can be stored in a separate section for dylibs. For example .rsdpfmt (needs to stay within the 8 char limit for COFF section names). Furthermore omitting it from the crate metadata is necessary for implementing bazelbuild/rules_rust#228 (comment) as otherwise the dependency formats included in the crate metadata of the first compilation wouldn't match the second compilation due to dylibs not being available yet. Not encoding it in the crate metadata may also improve performance a tiny bit although rlib only crates already encode an empty list anyway.

[davidbarsky]

I was talking to @jsgf about this issue, but I had two questions:

• Does this issue interact with Tracking issue for MIR-only RLIBs #38913? If so, the work described in 38913 might be less strenuous than expected or this might be more complex than this issue might otherwise imply—can you clarify, if you've got the time?
• If it's a relatively small tweak, does this issue boil down to skipping hashes in https://doc.rust-lang.org/stable/nightly-rustc/rustc_metadata/rmeta/encoder/fn.encode_metadata.html?

To summarize/get everyone on the same page, the reason that @jsgf took the "hollow .rlib" approach (as described in bazelbuild/rules_rust#228 (comment)) was that BuckV2's rules allows for reusing the generated .rlib across the equivalent of Buck's cargo check and cargo build flavors (Jeremy allude to that in the comment, but I didn't pick up on that until he made it explicit to me).

(On an aside, I've noticed that the approach that Jeremy took incurs about 20-30% more overhead compared to emitting plain metadata, so maybe resolving this issue can hopefully reduce that discrepancy.)

[bjorn3]

Currently the crate metadata is shared between all crate types. For dylibs we still need to store the dylib dependency formats. I think the best way is to store it in an additional section of the dylib and then remove it from the main crate metadata. I don't think this interacts with MIR-only rlibs at all.

[jsgf]

Just to make sure we're all talking about the same thing, "dependency format" just means this right?

That's only linkages and crate types, so there's no hashes or any other identifiers of specific crates. So @davidbarsky I don't think this is relevant to any of the things we're talking about.

[bjorn3]

Just to make sure we're all talking about the same thing, "dependency format" just means this right?

Correct.

[mati865]

For example .rsdpfmt (needs to stay within the 8 char limit for COFF section names).

If this is only for dylibs then longer names for COFF are doable. You can put the actual long name into the string table and then call the section /<offset>, where the offset is the offset of the string in the table in decimal ASCII form.

[bjorn3]

According to

rust/compiler/rustc_codegen_ssa/src/back/metadata.rs

Lines 539 to 552 in e0d014a

	// Historical note:
	//
	// When using link.exe it was seen that the section name `.note.rustc`
	// was getting shortened to `.note.ru`, and according to the PE and COFF
	// specification:
	//
	// > Executable images do not use a string table and do not support
	// > section names longer than 8 characters
	//
	// https://docs.microsoft.com/en-us/windows/win32/debug/pe-format
	//
	// As a result, we choose a slightly shorter name! As to why
	// `.note.rustc` works on MinGW, see
	// https://github.com/llvm/llvm-project/blob/llvmorg-12.0.0/lld/COFF/Writer.cpp#L1190-L1197

the linker will truncate the section name.

[mati865]

It will for executables, but is metadata even useful for them? For dylibs it does truncate names unless you put them into the string table and use the offset.