add trailing_zeros and leading_zeros to non zero types

[andjo403]

as a way towards being able to use the optimized intrinsics ctlz_nonzero and cttz_nonzero from stable.

have not crated any tracking issue if this is not a solution that is wanted

[rust-highfive]

r? @m-ou-se

(rust_highfive has picked a reviewer for you, use r? to override)

[scottmcm]

I'm a fan 👍 This is important for avoiding dead ASM for the default x64 target, but can even improve the generated code on newer target-cpus too because of the tighter value range. An example: #70835 (comment)

[m-ou-se]

Looks like a good addition!

Maybe it's good to explain the existence of these functions in the doc comments? Maybe something like On many architectures, this function can perform better than `leading_zeros()` on the underlying integer type, as special handling of zero can be avoided., or something in that direction.

[andjo403]

thanks for the comments have addressed them and pushed up the changes now

[m-ou-se]

Thanks! One more small comment:

[andjo403]

thanks again and fixed

[m-ou-se]

Thanks!

@bors r+ rollup=always

[bors]

📌 Commit 9bbc4c1 has been approved by m-ou-se

[leonardo-m]

I've tried this in my codebase, looking at the generated asm, and I've seen that in 100% of the cases, thanks sometimes to inlining, LLVM was able to infer that the input isn't zero, so no performance change has happened.

[m-ou-se]

In cases where the non-zero value are is created somewhat close to the place where leading_zeros/trailing_zeros is applied, it will definitely optimize this nicely. But just using NonZeroI32 by itself without the context that creates the value, doesn't result in this optimization (yet?): https://godbolt.org/z/bEoPE9

[scottmcm]

@leonardo-m As another example, note that LLVM is currently not capable of optimizing this even when there's an explicit check for zero in the code before doing the uN::leading_zeros: https://rust.godbolt.org/z/4EhnK4

[leonardo-m]

But just using NonZeroI32 by itself without the context that creates the value, doesn't result in this optimization (yet?): https://godbolt.org/z/bEoPE9

Using "-C opt-level=2 -C target-cpu=native" it seems to optimize it well.

[leonardo-m]

As another example, note that LLVM is currently not capable of optimizing this even when there's an explicit check for zero in the code before doing the uN::leading_zeros: https://rust.godbolt.org/z/4EhnK4

With "-C opt-level=3 -C target-cpu=native -Z mir-opt-level=3" it seems to optimize well.

[m-ou-se]

@leonardo-m With that option it makes use of an instruction that does not need a special case for zero. It still doesn't make use of the fact that NonZero* cannot be zero.

[leonardo-m]

Oh, fun, thank you. It's still the same LLVM limit, I guess:
#54868

[scottmcm]

With "-C opt-level=3 -C target-cpu=native -Z mir-opt-level=3" it seems to optimize well.

native there isn't doing anything different from haswell in my example -- the choice was those two was just to pick up instruction sets with and without BMI1.

Very interesting that mir-opts affect this, though...