Issue using -ffast-math with NaN

[tlepley-cadence]

With clang, libjit is compiled with the -ffast-math option, but it should be avoided in case of usage of NaN. For instance, in gcc, the -ffast-math flag activates the -ffinite-math-only flag that disables the NaN functionality.

With the ReplaceNaN operator, Glow makes use of NaN in libjit.

It turns out that in my Linux environment, libjit_element_is_nan_kernel_f doesn't work properly. Even when a NaN number is provided, isnan() returns false (while printf by the way correctly prints NaN).
It makes the CPU/InterpAndCPU.replaceNaN/0 test fail.

When I disable -ffast-math for the target CPURuntime, the test passes correctly

[SplitInfinity]

I remember seeing this on Travis CI ASAN runs shortly before we switched to Circle CI but I was never able to figure out why. I remember testing with ASAN locally (but on Darwin with clang) and not seeing any problems. Thanks for solving this mystery!

[nadavrot]

I suspect that disabling -ffast-math would also disable vectorization. How do we verify that vectorization still runs?

[tlepley-cadence]

Maybe an option can be to replace -ffast-math with a set of lower level math optim flags flags that don't disable NaN while keeping other math optims (for best vectorization).

For instance, for gcc, this is what the documentation tells:

It enables -fno-trapping-math, -funsafe-math-optimizations, -ffinite-math-only, -fno-errno-math, -fno-signaling-nans, -fno-rounding-math, -fcx-limited-range and -fno-signed-zeros.

What we only need for now is to have support for NaN, so -ffinite-math-only has to be avoided. But all other math optim flags may be fine for Glow.

But currently the issue is with Clang, and it's not clear to me what -ffast-math activates and if it can be replaced by a list of lower level math options.

[nadavrot]

I am not sure what the problem is. How is the generated assembly code changes and how it affects nans?

[bertmaher]

It looks like with -Ofast the compiler simply assumes NaNs don't happen, so std::isnan simply returns false: https://gcc.godbolt.org/z/--VZW7 (this is LLVM 7).

I fiddled a bit with -ffast-math this morning, and have a few thoughts:

• elementwise operations look OK. I only tested elementwise add but the code is identical.
• dot product is terrible, since it depends on associative reordering. But the expensive dot-product kernels (gemm, conv) are also probably hand-optimized so maybe this isn't so bad in practice
• operations like exp and log use vectorized approximations that assume finiteness, which can cause weird issues with NaNs

[tlepley-cadence]

My proposal is to add -fno-finite-math-only that disables from -ffast-math only what we need to get a correct NaN behaviour, while still benefiting from math optims. With this, the NaN related glow test stops failing in our environment. See #2117.