BUG: Inconsistent type resolution for 0d arrays

[eric-wieser]

Using direct ufunc calls below to eliminate operator overloading from the test case:

>>> one_eps = 1.00000001
>>> np.greater_equal(np.array(1.0, np.float32), np.array(one_eps, np.float64))
False
>>> np.greater_equal(np.array([1.0], np.float32), np.array(one_eps, np.float64))
array([ True])  # wrong!
>>> np.greater_equal(np.array([1.0], np.float32), np.array([one_eps], np.float64))
array([False])
>>> np.greater_equal(np.array(1.0, np.float32), np.array([one_eps], np.float64))
array([False])

Caused by result_type having some unusual rules:

>>> np.result_type(np.float32, np.float64)
dtype('float64') # ok
>>> np.result_type(np.float32, np.float64([1]))
dtype('float64') # ok
>>> np.result_type(np.float32, np.float64(1))
dtype('float32') # what

This seems wrong to me, but appears to be deliberate.

It seems the goal here is to interpret python scalars as loosely-typed (in the presence of arrays). This handles thing like:

• letting integer literals choose between signed and unsigned
• letting number literals have smaller representations than float64 and int_ if they'd fit

However, the method is too greedy, and also (in the presence of arrays) discards type information from:

1. np.number instances with an explicit type, like np.int64(1) and np.float32(1)
2. 0d ndarray instances, by decaying them to their scalars

The problem is that PyArray_ResultType (and the ufunc type resolver) has lost the information about where it's array arguments came from, and whether their types are explicit or implicit.

[seberg]

Yeah, this stuff is pretty annoying... But I think we need to do a major release (hehe, we can do those anyway ;)) to change it another time.

I suppose we could try to aim for 0d arrays be interpreted as arrays here in principle. In practice you need to also fix the ufunc machinery to actually also signal the scalar when it was one, I am sure.

Also as long as 0d arrays are second class citizens compared to scalars, I do not really see this improving much personally. My personal opinion is still that we should aim for (in the very long run) 0d arrays and scalars (not mutable normally) and things like arr.sum() would be a scalar, while arr.sum((0, 1)) would be a 0d array. I actually believe that it is transparent, though its probably just consistent and that makes me think it is transparent ;).

[mhvk]

Duplicate of #9194. It is probably worth noting that one can avoid the problem by also passing a dtype to the ufunc.

It also worth noting that the solution is not that obvious. E.g., the following would, generally be the expected result:

np.equal(np.arange(0., 0.3, 0.1, dtype=np.float32), 0.1)
# array([False,  True, False], dtype=bool)

At least, I think we'd get even more bug reports if the following was the default!

np.equal(np.arange(0., 0.3, 0.1, dtype=np.float32), 0.1, dtype=float)
# array([False, False, False], dtype=bool)

[eric-wieser]

E.g., the following would, generally be the expected result:

Right, but I'd argue that's because it uses the float type for 0.1. I'd also expect:

np.equal(np.arange(0., 0.3, 0.1, dtype=np.float32), np.float64(0.1))
# array([False, False, False], dtype=bool)

[seberg]

Hehe, true, but equality with floats is a tricky beast in any case and nobody is suggesting to change it for the python scalar case I believe, possibly for np.equal(np.arange(0., 0.3, 0.1, dtype=np.float32), np.float64(0.1)) or at least for np.equal(np.arange(0., 0.3, 0.1, dtype=np.float32), np.array(0.1)).

Which both are of course a bit annoying since it breaks the "casting everything first with np.array" is the same thing identity of course….

[eric-wieser]

@mhvk: I've leave it to you to decide whether to close this or the old one

[mhvk]

Closed #9194, so we can discuss in one place.

It does seem to me array scalars should always be treated like arrays.

My ideal future would have gotten rid of numpy scalars altogether - use array scalars mostly and python numbers when needed - but perhaps I am missing what they can be useful for.