ValueError from operations between Series with specific names

[dan1261]

The following raises a ValueError due to comparison of the two names in the function _maybe_match_name (core/ops/common.py).

>>> from pandas import Series
>>> import numpy as np
>>> series_a = Series([1,2,3], name=np.float64(1.0))
>>> series_b = Series([1,2,3], name=(np.float64(1.0), np.float64(2.0)))
>>> series_a + series_b

It seems to only occur when comparison a tuple of numpy dtypes to a single numpy dtype, but I didn't test every possible combination. It does not occur when using built-in floats, I imagine this is due to overloading of the == operator by the numpy dtypes when the other operand is a tuple.

There is a nearby TODO "what if they both have np.nan for their names?" I don't know how feasible it is to consider every case where equality fails, but it seems a try/except returning None would seem to resolve this issue, and be in line with the default return value if the names do not compare equal. I would be keen to implement this simple fix if it seems sensible.

INSTALLED VERSIONS

commit : None
python : 3.8.0.final.0
python-bits : 64
OS : Windows
OS-release : 10
machine : AMD64
processor : Intel64 Family 6 Model 158 Stepping 9, GenuineIntel
byteorder : little
LC_ALL : None
LANG : None
LOCALE : English_United Kingdom.1252
pandas : 1.0.3
numpy : 1.19.0
pytz : 2020.1
dateutil : 2.8.1
pip : 20.2.4
setuptools : 47.3.1
Cython : None
pytest : None
hypothesis : None
sphinx : None
blosc : None
feather : None
xlsxwriter : None
lxml.etree : 4.5.0
html5lib : 1.1
pymysql : None
psycopg2 : None
jinja2 : 2.11.2
IPython : 7.19.0
pandas_datareader: 0.8.1
bs4 : 4.9.1
bottleneck : None
fastparquet : None
gcsfs : None
lxml.etree : 4.5.0
matplotlib : 3.2.2
numexpr : 2.7.1
odfpy : None
openpyxl : None
pandas_gbq : None
pyarrow : None
pytables : None
pytest : None
pyxlsb : None
s3fs : None
scipy : 1.5.0
sqlalchemy : None
tables : None
tabulate : None
xarray : None
xlrd : None
xlwt : None
xlsxwriter : None
numba : 0.51.2

[rhshadrach]

Thanks for the report! From https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.Series.html:

name: str, optional

Certainly other types (integers) need to be allowed, but I wonder how much of pandas also depends on the name being comparable. An alternative would be to add this requirement.

[dan1261]

It seems to me that None is the default output when a name cannot be reasonably deduced, and so a try/except block could be a clean fix, if not a permanent design choice. Series with tuple-type names can arise naturally when accessing a DataFrame with multi-level columns for example.

[rhshadrach]

@dan1261 - I think this issue is not the presence of tuples but rather numpy objects. Tuples are comparable (edit: assuming their components are).

[dan1261]

@rhshadrach indeed it seems the only problem is a comparison of a numpy object with a tuple of numpy objects. Just wanted to highlight that tuples of such objects can occur naturally. I believe the try/except fix should work in any case

[rhshadrach]

@dan1261 - In this case yes, but other methods fail, e.g.:

series_a = Series([1,2,3], name=np.float64(1.0))
series_b = Series([1,2,3], name=(np.float64(1.0), np.float64(2.0)))
pd.concat([series_a, series_b], axis=1).columns.sort_values()

raises

File "pandas/pandas/pandas/core/sorting.py", line 385, in nargsort
    indexer = non_nan_idx[non_nans.argsort(kind=kind)]

which would also need to be fixed. It seems to me trying to support such objects is opening a can of worms.