BUG:nlargest/nsmallest does not act according to documentation with nan values

[jeffpollock9]

• I have checked that this issue has not already been reported.

• I have confirmed this bug exists on the latest version of pandas.

• (optional) I have confirmed this bug exists on the master branch of pandas.

---

Code Sample, a copy-pastable example

import pandas as pd
import numpy as np

x = pd.Series([np.nan, np.nan, 1, 2, 3])

print(x.nlargest(n=4))
# 4    3.0
# 3    2.0
# 2    1.0
# dtype: float64

print(x.sort_values(ascending=False).head(4))
# 4    3.0
# 3    2.0
# 2    1.0
# 0    NaN
# dtype: float64

Problem description

The docs state:

Faster than .sort_values(ascending=False).head(n) for small n relative to the size of the Series object.

which sort of suggests they are the same when they are not.

Expected Output

I think the sort/head approach gives the correct answer here, so we should have:

print(x.nlargest(n=4))
# 4    3.0
# 3    2.0
# 2    1.0
# 0    NaN
# dtype: float64

or the difference should be documented.

Output of pd.show_versions()

INSTALLED VERSIONS

commit : c7f7443
python : 3.9.6.final.0
python-bits : 64
OS : Linux
OS-release : 5.4.0-80-generic
Version : #90~18.04.1-Ubuntu SMP Tue Jul 13 19:40:02 UTC 2021
machine : x86_64
processor : x86_64
byteorder : little
LC_ALL : None
LANG : en_GB.UTF-8
LOCALE : en_GB.UTF-8

pandas : 1.3.1
numpy : 1.21.1
pytz : 2021.1
dateutil : 2.8.2
pip : 21.2.3
setuptools : 57.4.0
Cython : None
pytest : None
hypothesis : None
sphinx : None
blosc : None
feather : None
xlsxwriter : None
lxml.etree : None
html5lib : None
pymysql : None
psycopg2 : None
jinja2 : None
IPython : None
pandas_datareader: None
bs4 : None
bottleneck : None
fsspec : None
fastparquet : None
gcsfs : None
matplotlib : None
numexpr : None
odfpy : None
openpyxl : None
pandas_gbq : None
pyarrow : None
pyxlsb : None
s3fs : None
scipy : None
sqlalchemy : None
tables : None
tabulate : None
xarray : None
xlrd : None
xlwt : None
numba : None

[usersblock]

take

[usersblock]

On a quick debug, it seems that the latter example uses nargsort(...) in pandas\core\sorting.py and the former uses compute(...) in pandas\core\algorithms.py. Nargsort(...) splits the array into Nan and NonNan before concatenating after NonNan is sorted. On the other hand, compute(...) drops all Nan values before sorting, leading to a series with no Nan. I'll work on seeing whether modifying compute(...) to not drop Nan values would work.

[simonjayhawkins]

Thanks @jeffpollock9 for the report.

• I have checked that this issue has not already been reported.

from #28984 (comment)

Put differently, this is a case where nlargest differs from the corresponding sort_values(...).head(...) code.

I'll close this as a duplicate to keep related discussion in the same place.

[jeffpollock9]

@simonjayhawkins apologies, I saw that issue but didn't realise it was the same. Thanks for fixing.