groupby.nth with dropna is buggy

[behzadnouri]

>>> df
   1st  2nd
0    1  NaN
1    2    3
2    1  NaN
3    2    4
>>> df.groupby('1st')['2nd'].nth(0, dropna=True)
1st
1    3
2    4
Name: 2nd, dtype: float64

[tnir]

@behzadnouri You should pass either of None, 'any' or 'all' as dropna, instead of True. We should deprecate passing True to nth.

When you however pass either 'any' or 'all', there is still a bug.

>>> df = pd.DataFrame([[1],[2,3],[1],[2,4]], columns=['1st', '2nd'])
>>> df
   1st  2nd
0    1  NaN
1    2    3
2    1  NaN
3    2    4
>>> df.groupby('1st')['2nd'].nth(0, dropna='any')
1st
1    3
2    4
Name: 2nd, dtype: float64

>>> df.groupby('1st')['2nd'].nth(0, dropna='all')
1st
1    3
2    4
Name: 2nd, dtype: float64

Exchanging the order of selecting a column and nth(), we expect the correct results as follows:

>>> df.groupby('1st').nth(0, dropna='any')['2nd']
1st
1   NaN
2     3
Name: 2nd, dtype: float64

>>> df.groupby('1st').nth(0, dropna='all')['2nd']
1st
1   NaN
2     3
Name: 2nd, dtype: float64

[jreback]

@tnir yep, the argument is a bit too free and should be checked at the top of nth, rather than in the conditional (which misses some cases).

As always pull-requests to fix are welcome.

[tnir]

This is still reproducible with pandas-0.20.3 on Python 3.6:

$ python
Python 3.6.0 (default, Dec 26 2016, 15:43:50)
[GCC 4.2.1 Compatible Apple LLVM 8.0.0 (clang-800.0.42.1)] on darwin

>>> import pandas as pd
>>> df = pd.DataFrame([[1],[2,3],[1],[2,4]], columns=['1st', '2nd'])
>>> df
   1st  2nd
0    1  NaN
1    2  3.0
2    1  NaN
3    2  4.0
>>> df.groupby('1st')['2nd'].nth(0, dropna='any')
1st
1    3.0
2    4.0
Name: 2nd, dtype: float64
>>> df.groupby('1st')['2nd'].nth(0, dropna='all')
1st
1    3.0
2    4.0
Name: 2nd, dtype: float64
>>> df.groupby('1st').nth(0, dropna='any')['2nd']
1st
1    NaN
2    3.0
Name: 2nd, dtype: float64
>>> df.groupby('1st').nth(0, dropna='all')['2nd']
1st
1    NaN
2    3.0
Name: 2nd, dtype: float64