ENH: add quantiles kw to Series.describe to create dynamic quantiles

[jreback]

This of course can be done directly, but adding the keyword quantiles that accepts a number or list (to create multiple fields), seems like a nice idea

and maybe deprecate percentile_width / replace with quantiles=[50]

like this:
http://stackoverflow.com/questions/17578115/pass-percentiles-to-pandas-agg-function

[hayd]

Assigned this to me, think it should be pretty straightforward (I don' think it even needs to be cythonized...)

removed @hayd as assignee

[TomAugspurger]

I've got an implementation for this.

If you were to do

In [7]: df = pd.DataFrame(np.array([[1, 2, 3], [4, 5, 6], [7, 8, 9]]), columns=['a', 'b', 'c'])

In [8]: df
Out[8]: 
   a  b  c
0  1  2  3
1  4  5  6
2  7  8  9

[3 rows x 3 columns]

In [9]: df.quantile([.1, .9])

Would you expect

Out[9]: 
   0.1  0.9
a  1.6  6.4
b  2.6  7.4
c  3.6  8.4

[3 rows x 2 columns]

or

Out[10]: 
       a    b    c
0.1  1.6  2.6  3.6
0.9  6.4  7.4  8.4

[2 rows x 3 columns]

For comparison:

In [11]: df.quantile(.1)
Out[11]: 
a    1.6
b    2.6
c    3.6
dtype: float64

and

In [13]: df['a'].quantile([.1, .9])
Out[13]: 
0.1    1.6
0.9    6.4
dtype: float64

[jreback]

I think this one makes the most sense as you get back the same columns (of course they are just a transpose away)

Out[10]: 
       a    b    c
0.1  1.6  2.6  3.6
0.9  6.4  7.4  8.4

[2 rows x 3 columns]

[hayd]

👍 df['a'].quantile(..) should be the same as df.quantile(..)['a'](so I guess there should be some name attributes kicking around here). This would be nice addition!

[TomAugspurger]

Thanks, that's what I was thinking.

I've got one failing test, it may be a separate issue.

In pandas/core/frame.py,

    def _apply_empty_result(self, func, axis, reduce):
        if reduce is None:
            reduce = False
            try:
                reduce = not isinstance(func(_EMPTY_SERIES), Series)  <---
            except Exception:
                pass

        if reduce:
            return Series(NA, index=self._get_agg_axis(axis))
        else:
            return self.copy()

Should this function take the *args and **kwargs** passed to df.apply? When I get to the line I marked <---, my func expects a second argument (the quantile) given in *args. Since it isn't passed, a Exception is raised.

[jreback]

hmm...this is a heuristic to figure out if its a reduction function, so yes I think that would be right (take the args)

[jreback]

I think do that fix as a separate PR