qcut() should make sure the bins bounderies are unique before passing them to _bins_to_cuts

[yonil7]

xref #8309

for example:

pd.qcut([1,1,1,1,1,1,1,1,1,1,1,1,1,5,5,5], [0.00001, 0.5])

will raise "ValueError: Bin edges must be unique: array([ 1., 1.])" exception

Fix suggestion - add one new line:

def qcut(x, q, labels=None, retbins=False, precision=3):
    if com.is_integer(q):
        quantiles = np.linspace(0, 1, q + 1)
    else:
        quantiles = q
    bins = algos.quantile(x, quantiles)
--->bins = np.unique(bins)
    return _bins_to_cuts(x, bins, labels=labels, retbins=retbins,
        precision=precision, include_lowest=True)

[jreback]

why would you not just catch the ValueError ? better to be informed of the issue, right?

[yonil7]

I think this should not throw exception as the is legitimate use.
maybe a better example:

pd.qcut([1,2,3,4,4], [0, .25, .5, .75, 1.])

also throw: ValueError: Bin edges must be unique: array([1, 2, 3, 4, 4], dtype=int64)

[yonil7]

INSTALLED VERSIONS

commit: None
python: 2.7.5.final.0
python-bits: 64
OS: Windows
OS-release: 7
machine: AMD64
processor: Intel64 Family 6 Model 60 Stepping 3, GenuineIntel
byteorder: little
LC_ALL: None
LANG: None

pandas: 0.14.0
nose: 1.3.0
Cython: 0.19.1
numpy: 1.8.1
scipy: 0.14.0
statsmodels: 0.5.0
IPython: 1.0.0
sphinx: 1.1.3
patsy: 0.2.1
scikits.timeseries: None
dateutil: 2.2
pytz: 2014.4
bottleneck: 0.8.0
tables: 3.1.1
numexpr: 2.3.1
matplotlib: 1.3.1
openpyxl: 1.8.5
xlrd: 0.9.3
xlwt: 0.7.5
xlsxwriter: 0.5.5
lxml: 3.3.5
bs4: 4.3.1
html5lib: None
bq: None
apiclient: None
rpy2: None
sqlalchemy: 0.9.6
pymysql: None
psycopg2: None

[jreback]

maybe you misunderstand me. This is a user error (in that you generally pass unique bins, and you CAN make them unique before passing them in). Yes it could be done in the function, but wouldn't you as a user want to know that you are passing non-unique bins?

[yonil7]

The issue is regarding the pd.qcut() function (not pd.cut()). It should get array of samples and quantiles/ranks.
In the prev example, I passed array of samples [1,2,3,4,4] and would like it to calculate for each sample in that array to which of the 4 quantiles it belongs. (quantile 1,2,3 or 4)

[jreback]

its the same issue. If you uniqfy in one, you would in the other. That's not the question though. why should this happen silently? (automatically). maybe I am missing something.

@jseabold ?

[jseabold]

I would expect this to raise an informative error to the user. "The quantiles you selected do not result in unique bins. Try selecting fewer quantiles." I.e., if your 25th quantile is the same as your median, then you should be told this and you shouldn't ask for both. Trying to get fancy under the hood is asking for confusion IMO.

[jreback]

@jseabold ok, I can buy that this could produce a better msg.

@yonil7 want to take this on with a pull-request?