Plotting of DatetimeIndex directly with matplotlib no longer gives datetime formatted axis (0.15)

[jorisvandenbossche]

From http://stackoverflow.com/questions/26526230/plotting-datetimeindex-on-x-axis-with-matplotlib-creates-wrong-ticks-in-pandas-0

df = pd.DataFrame({'RandomValues': np.random.randint(1, 50, 60)},
                  index=pd.date_range("2012-01-01", periods=60))

Plotting of course works:

In [79]: df.plot()
Out[79]: <matplotlib.axes._subplots.AxesSubplot at 0xf571550>

But plotting with matplotlib's plot functions gives no longer a datetime formatted x-axis, but just ints:

In [80]: plt.plot(df.index, df.RandomValues)
Out[80]: [<matplotlib.lines.Line2D at 0xfb5f748>]

Reason: matplotlibs plot calls np.atleast_1d on the input (https://github.com/matplotlib/matplotlib/blob/v1.4.0/lib/matplotlib/axes/_base.py#L220). Previously this returned an Index, now it returns a array of datetime64 values, which matplotlib can't handle.

Issues:

• This is actually a problem of matplotlib (they should support datetime64), but of course: a lot of users rely now on the behaviour that it did work with a pandas Index also.
• Is there a way to restore this behaviour? I don't see directly a solution?
• We should update our docs, as this behaviour is also mentioned there briefly: http://pandas.pydata.org/pandas-docs/stable/visualization.html#plotting-directly-with-matplotlib + figure out why this didn't fail in the doc build (the figure seems not updated with the 0.15 doc build)

It seems there was some discussion about this in the Index-no-subclass PR: #7891 (comment) and #7891 (comment) (I commented there about exactly this issue, but I don't know why we didn't look further in it then)

[jorisvandenbossche]

@sinhrks @TomAugspurger @jreback

[shoyer]

Maybe someone should bite the bullet and try to get plots of np.datetime64 working upstream in matplotlib? Here's the relevant issue: matplotlib/matplotlib#1097

[jorisvandenbossche]

yes, good point :-)
I have already been thinking a long time about that, but never found the time for it ..

[jreback]

I would just stick a note in the docs (we already have a small section about using matplotlib directly for certain higher perf plots). this is a matplotlib issue and the pandas behavior is correct (and was noted in the whatsnew for 0.15.0; PeriodIndex plotting already exhibited this) - I think you workaround is fine.

[jorisvandenbossche]

@jreback It was not mentioned in the whatsnew docs AFAIK (or did I miss it somewhere? there is something about plotting a PeriodIndex, but this is about DatetimeIndex and about plotting directly with matplotlib), but I will add it when finishing #8586

I know it is matplotlib issue, but it is just that I think a lot of people relied on this behaviour (as I did myself in my code). But as I said, don't think there is much we can do about it

[jreback]

@jorisvandenbossche oh, I meant that comment :). Ok, not sure much to do then except doc it.

[jorisvandenbossche]

It seems simply doing units.registry[np.datetime64] = DatetimeConverter() does already do the trick (see #8655), which means this can be easily solved from the pandas side (just add a units registry entry), and it should also not be that difficult to push this upstream to matplotlib for someone who has some time for this.

[jorisvandenbossche]

So we can easily register a converter for datetime64 data, as we already do for datetime.date/datetime (see https://github.com/pydata/pandas/blob/v0.15.0/pandas/tseries/converter.py#L30).
The question is: would there be people who rely on the fact that datetime64 arrays are regarded as ints in matplotlib? As this will change such plots by just importing pandas ..

Other possibility is to add to the docs that users who want to do this direct plotting with matplotlib, can always add units.registry[np.datetime64] = DatetimeConverter() themselves to their script.

[jreback]

+1 on this
we work with all kinds of other types and register converters. This is probably the most common datetime like type. so should just work. if it's this simple then let's just do it.

you can always disable by resetting the index and convert to ints.

[agijsberts]

@jorisvandenbossche The chances of this being accepted upstream in matplotlib seem slim: matplotlib/matplotlib#1097