Closed
Description
In Pandas 0.17, if I put a Series into an HDFStore in table format, and if this Series contains a day light saving transition, then Pandas crashes with an AmbiguousTimeError.
Here's the minimal code example to reproduce the bug:
import pandas as pd
index = pd.date_range("2013-10-26 23:00", "2013-10-27 01:00",
tz="Europe/London", freq="H")
series = pd.Series(0, index=index)
with pd.HDFStore("test.hdf", mode="w") as store:
store.put("series", series, format='table')
store["series"]And here's the traceback:
In [4]: ---------------------------------------------------------------------------
AmbiguousTimeError Traceback (most recent call last)
<ipython-input-4-5deb3c83fafd> in <module>()
----> 1 __pyfile = open('''/tmp/py7201vZi''');exec(compile(__pyfile.read(), '''/home/jack/temp/pandas_dst_bug.py''', 'exec'));__pyfile.close(); import os; os.remove('''/tmp/py7201vZi''')
/home/jack/temp/pandas_dst_bug.py in <module>()
2 index = pd.date_range("2013-10-26 23:00", "2013-10-27 01:00",
3 tz="Europe/London", freq="H")
----> 4 series = pd.Series(0, index=index)
5 with pd.HDFStore("test.hdf", mode="w") as store:
6 store.put("series", series, format='table')
/usr/local/lib/python2.7/dist-packages/pandas/tseries/index.pyc in date_range(start, end, periods, freq, tz, normalize, name, closed)
1912 return DatetimeIndex(start=start, end=end, periods=periods,
1913 freq=freq, tz=tz, normalize=normalize, name=name,
-> 1914 closed=closed)
1915
1916
/usr/local/lib/python2.7/dist-packages/pandas/util/decorators.pyc in wrapper(*args, **kwargs)
87 else:
88 kwargs[new_arg_name] = new_arg_value
---> 89 return func(*args, **kwargs)
90 return wrapper
91 return _deprecate_kwarg
/usr/local/lib/python2.7/dist-packages/pandas/tseries/index.pyc in __new__(cls, data, freq, start, end, periods, copy, name, tz, verify_integrity, normalize, closed, ambiguous, dtype, **kwargs)
234 return cls._generate(start, end, periods, name, freq,
235 tz=tz, normalize=normalize, closed=closed,
--> 236 ambiguous=ambiguous)
237
238 if not isinstance(data, (np.ndarray, Index, ABCSeries)):
/usr/local/lib/python2.7/dist-packages/pandas/tseries/index.pyc in _generate(cls, start, end, periods, name, offset, tz, normalize, ambiguous, closed)
450
451 if end is not None and end.tz is None:
--> 452 end = end.tz_localize(tz)
453
454 if start and end:
pandas/tslib.pyx in pandas.tslib.Timestamp.tz_localize (pandas/tslib.c:11965)()
pandas/tslib.pyx in pandas.tslib.tz_localize_to_utc (pandas/tslib.c:64516)()
AmbiguousTimeError: Cannot infer dst time from Timestamp('2013-10-27 01:00:00'), try using the 'ambiguous' argumentEverything works fine if:
- we use
format=fixedinstead offormat=table - we used
format=tablebut theSeriesdoes not include a DST change
I'm not certain but I expect this bug is related to #11619
I'm 99.999% certain that this bug did not exist in Pandas 0.16.2
In [30]: show_versions()
INSTALLED VERSIONS
------------------
commit: None
python: 2.7.9.final.0
python-bits: 64
OS: Linux
OS-release: 3.19.0-33-generic
machine: x86_64
processor: x86_64
byteorder: little
LC_ALL: None
LANG: en_GB.UTF-8
pandas: 0.17.0
nose: 1.3.7
pip: 1.5.6
setuptools: 15.2
Cython: 0.23.1
numpy: 1.10.1
scipy: 0.16.0
statsmodels: 0.6.1
IPython: 4.0.0
sphinx: 1.2.3
patsy: 0.3.0
dateutil: 2.4.2
pytz: 2015.7
blosc: None
bottleneck: None
tables: 3.2.2
numexpr: 2.4.6
matplotlib: 1.4.3
openpyxl: None
xlrd: 0.9.2
xlwt: None
xlsxwriter: None
lxml: None
bs4: 4.3.2
html5lib: 0.999
httplib2: 0.9
apiclient: None
sqlalchemy: None
pymysql: None
psycopg2: 2.5.3 (dt dec pq3 ext)
Finally, just to say another huge thank you to everyone who supports Pandas. It must be a huge amount of work and I am hugely grateful. Pandas is an awesome tool. Thank you.
Metadata
Metadata
Assignees
Labels
No labels