read_html failing in many tests: AssertionError: No tables found

[yarikoptic]

typical error with

======================================================================
FAIL: test_banklist (pandas.io.tests.test_html.TestReadHtmlBase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/yoh/deb/gits/pkg-exppsy/build-area/pandas-0.11.0+git43-g7b2eaa4/debian/tmp/usr/lib/python2.7/dist-packages/pandas/io/tests/test_html.py", line 114, in test_banklist
    attrs={'id': 'table'})
  File "/home/yoh/deb/gits/pkg-exppsy/build-area/pandas-0.11.0+git43-g7b2eaa4/debian/tmp/usr/lib/python2.7/dist-packages/pandas/io/tests/test_html.py", line 67, in run_read_html
    return read_html(*args, **kwargs)
  File "/home/yoh/deb/gits/pkg-exppsy/build-area/pandas-0.11.0+git43-g7b2eaa4/debian/tmp/usr/lib/python2.7/dist-packages/pandas/io/html.py", line 900, in read_html
    attrs)
  File "/home/yoh/deb/gits/pkg-exppsy/build-area/pandas-0.11.0+git43-g7b2eaa4/debian/tmp/usr/lib/python2.7/dist-packages/pandas/io/html.py", line 769, in _parse
    raise retained
AssertionError: No tables found

[yarikoptic]

I guess another related failure

======================================================================
FAIL: pandas.io.tests.test_html.test_bs4_finds_tables
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/nose/case.py", line 197, in runTest
    self.test(*self.arg)
  File "/home/yoh/deb/gits/pkg-exppsy/build-area/pandas-0.11.0+git43-g7b2eaa4/debian/tmp/usr/lib/python2.7/dist-packages/pandas/io/tests/test_html.py", line 453, in test_bs4_finds_tables
    assert get_elements_from_url(filepath, 'table')
AssertionError

[jreback]

can you show ci/print_versions.py?

[yarikoptic]

Here is what I have ATM (without running the build with adjusted
matplotlib backend and PYTHONPATH... which shouldn't matter -- just want
to make clear if I am not missing anything):

$> ci/print_versions.py

INSTALLED VERSIONS

Python: 2.7.5.final.0
OS: Linux 3.2.0-4-amd64 #1 SMP Debian 3.2.41-2 x86_64
LC_ALL: None
LANG: en_US

Cython: 0.19
Numpy: 1.7.1
Scipy: 0.10.1
statsmodels: 0.4.2
patsy: Not installed
scikits.timeseries: Not installed
dateutil: 1.5
pytz: 2012c
bottleneck: Not installed
PyTables: 2.3.1
numexpr: 2.0.1
matplotlib: 1.1.1rc2
openpyxl: 1.6.1
xlrd: 0.6.1
xlwt: 0.7.4
sqlalchemy: 0.7.9
lxml: 3.2.0
bs4: 4.2.0
html5lib: 0.95-dev

On Thu, 11 Jul 2013, jreback wrote:

can you show ci/print_versions.py?

—
Reply to this email directly or [1]view it on GitHub.

References

Visible links

1. read_html failing in many tests: AssertionError: No tables found #4214 (comment)

Yaroslav O. Halchenko, Ph.D.
http://neuro.debian.net http://www.pymvpa.org http://www.fail2ban.org
Senior Research Associate, Psychological and Brain Sciences Dept.
Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755
Phone: +1 (603) 646-9834 Fax: +1 (603) 646-1419
WWW: http://www.linkedin.com/in/yarik

[jreback]

@cpcloud ?

[cpcloud]

@yarikoptic this is master correct?

[cpcloud]

7097368 should have fixed these issues

it looks like you're using an older version of html5lib. i believe 1.0b2 is out.

also, check out the reading html gotchas

[yarikoptic]

yes -- master 0.11.0+git43-g7b2eaa4

thanks for the fix -- I will check it out

html5lib -- well, I have the freshiest non-beta release ;) thanks for the note though!

[cpcloud]

i'll try this on a vagrant box

the commit i referenced above is already in master

[cpcloud]

@yarikoptic is it of any significance that your version says 0.11.0?

$ git describe
v0.12.0rc1-43-g7b2eaa4

[cpcloud]

@yarikoptic there are no guarantees that your bs4 version will work...

see the optional dependencies docs

also your lxml version might need to be upgraded.

i would say: leave html5lib alone for now that should be fine, but change your bs4 from 4.2.0 to either 4.2.1, 4.1.3 or 4.0.2. then run tests. if lxml still fails upgrade to 3.2.1

if none of this works then i'll mark it as a bug

[cpcloud]

there's a release of bs4 that really shouldn't have existed, i believe it is 4.2.0.

[cpcloud]

fyi travis installs bs4 4.0.2 and that works...i'm not sure how different ubuntu 12.04 is from debian sid, but i imagine they share a lot of similarity, i'm sure you know much more than i about this

[jreback]

It think @cpcloud should setup a version dependency hotline, kind of like AA :)

[cpcloud]

Hello, this is read_html-one-one, what's your emergency?

[yarikoptic]

ah -- thanks -- should have been indeed 0.12.0~rc1+git... as the Debian
perspective version -- fixed in my debian branch.

On Thu, 11 Jul 2013, Phillip Cloud wrote:

[1]@yarikoptic is it of any significance that your version says 0.11.0?

$ git describe
v0.12.0rc1-43-g7b2eaa4

Yaroslav O. Halchenko, Ph.D.
http://neuro.debian.net http://www.pymvpa.org http://www.fail2ban.org
Senior Research Associate, Psychological and Brain Sciences Dept.
Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755
Phone: +1 (603) 646-9834 Fax: +1 (603) 646-1419
WWW: http://www.linkedin.com/in/yarik