BUG: Python parser breaks with quotes and multi-char sep

[gfyoung]

On master (b722222):

>>> data = 'a,,b\n1,,a\n2,,"2,,b"'
>>> read_csv(StringIO(data), sep=',,', engine='python')
...
ValueError: Expected 2 fields in line 3, saw 3

I expect this command to work, but because no parsing is done on quoted fields as can be seen here, an extra field is produced, breaking the parser. Note that this does not affect the C parser because multi-char delimiters are not supported. Similar to what we saw in #10911 and #12775, but unless we want to write the tokenizer.c code in Python, a similar fix does not seem trivial.

[jreback]

isn't a multi-char sep turned into a regex? (in python engine)

[gfyoung]

Yes, but that's besides the point in the grand scheme of things. It's just that the support multi-char or regex isn't really there in the Python engine if it breaks in simple cases like this.

[jreback]

does it work when no quotes are there?

[gfyoung]

Should work AFAICT without quotes. I only stumbled across it when I was trying to figure out a clever way to get regex or multi-char sep to work on the C engine by doing some pre-processing in Python layer, but this is an obstacle to that.

[jreback]

ok, why don't we just raise NotImplementedError if quoting is not None and python engine is selected (onlly for multi-char sep?)

I suspect its not worth it to actually implement, but an error would be fine.

[gfyoung]

The extra field will be created irregardless of the quoting style. It's because of the faulty splitting that the error will be raised, not the quoting style. The quotes in this example just serve to illustrate why the splitting is faulty.

The alternative to make it "work" is to regex replace the sep with a more conventional one like , but of course that means loss of data in the cases of quoted or commented data. So my first instinct is to say that we actually don't support regex or multi-char sep for either engine. How about that?