unicodedata: unicodedata.name() undocumented TypeError on empty string

[wesinator]

Bug report

Bug description:

Consider the following:

import unicodedata
unicodedata.name('')

unicodedata.name('a')

unicodedata.name('\x00')

Traceback (most recent call last):
  File "<pyshell#1>", line 1, in <module>
    unicodedata.name('')
TypeError: name() argument 1 must be a unicode character, not str

'LATIN SMALL LETTER A'

Traceback (most recent call last):
  File "<pyshell#1>", line 1, in <module>
    unicodedata.name('\x00')
ValueError: no such name

It seems there are two issues here:

1. unicodedata.name() and possibly other unicodedata methods can have an undocumented TypeError exception. https://docs.python.org/3/library/unicodedata.html
2. The TypeError on an empty string doesn't really seem accurate, since the function can still take other string values of the same str type.

Either TypeError should be mentioned in documentation, or the handling of empty strings should be fixed in the library to return a ValueError instead.

CPython versions tested on:

3.12

Operating systems tested on:

Windows, Other

Linked PRs

• gh-117557: Improve error message for 'C' format if given string is not 1 char long #117558
• gh-117557: Improve error messages when a string, bytes or bytearray of length 1 are expected #117631

[terryjreedy]

We do not try to document every possible exception, especially the common ones when they are completely clear as to the cause. There are other cases where one could argue that TypeError should be ValueError, or vice versa, but changing the status quo will break code that depends on the status quo. We could leave this alone.

The message for the empty string looks like it might possibly be a leftover from when str was bytes.

[ronaldoussoren]

This appears to be an argument clinic issue, in particular the handling of int(accept={str}), which ends up in this part of argument clinic:

cpython/Tools/clinic/libclinic/converters.py

Lines 268 to 282 in 0edde64

	elif self.format_unit == 'C':
	return self.format_code("""
	if (!PyUnicode_Check({argname})) {{{{
	{bad_argument}
	goto exit;
	}}}}
	if (PyUnicode_GET_LENGTH({argname}) != 1) {{{{
	{bad_argument}
	goto exit;
	}}}}
	{paramname} = PyUnicode_READ_CHAR({argname}, 0);
	""",
	argname=argname,
	bad_argument=self.bad_argument(displayname, 'a unicode character', limited_capi=limited_capi),
	)

This ends up with the same general error format for invalid types (unicodedata.name(42.5)) and for strings with an unexpected length like here (both empty strings and strings longer than one code point).

[erlend-aasland]

I proposed a fix in #117558. Exceptions raised and their error messages are a part of the Python API. We should take this in consideration before eventually settling on the proposed solution.

Moreover, I agree with what Terry said: we do not need to¹ document every kind of exception that can be raised.

Footnotes

1. and in some cases we should not ↩

[erlend-aasland]

cc. @serhiy-storchaka

[serhiy-storchaka]

Historically, it happened that in some cases character and string are treated as different types, even if technically characters are implemented as strings with length 1 in Python. It is not an Argument Clinic issue, PyArg_Parse* functions do the same, and maybe some handwritten parsers too.

It is a very old behavior, and I do not think that changing TypeError to ValueError is well justified here. It will break user code for sure. From semantic point of view, passing a non-1-character string is usually a programming error instead of a data driven error, and TypeError is more suitable for this.

[serhiy-storchaka]

Example of such behavior besides Argument Clinic and the PyArg_Parse* C API:

>>> 'abc'.center(10, '')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
    'abc'.center(10, '')
    ~~~~~~~~~~~~^^^^^^^^
TypeError: The fill character must be exactly one character long

[erlend-aasland]

We could align Argument Clinic to match the error message of PyArg_Parse.

[serhiy-storchaka]

Other examples:

>>> ord('')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
    ord('')
    ~~~^^^^
TypeError: ord() expected a character, but string of length 0 found
>>> '%c' % ''
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
    '%c' % ''
    ~~~~~^~~~
TypeError: %c requires int or char
>>> b'%c' % b''
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
    b'%c' % b''
    ~~~~~~^~~~~
TypeError: %c requires an integer in range(256) or a single byte

[erlend-aasland]

In any case, the error message can be improved, so I think it is still an Argument Clinic issue.

[serhiy-storchaka]

If the code of the Argument Clinic converter looks weird to you, it is because it was aligned to match PyArg_Parse.