TypeError when parsing regexp with unicode named character sequence escape

[jirkamarsik]

BPO	46410
Nosy	@vstinner, @ezio-melotti, @serhiy-storchaka, @jirkamarsik

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2022-01-17.12:31:30.830>
labels = ['expert-regex', 'interpreter-core', 'type-feature', 'expert-unicode', '3.11']
title = 'TypeError when parsing regexp with unicode named character sequence escape'
updated_at = <Date 2022-03-19.11:22:19.310>
user = 'https://github.com/jirkamarsik'

bugs.python.org fields:

activity = <Date 2022-03-19.11:22:19.310>
actor = 'serhiy.storchaka'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Interpreter Core', 'Regular Expressions', 'Unicode']
creation = <Date 2022-01-17.12:31:30.830>
creator = 'jirkamarsik'
dependencies = []
files = []
hgrepos = []
issue_num = 46410
keywords = []
message_count = 3.0
messages = ['410770', '410874', '415540']
nosy_count = 5.0
nosy_names = ['vstinner', 'ezio.melotti', 'mrabarnett', 'serhiy.storchaka', 'jirkamarsik']
pr_nums = []
priority = 'normal'
resolution = None
stage = None
status = 'open'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue46410'
versions = ['Python 3.11']

[jirkamarsik]

re.compile(r"\N{name of Unicode Named Character Sequence}"), e.g. re.compile(r"\N{KEYCAP NUMBER SIGN}"), throws a TypeError. The regular expression parser relies on 'unicodedata' to lookup character names. The 'unicodedata' module recently added support for Unicode Named Character Sequences (https://www.unicode.org/Public/13.0.0/ucd/NamedSequences.txt). Trying to use these named character sequences in a regular expression leads to a 'TypeError', as the regexp parser tries to call 'ord' on a string with length > 1.

[mrabarnett]

They're not supported in string literals either:

Python 3.10.1 (tags/v3.10.1:2cd268a, Dec  6 2021, 19:10:37) [MSC v.1929 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> "\N{KEYCAP NUMBER SIGN}"
  File "<stdin>", line 1
    "\N{KEYCAP NUMBER SIGN}"
                            ^
SyntaxError: (unicode error) 'unicodeescape' codec can't decode bytes in position 0-21: unknown Unicode character name

[serhiy-storchaka]

>>> import unicodedata
>>> unicodedata.lookup('KEYCAP NUMBER SIGN')
'#️'
>>> print(ascii(unicodedata.lookup('KEYCAP NUMBER SIGN')))
'#\ufe0f\u20e3'

Support of Unicode Named Character Sequences in the unicodeescape codec and in the RE parser would be a new feature.