Skip to content

ast.parse(type_comments=True) fails on some parenthesized context managers #111420

Closed
@JelleZijlstra

Description

@JelleZijlstra

Bug report

Bug description:

ast.parse(..., type_comments=True) doesn't deal well with parenthesized context managers, as introduced by PEP-617.

>>> ast.dump(ast.parse("with (a as b): # type: something\n    pass", type_comments=True))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
    ast.dump(ast.parse("with (a as b): # type: something\n    pass", type_comments=True))
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/jelle/py/cpython/Lib/ast.py", line 61, in parse
    return compile(
           ^^^^^^^^
  File "<unknown>", line 1
    with (a as b): # type: something
                           ^^^^^^^^^
SyntaxError: invalid syntax

But this works without the parentheses:

>>> ast.dump(ast.parse("with a as b: # type: something\n    pass", type_comments=True))
"Module(body=[With(items=[withitem(context_expr=Name(id='a', ctx=Load()), optional_vars=Name(id='b', ctx=Store()))], body=[Pass()], type_comment='something')], type_ignores=[])"

This code gets interpreted incorrectly:

>>> ast.dump(ast.parse("with (a, b): # type: something\n    pass", type_comments=True))
"Module(body=[With(items=[withitem(context_expr=Tuple(elts=[Name(id='a', ctx=Load()), Name(id='b', ctx=Load())], ctx=Load()))], body=[Pass()], type_comment='something')], type_ignores=[])"

There is a single withitem containing a tuple, when it should instead be two withitems, as happens without the comment:

>>> ast.dump(ast.parse("with (a, b):\n    pass", type_comments=True))
"Module(body=[With(items=[withitem(context_expr=Name(id='a', ctx=Load())), withitem(context_expr=Name(id='b', ctx=Load()))], body=[Pass()])], type_ignores=[])"

Found this while looking into psf/black#3677.

CPython versions tested on:

CPython main branch

Operating systems tested on:

macOS

Linked PRs

Metadata

Metadata

Assignees

No one assigned

    Labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions