3.13.0 REPL crashes on double ctrl+z in line overflow

[base-13]

Bug report

Bug description:

The new REPL in 3.13.0 crashes on pressing ctrl+Z twice at the end of a overflowing line, refer to the attachment

2024-11-02.17-22-35.mov

CPython versions tested on:

3.13

Operating systems tested on:

Windows

Linked PRs

• gh-126332: Fix pyrepl crash for double ctrl-z in line overflow #126650
• [3.13] gh-126332: Fix pyrepl crash for double ctrl-z in line overflow (GH-126650) #129154
• gh-126332: Add tests for _pyrepl.utils #129325
• [3.13] gh-126332: Add tests for _pyrepl.utils (GH-129325) #130414

[base-13]

I found even more weird things which happens when handing line overflows

2024-11-02.17-36-48.mov

[JamesParrott]

Something similar happens with a numeric literal:

Double Ctrl+Z

Python 3.13.0 (tags/v3.13.0:60403a5, Oct  7 2024, 09:38:07) [MSC v.1941 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> 1111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111\11111111111111111111^ZTraceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "%LOCALAPPDATA%\Programs\Python\Python313\Lib\_pyrepl\__main__.py", line 6, in <module>
    __pyrepl_interactive_console()
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^
  File "%LOCALAPPDATA%\Programs\Python\Python313\Lib\_pyrepl\main.py", line 59, in interactive_console
    run_multiline_interactive_console(console)
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^
  File "%LOCALAPPDATA%\Programs\Python\Python313\Lib\_pyrepl\simple_interact.py", line 151, in run_multiline_interactive_console
    statement = multiline_input(more_lines, ps1, ps2)
  File "%LOCALAPPDATA%\Programs\Python\Python313\Lib\_pyrepl\readline.py", line 389, in multiline_input
    return reader.readline()
           ~~~~~~~~~~~~~~~^^
  File "%LOCALAPPDATA%\Programs\Python\Python313\Lib\_pyrepl\reader.py", line 801, in readline
    self.handle1()
    ~~~~~~~~~~~~^^
  File "%LOCALAPPDATA%\Programs\Python\Python313\Lib\_pyrepl\reader.py", line 784, in handle1
    self.do_cmd(cmd)
    ~~~~~~~~~~~^^^^^
  File "%LOCALAPPDATA%\Programs\Python\Python313\Lib\_pyrepl\reader.py", line 714, in do_cmd
    self.refresh()
    ~~~~~~~~~~~~^^
  File "%LOCALAPPDATA%\Programs\Python\Python313\Lib\_pyrepl\reader.py", line 692, in refresh
    self.console.refresh(self.screen, self.cxy)
    ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^
  File "%LOCALAPPDATA%\Programs\Python\Python313\Lib\_pyrepl\windows_console.py", line 191, in refresh
    self.__write_changed_line(y, oldline, newline, px)
    ~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^
  File "%LOCALAPPDATA%\Programs\Python\Python313\Lib\_pyrepl\windows_console.py", line 238, in __write_changed_line
    and oldline[x_pos] == newline[x_pos]
        ~~~~~~~^^^^^^^
IndexError: string index out of range

I'm not sure what the intended behaviour is with overflowed lines in the command history, but it does strike me as odd too.

[base-13]

Something similar happens with a numeric literal:

Double Ctrl+Z

Python 3.13.0 (tags/v3.13.0:60403a5, Oct  7 2024, 09:38:07) [MSC v.1941 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> 1111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111\11111111111111111111^ZTraceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "%LOCALAPPDATA%\Programs\Python\Python313\Lib\_pyrepl\__main__.py", line 6, in <module>
    __pyrepl_interactive_console()
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^
  File "%LOCALAPPDATA%\Programs\Python\Python313\Lib\_pyrepl\main.py", line 59, in interactive_console
    run_multiline_interactive_console(console)
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^
  File "%LOCALAPPDATA%\Programs\Python\Python313\Lib\_pyrepl\simple_interact.py", line 151, in run_multiline_interactive_console
    statement = multiline_input(more_lines, ps1, ps2)
  File "%LOCALAPPDATA%\Programs\Python\Python313\Lib\_pyrepl\readline.py", line 389, in multiline_input
    return reader.readline()
           ~~~~~~~~~~~~~~~^^
  File "%LOCALAPPDATA%\Programs\Python\Python313\Lib\_pyrepl\reader.py", line 801, in readline
    self.handle1()
    ~~~~~~~~~~~~^^
  File "%LOCALAPPDATA%\Programs\Python\Python313\Lib\_pyrepl\reader.py", line 784, in handle1
    self.do_cmd(cmd)
    ~~~~~~~~~~~^^^^^
  File "%LOCALAPPDATA%\Programs\Python\Python313\Lib\_pyrepl\reader.py", line 714, in do_cmd
    self.refresh()
    ~~~~~~~~~~~~^^
  File "%LOCALAPPDATA%\Programs\Python\Python313\Lib\_pyrepl\reader.py", line 692, in refresh
    self.console.refresh(self.screen, self.cxy)
    ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^
  File "%LOCALAPPDATA%\Programs\Python\Python313\Lib\_pyrepl\windows_console.py", line 191, in refresh
    self.__write_changed_line(y, oldline, newline, px)
    ~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^
  File "%LOCALAPPDATA%\Programs\Python\Python313\Lib\_pyrepl\windows_console.py", line 238, in __write_changed_line
    and oldline[x_pos] == newline[x_pos]
        ~~~~~~~^^^^^^^
IndexError: string index out of range

I'm not sure what the intended behaviour is with overflowed lines in the command history, but it does strike me as odd too.

the expression is not the problem, how the REPL is handling overflowing lines is the problem, so it doesn't matter whether its a number literal or a identifier

[eendebakpt]

I can confirm the issue in current main. In the __write_changed_line method there is a guard for the while loop using x_coord:

        while (
            x_coord < minlen
            and oldline[x_pos] == newline[x_pos]
            and newline[x_pos] != "\x1b"
        ):
            x_coord += wlen(newline[x_pos])
            x_pos += 1

The wlen method has some special casing for ctrl-z, but that is not taken into account when the length of a string is 1:

def wlen(s: str) -> int:
    if len(s) == 1:
        return str_width(s)
    length = sum(str_width(i) for i in s)
    # remove lengths of any escape sequences
    sequence = ANSI_ESCAPE_SEQUENCE.findall(s)
    ctrl_z_cnt = s.count('\x1a')
    return length - sum(len(i) for i in sequence) + ctrl_z_cnt

The following diff avoids the crash:

diff --git a/Lib/_pyrepl/utils.py b/Lib/_pyrepl/utils.py
index 0f36083b6ff..4651717bd7e 100644
--- a/Lib/_pyrepl/utils.py
+++ b/Lib/_pyrepl/utils.py
@@ -16,7 +16,7 @@ def str_width(c: str) -> int:


 def wlen(s: str) -> int:
-    if len(s) == 1:
+    if len(s) == 1 and s != '\x1a':
         return str_width(s)
     length = sum(str_width(i) for i in s)
     # remove lengths of any escape sequences

[eendebakpt]

@pablogsal @godlygeek The optimization in wlen for the single character case was added in #120253. Should the special casing for CTRL-Z be taken into account there?

[pablogsal]

Should the special casing for CTRL-Z be taken into account there?

I think so. @eendebakpt do you mind creating a PR with the fix?

[eendebakpt]

@pablogsal PR is already open: #126650