Incomplete stack traces when throwing into a generator chain that ends in a custom generator

[jbower-fb]

Bug report

Bug description:

bpo-29590 identified an issue where only the head of a chain of generators was included in stack-traces when processing a thrown-in exception. I think this issue still exists but only in the case where the head of a chain of generators is a custom generator.

Consider the following example in which a custom generator's throw() method does not see its predecessor in the yield from chain (the g() generator), but the builtin in generator does:

$ cat test.py
import sys

class UserGen:
  def throw(self, *args):
    print("Traceback from UserGen:")
    f = sys._getframe()
    while f:
      print(f)
      f = f.f_back
  def __iter__(self):
    return self
  def __next__(self):
    return 42

def real_gen():
  yield 43

def g(target):
  yield from target

gg = g(UserGen())
gg.send(None)
gg.throw(RuntimeError)

print()

gg = g(real_gen())
gg.send(None)
gg.throw(RuntimeError)


$ python3 test.py
Traceback from UserGen:
<frame at 0x10273e230, file 'test.py', line 8, code throw>
<frame at 0x10270d440, file 'test.py', line 26, code <module>>

Traceback (most recent call last):
  File "test.py", line 32, in <module>
    gg.throw(RuntimeError)
  File "test.py", line 22, in g
    yield from target
  File "test.py", line 18, in real_gen
    yield 43
RuntimeError

The fix for bpo-29590 is in #19896 and appears to specifically only link frames together for chains of builtin generators only. I suspect this was just an oversight rather than a deliberate choice though.

I attach a proposed fix in a PR.

CPython versions tested on:

CPython main branch

Operating systems tested on:

No response

Linked PRs

• gh-126091: Always link generator frames when propagating a thrown-in exception through a yield-from chain #126092

[jbower-fb]

cc @1st1 , @cjerdonek , @markshannon , and @njsmith who were involved in the first iteration of this issue.

[ZeroIntensity]

Is this done? Or was this supposed to get backported?