PEP 667 + PEP 709 segfaults from accessing closure variables bound by any inlined comprehensions

[bswck]

Bug report

Bug description:

The segfault described here is related to PEP-667 and PEP-709.

I think I experienced a couple of similar but not same segfaults, and this here is just one of a few of them. I might be wrong though.
I'd like to start working on fixing this in order to possibly find more unhappy paths.

Reproduction

Here are a bunch of reproductions:

def f():
    lambda: k
    k = [k for k in [0] if locals()]

f()

def f():
    lambda: k
    k = [sys._getframe(0).f_locals["k"] for k in [0]]

f()

def f():
    lambda: k
    k = [eval("") for k in [0]]

f()

@alexmojaki also found

def f():
    lambda: k
    k = 1
    [locals() for k in [0]]

f()

def f():
    lambda: k
    k: int
    [locals() for k in [0]]

f()

This one shows the problem the best:

def fetch_locals_proxy():
    # anything that fetches PEP 667 FrameLocalsProxy contents
    # causes the crash
    sys._getframe(1).f_locals.items()

def f():
    lambda: k
    k = 1
    [fetch_locals_proxy() for k in [0]]

f()

Traceback

Fatal Python error: Segmentation fault

Current thread 0x00007fd059596740 (most recent call first):
  File "/tmp/repro/dump-1.py", line 9 in fetch_locals_proxy
  File "/tmp/repro/dump-1.py", line 14 in f
  File "/tmp/repro/dump-1.py", line 17 in <module>
Segmentation fault (core dumped)

Conclusions

1. There must be a statement that allows to create a closure for a variable, for instance k, in a scope ("target scope"). It's either k = [...] or k: int in the repros.

2. There must be an inlined comprehension that binds k as target (e.g. k = 1; [x for x in [0] if locals()] doesn't cause segfault).

3. After binding the variable during the evaluation of the inlined comprehension, something must fetch the contents of the PEP 667 proxy of the target scope.

4. The target scope must be any optimized scope (not confirmed for generator expressions), since only those involve PEP 667 FrameLocalsProxy. That's the reason why the crash doesn't happen at a module/class level:

   class Spam:
       lambda: k
       k = [k for k in [0] if locals()]

   or on Python <3.13, before PEP 667 had been introduced.

5. The iterable traversed in the comprehension isn't special (the crash doesn't happen for certain objects, e.g. [None]¹, (...,), but happens for others, e.g. [0], range(1)). Not confirmed for every platform.

Thanks @Eclips4 for hints to finding the best reproduction. Thanks @trag1c for reporting the problem after accidentally running into it in a real-life use case.

I'm delighted to accept any help or guidance (cc @JelleZijlstra @carljm), as I want to author the patch to fix this issue and learn something in the process. Thanks!

CPython versions tested on:

3.13, 3.14, CPython main branch

Operating systems tested on:

Linux

Footnotes

1. Const lists (from displays) are compiled into tuples under the hood before the loop begins, so [None] here is equivalent to (None,). ↩

[ZeroIntensity]

On a debug build, an assertion fails for all three reproducers:

python: Objects/frameobject.c:40: framelocalsproxy_getval: Assertion `PyCell_Check(value)' failed.