gh-120906: Support arbitrary hashable keys in FrameLocalsProxy

[encukou]

Make FrameLocalsProxy support arbitrary keys for lookup and update.

The given key needs to be hashable. It's a mapping, after all.
For the linear scan through the "real" locals:

• If the given key is an interned exact string, we use pointer comparison
• Otherwise, we fall back to generic PyObject_RichCompareBool
  If there's no match, we know it matches none of the "real" locals,
  and the read/write functions fall back to the "extras" dict.

Add tests, including tests for the mapping protocol (with selected exceptions).

• Issue: FrameLocalsProxy is stricter than dict about what constitutes a match #120906

[gaogaotiantian]

So for the first glance, there are two questions:

1. What if the key equals to a unicode string, but has a different hash? I know it's not "supposed" to be, but that could happen and this implementation has a different behavior than a real dict.
2. The separate loop was intentional - do a quick pointer equality check first, then do the slow check. It will save us some time. I think we should keep that.

I did no go through the whole change because I don't believe we've reached to the point where we agree how to do this.

[encukou]

1. I don't think we need to match dict in this anomalous case.
   The dict behaviour itself depends on internal implementation details one shouldn't rely on.

   However, we can use the hash to skip PyObject_RichCompareBool calls, which seems beneficial on its own.
   (But it doesn't match dict behaviour exactly. Dict doesn't use the full hash; how much of the hash it uses depends on the internal table size.)

2. OK. (I'd be surprised if the speedup didn't matter in real-world use cases, so I went for simpler code before.)

[ncoghlan]

General approach LGTM, some suggestions inline regarding test case details.

[gaogaotiantian]

If you removed found_key check here, maybe just return -1 if the condition does not meet. Otherwise it will go through the loop again and check something we already knew.

[gaogaotiantian]

The rest of the PR LGTM.

[encukou]

Right, thanks for the catch!

[encukou]

Could you check the change?
I'm still not too familiar with the internals of this mapping; I just found out (hopefully correctly) what the undocumented CO_FAST_HIDDEN does.

[ncoghlan]

The change looks correct to me (with the search flag restored, the CO_FAST_HIDDEN handling is unchanged from @gaogaotiantian's original PR).

I did suggest an explanatory comment on the early return, but that's definitely not a merge blocker. So even if @gaogaotiantian doesn't get time to take a look, I think this is good to go for rc1.

[ncoghlan]

Capturing my understanding of the CO_FAST_HIDDEN handling, since it's genuinely not clear:

• in Python 3.12, the iteration variables for inlined comprehensions are visible for reads while the comprehension(s) is(/are) running, but not writable (even with LocalsToFast shenanigans), and not visible outside the comprehension(s)
• in Python 3.13, all of those properties remain true even though sys._getframe().f_locals produces fast locals proxy instances during iteration (writes to hidden variables go into the frame's "extra locals" storage rather than modifying the comprehension iteration variable)

Future versions could revisit that design if debugger developers ask for it to change, but live editing loop variables seems sufficiently fraught with peril that I'll be genuinely surprised if anyone ever notices this small discrepancy between module level comprehensions and function level ones.

[encukou]

@gaogaotiantian and/or @markshannon, your review would be appreciated, so we can get this in 3.13.

[markshannon]

Looks good.

There is one test I'd like to added, though.
This little program now succeeds (it fails on main):

class Str(str):
    pass

def foo():
    sys._getframe().f_locals[Str("x")] = 2
    assert x == 2
    x = 1

foo()

but it doesn't seem like there is a test for this case (writing a fast local using a non-str key).
Could you add a test for this?

[encukou]

Will do in ~6 hours, feel free to beat me to it.

[encukou]

Test added. (There was one for writing a fast local using a non-str key, but not for writing a so far uninitialized local like your example, so I added that.)

Thank you for the reviews!

[miss-islington-app]

Thanks @encukou for the PR 🌮🎉.. I'm working now to backport this PR to: 3.13.
🐍🍒⛏🤖

[miss-islington-app]

Sorry, @encukou, I could not cleanly backport this to 3.13 due to a conflict.
Please backport using cherry_picker on command line.

cherry_picker 5912487938ac4b517209082ab9e6d2d3d0fb4f4d 3.13

[bedevere-app]

GH-122488 is a backport of this pull request to the 3.13 branch.