Make the specializing interpreter thread-safe in --disable-gil builds

[swtaarrs]

Feature or enhancement

Proposal:

In free-threaded builds, the specializing adaptive interpreter needs to be made thread-safe. We should start with a small PR to simply disable it in free-threaded builds, which will be correct but will incur a performance penalty. Then we can work out how to properly support specialization in a free-threaded build.

These two commits from Sam's nogil-3.12 branch can serve as inspiration:

1. specialize: make specialization thread-safe
   
2. specialize: optimize for single-threaded programs

There are two primary concerns to balance while implementing this functionality on main:

1. Runtime overhead: There should be no performance impact on normal builds, and minimal performance impact on single-threaded code running in free-threaded builds.
2. Reducing code duplication/divergence: We should come up with a design that is minimally disruptive to ongoing work on the specializing interpreter. It should be easy for other devs to keep the free-threaded build working without having to know too much about it.

Has this already been discussed elsewhere?

I have already discussed this feature proposal on Discourse

Links to previous discussion of this feature:

• https://peps.python.org/pep-0703/
• PEP 703 -- Making the Global Interpreter Lock Optional in CPython #108219

Specialization Families

Tasks

Beta Give feedback

1. BINARY_OP
2. BINARY_SUBSCR - @corona10
3. CALL - @mpage
4. CALL_KW - @mpage
5. COMPARE_OP - @Yhg1s
6. CONTAINS_OP - @corona10
7. FOR_ITER - @Yhg1s
8. LOAD_ATTR - @mpage
9. LOAD_CONST
10. LOAD_GLOBAL - @mpage
11. LOAD_SUPER_ATTR
12. RESUME
13. SEND
14. STORE_ATTR
15. STORE_SUBSCR - @corona10
16. TO_BOOL - @corona10
17. UNPACK_SEQUENCE - @Eclips4

Linked PRs

• gh-115999: Disable the specializing adaptive interpreter in free-threaded builds #116013
• gh-115999: Implement thread-local bytecode and enable specialization for BINARY_OP #123926
• gh-115999: Refactor LOAD_GLOBAL specializations to avoid reloading {globals, builtins} keys #124953
• gh-115999: Stop the world when invalidating function versions #124997
• gh-115999: Add free-threaded specialization for COMPARE_OP #126410
• gh-115999: Introduce helpers for (un)specializing instructions #126414
• gh-115999: Fix gdb support for libpython.so after thread-local bytecode change #126440
• gh-115999: Add free-threaded specialization for CONTAINS_OP #126450
• gh-115999: Move specializer test from test_dis to test_opcache #126498
• gh-115999: Restore SET_OPCODE_OR_RETURN removed in GH-126414 #126515
• gh-115999: Add free-threaded specialization for UNPACK_SEQUENCE #126600
• gh-115999: Specialize LOAD_GLOBAL in free-threaded builds #126607
• gh-115999: Add free-threaded specialization for TO_BOOL #126616

[brandtbucher]

(subscribing myself)

[swtaarrs]

This is now a performance (rather than correctness) issue for free-threaded builds, so I'm going to focus on more time-sensitive issues for a while.

[corona10]

@swtaarrs Out of curiosity, is there any progress or plan for this issue?

[Fidget-Spinner]

@corona10 I'm planning to work on this after I get the deferred reference stack in. However, there are no concrete plans as of now. I'm really happy for you or anyone else to propose a design for the specializing interpreter with free-threaded safety!

[corona10]

@Fidget-Spinner cc @swtaarrs
Nice. I was also thinking about how to make it thread-safe in a seamless way since I agree with @swtaarrs.
But there is no good idea yet to solve the issue right now since I am not in a full-time position for this task :)
So it will be happy to see you have a good plan.
(I am curious that we can make them per-thread mechanism...)

By the way, in the short term, can we enable the specializer to be used only for the main thread if we can not solve the issue before 3.13 is released?
We can easily track the performance degradation between the default build because most of pyperformance benchmark are based on a single thread :)

[Fidget-Spinner]

@corona10 for 3.13, I think generally we're focusing on scalability across multicore rather than single-threaded perf for 3.13. It's a bit too near to feature freeze for me to feel safe re-enabling specialization at this point. There are a lot of unsolved problems still even with specialization only on the main thread. Consider the following:

Two threads sharing the same code object, A and B. A is main thread.
Thread B is in LOAD_ATTR_METHOD_WITH_VALUES's action (after guards, it is in the middle of loading from a method)
Thread A is in LOAD_ATTR_METHOD_WITH_VALUES's guard, but then deopts, meaning the method reference is now most likely dead/invalid.
Thread B loads from LOAD_ATTR_METHOD_WITH_VALUE's method, it is now holding a dangling pointer.
Thread B pushes dangling pointer to the stack. Everything crashes.

I'm reading a few papers to get some inspiration and also looking at how CRuby and other runtimes deal with this. Will post back when I have an actual plan.

[serhiy-storchaka]

#126414 broke the main branch.

Python/specialize.c: In function ‘_Py_Specialize_ContainsOp’:
Python/specialize.c:2801:5: error: implicit declaration of function ‘SET_OPCODE_OR_RETURN’ [-Werror=implicit-function-declaration]
 2801 |     SET_OPCODE_OR_RETURN(instr, CONTAINS_OP);
      |     ^~~~~~~~~~~~~~~~~~~~
cc1: some warnings being treated as errors
make: *** [Makefile:3116: Python/specialize.o] Помилка 1
make: *** Очікування завершення завдань...

[mpage]

#126414 broke the main branch.

Python/specialize.c: In function ‘_Py_Specialize_ContainsOp’:
Python/specialize.c:2801:5: error: implicit declaration of function ‘SET_OPCODE_OR_RETURN’ [-Werror=implicit-function-declaration]
 2801 |     SET_OPCODE_OR_RETURN(instr, CONTAINS_OP);
      |     ^~~~~~~~~~~~~~~~~~~~
cc1: some warnings being treated as errors
make: *** [Makefile:3116: Python/specialize.o] Помилка 1
make: *** Очікування завершення завдань...

Ugh sorry. #126414 raced with #126450.

[corona10]

I am working on TO_BOOL and BINARY_SUBSCR