Closed
Description
Bug report
Bug description:
Part of #127945.
ctypes C data objects have an internal pointer for what they're looking at (b_ptr). This field itself is generally fine to read non-atomically, because ctypes objects don't tend to overwrite the pointer that they're pointing to, but reading and writing the pointer's contents (such as via memcpy) isn't thread safe. This can be seen primarily with arrays:
from threading import Thread
import ctypes
buffer = (ctypes.c_char_p * 10)()
def main():
for i in range(100):
buffer.value = b"hello"
buffer[1] = b"j"
threads = [Thread(target=main) for _ in range(100)]
for thread in threads:
thread.start()There's really only two options here, because the lockless _Py_atomic APIs can't be used for allocations of arbitrary sizes: add a critical section around all functions touching b_ptr, or just add a PyMutex around it.
I think that a mutex is the right way to go here:
- There's no chance of re-entrancy with
memcpy. - AC doesn't work for some type slots (e.g.
__call__), so they can't get wrapped with@critical_section. We'd need ugly wrapper functions.
CPython versions tested on:
CPython main branch
Operating systems tested on:
Linux
Linked PRs
- gh-128182: Add per-object memory access synchronization to
ctypes#128490 - gh-128182: add critical section to _ctypes.Simple getters and setters #132081
- gh-128182: add critical section to
_ctypes.PyCDatagetters and setters #132082 - gh-128182: Switch
ctypesto solely critical sections #132133 - gh-128182: add critical sections to ctypes arrays getters and setters #132152