Skip to content

_Py_MergeZeroLocalRefcount should not set ob_tid to zero in fast-path dealloc #121794

Closed
@colesbury

Description

@colesbury

Bug report

The _Py_MergeZeroLocalRefcount function is called when the local refcount field reaches zero. We generally maintain the invariant 1 that ob_tid == 0 implies that the refcount fields are merged (i.e., ob_ref_shared flags are _Py_REF_MERGED) and vice versa.

The current implementation breaks the invariant by setting ob_tid to zero when the refcount fields aren't merged. Typically, this isn't a problem because:

  • Most commonly, the object is deallocated so the values do not matter
  • If the object is resurrected in subtype_dealloc (e.g., for a finalizer), we use Py_SET_REFCNT, which will mark the fields as merged, restoring the invariant

However, if resurrection is done slightly differently, such as by Py_INCREF(), then things can break in very strange ways:

  • The GC may restore ob_tid from the allocator (because it's not merged), but ob_ref_local is still zero. The next Py_DECREF then leads to a "negative refcount" error.

Summary

We should maintain the invariant that ob_tid == 0 <=> _Py_REF_IS_MERGED() and check it with assertions when possible.

cpython/Objects/object.c

Lines 373 to 398 in 8303d32

void
_Py_MergeZeroLocalRefcount(PyObject *op)
{
assert(op->ob_ref_local == 0);
_Py_atomic_store_uintptr_relaxed(&op->ob_tid, 0);
Py_ssize_t shared = _Py_atomic_load_ssize_acquire(&op->ob_ref_shared);
if (shared == 0) {
// Fast-path: shared refcount is zero (including flags)
_Py_Dealloc(op);
return;
}
// Slow-path: atomically set the flags (low two bits) to _Py_REF_MERGED.
Py_ssize_t new_shared;
do {
new_shared = (shared & ~_Py_REF_SHARED_FLAG_MASK) | _Py_REF_MERGED;
} while (!_Py_atomic_compare_exchange_ssize(&op->ob_ref_shared,
&shared, new_shared));
if (new_shared == _Py_REF_MERGED) {
// i.e., the shared refcount is zero (only the flags are set) so we
// deallocate the object.
_Py_Dealloc(op);
}
}

Originally reported by @albanD

Linked PRs

Footnotes

  1. There are a few places where we deliberately re-use ob_tid for other purposes, such as the trashcan mechanism and during GC, but these objects are not visible to other parts of the program.

Metadata

Metadata

Assignees

No one assigned

    Labels

    3.13bugs and security fixes3.14bugs and security fixestopic-free-threadingtype-bugAn unexpected behavior, bug, or error

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions