Don't use memmove/memcpy/memset for memory that can be scanned concurrently
https://bugs.webkit.org/show_bug.cgi?id=203228
<rdar://problem/56401852>
Reviewed by Robin Morisset.
JSTests:
- stress/torn-js-value-concurrent-collector.js: Added.
(foo):
Source/JavaScriptCore:
We had code inside various places of the runtime which would call into system
memcpy/memmove/memset when updating a live butterfly. This means that the
concurrent collector could be scanning such butterflies while a memcpy/memmove/memset
was running. Those functions don't guarantee anything about the minimum
alignment of the stores they do. And implementations for them frequently have
byte copy loops for low byte copy counts. This lead to us seeing torn JSValues
inside the concurrent collector during Array.prototype.splice. This patch
introduces new functions for doing memcpy/memmove/memset for data structures
which may be concurrently scanned. The loops are written using inline assembly
for gcc compatible compilers on 64 bit platforms. The inline assembly
ensures we never write to memory using instructions that store fewer
than 8 bytes. On other platforms, we just use a volatile pointer to
ensure the compiler doesn't turn the loop into a function call or a
series of stores which may be smaller than 8 bytes.
- CMakeLists.txt:
- JavaScriptCore.xcodeproj/project.pbxproj:
- heap/GCMemoryOperations.h: Added.
(JSC::gcSafeMemcpy):
(JSC::gcSafeMemmove):
(JSC::gcSafeZeroMemory):
- heap/Heap.h:
- runtime/ArrayConventions.cpp:
(JSC::clearArrayMemset):
- runtime/ArrayPrototype.cpp:
(JSC::copyElements):
- runtime/ButterflyInlines.h:
(JSC::Butterfly::tryCreate):
(JSC::Butterfly::createOrGrowPropertyStorage):
(JSC::Butterfly::growArrayRight):
(JSC::Butterfly::reallocArrayRightIfPossible):
(JSC::Butterfly::resizeArray):
(JSC::Butterfly::unshift):
(JSC::Butterfly::shift):
(JSC::JSArray::unshiftCountSlowCase):
(JSC::JSArray::appendMemcpy):
(JSC::JSArray::fastSlice):
(JSC::JSArray::shiftCountWithArrayStorage):
(JSC::JSArray::shiftCountWithAnyIndexingType):
(JSC::JSArray::unshiftCountWithArrayStorage):
(JSC::JSObject::constructConvertedArrayStorageWithoutCopyingElements):
(JSC::JSObject::convertFromCopyOnWrite):
(JSC::JSObject::shiftButterflyAfterFlattening):
- runtime/JSObject.h:
- runtime/RegExpMatchesArray.h:
(JSC::createRegExpMatchesArray):
(JSC::Structure::flattenDictionaryStructure):
