Changeset 225314 in webkit for trunk/Source/JavaScriptCore/runtime/VM.cpp

Timestamp:

Nov 29, 2017, 8:39:50 PM (8 years ago)

Author:

[email protected]

Message:

GC should support isoheaps
​https://bugs.webkit.org/show_bug.cgi?id=179288

Reviewed by Saam Barati.
Source/JavaScriptCore:

This expands the power of the Subspace API in JSC:

• Everything associated with describing the types of objects is now part of the HeapCellType class. We have different HeapCellTypes for different destruction strategies. Any Subspace can use any HeapCellType; these are orthogonal things.

• There are now two variants of Subspace: CompleteSubspace, which can allocate any size objects using any AlignedMemoryAllocator; and IsoSubspace, which can allocate just one size of object and uses a special virtual memory pool for that purpose. Like bmalloc's IsoHeap, IsoSubspace hoards virtual pages but releases the physical pages as part of the respective allocator's scavenging policy (the Scavenger in bmalloc for IsoHeap and the incremental sweep and full sweep in Riptide for IsoSubspace).

So far, this patch just puts subtypes of ExecutableBase in IsoSubspaces. If it works, we can use it
for more things.

This does not have any effect on JetStream (0.18% faster with p = 0.69).

• JavaScriptCore.xcodeproj/project.pbxproj:
• Sources.txt:
• bytecode/AccessCase.cpp:

(JSC::AccessCase::generateImpl):

• bytecode/ObjectAllocationProfileInlines.h:

(JSC::ObjectAllocationProfile::initializeProfile):

• dfg/DFGSpeculativeJIT.cpp:

(JSC::DFG::SpeculativeJIT::emitAllocateRawObject):
(JSC::DFG::SpeculativeJIT::compileMakeRope):
(JSC::DFG::SpeculativeJIT::compileAllocatePropertyStorage):
(JSC::DFG::SpeculativeJIT::compileReallocatePropertyStorage):

• dfg/DFGSpeculativeJIT64.cpp:

(JSC::DFG::SpeculativeJIT::compile):

• ftl/FTLAbstractHeapRepository.h:
• ftl/FTLLowerDFGToB3.cpp:

(JSC::FTL::DFG::LowerDFGToB3::compileMakeRope):
(JSC::FTL::DFG::LowerDFGToB3::compileMaterializeNewObject):
(JSC::FTL::DFG::LowerDFGToB3::allocatePropertyStorageWithSizeImpl):
(JSC::FTL::DFG::LowerDFGToB3::allocateObject):
(JSC::FTL::DFG::LowerDFGToB3::allocatorForSize):

• heap/AlignedMemoryAllocator.cpp:

(JSC::AlignedMemoryAllocator::registerAllocator):
(JSC::AlignedMemoryAllocator::registerSubspace):

• heap/AlignedMemoryAllocator.h:

(JSC::AlignedMemoryAllocator::firstAllocator const):

• heap/AllocationFailureMode.h: Added.
• heap/CompleteSubspace.cpp: Added.

(JSC::CompleteSubspace::CompleteSubspace):
(JSC::CompleteSubspace::~CompleteSubspace):
(JSC::CompleteSubspace::allocatorFor):
(JSC::CompleteSubspace::allocate):
(JSC::CompleteSubspace::allocateNonVirtual):
(JSC::CompleteSubspace::allocatorForSlow):
(JSC::CompleteSubspace::allocateSlow):
(JSC::CompleteSubspace::tryAllocateSlow):

• heap/CompleteSubspace.h: Added.

(JSC::CompleteSubspace::offsetOfAllocatorForSizeStep):
(JSC::CompleteSubspace::allocatorForSizeStep):
(JSC::CompleteSubspace::allocatorForNonVirtual):

• heap/HeapCellType.cpp: Added.

(JSC::HeapCellType::HeapCellType):
(JSC::HeapCellType::~HeapCellType):
(JSC::HeapCellType::finishSweep):
(JSC::HeapCellType::destroy):

• heap/HeapCellType.h: Added.

(JSC::HeapCellType::attributes const):

• heap/IsoAlignedMemoryAllocator.cpp: Added.

(JSC::IsoAlignedMemoryAllocator::IsoAlignedMemoryAllocator):
(JSC::IsoAlignedMemoryAllocator::~IsoAlignedMemoryAllocator):
(JSC::IsoAlignedMemoryAllocator::tryAllocateAlignedMemory):
(JSC::IsoAlignedMemoryAllocator::freeAlignedMemory):
(JSC::IsoAlignedMemoryAllocator::dump const):

• heap/IsoAlignedMemoryAllocator.h: Added.
• heap/IsoSubspace.cpp: Added.

(JSC::IsoSubspace::IsoSubspace):
(JSC::IsoSubspace::~IsoSubspace):
(JSC::IsoSubspace::allocatorFor):
(JSC::IsoSubspace::allocatorForNonVirtual):
(JSC::IsoSubspace::allocate):
(JSC::IsoSubspace::allocateNonVirtual):

• heap/IsoSubspace.h: Added.

(JSC::IsoSubspace::size const):

• heap/MarkedAllocator.cpp:

(JSC::MarkedAllocator::MarkedAllocator):
(JSC::MarkedAllocator::setSubspace):
(JSC::MarkedAllocator::allocateSlowCase):
(JSC::MarkedAllocator::tryAllocateSlowCase): Deleted.
(JSC::MarkedAllocator::allocateSlowCaseImpl): Deleted.

• heap/MarkedAllocator.h:

(JSC::MarkedAllocator::nextAllocatorInAlignedMemoryAllocator const):
(JSC::MarkedAllocator::setNextAllocatorInAlignedMemoryAllocator):

• heap/MarkedAllocatorInlines.h:

(JSC::MarkedAllocator::allocate):
(JSC::MarkedAllocator::tryAllocate): Deleted.

• heap/MarkedBlock.h:
• heap/MarkedBlockInlines.h:

(JSC::MarkedBlock::Handle::finishSweepKnowingHeapCellType):
(JSC::MarkedBlock::Handle::finishSweepKnowingSubspace): Deleted.

• heap/MarkedSpace.cpp:

(JSC::MarkedSpace::addMarkedAllocator):

• heap/MarkedSpace.h:
• heap/Subspace.cpp:

(JSC::Subspace::Subspace):
(JSC::Subspace::initialize):
(JSC::Subspace::finishSweep):
(JSC::Subspace::destroy):
(JSC::Subspace::prepareForAllocation):
(JSC::Subspace::findEmptyBlockToSteal):
(): Deleted.
(JSC::Subspace::allocate): Deleted.
(JSC::Subspace::tryAllocate): Deleted.
(JSC::Subspace::allocatorForSlow): Deleted.
(JSC::Subspace::allocateSlow): Deleted.
(JSC::Subspace::tryAllocateSlow): Deleted.
(JSC::Subspace::didAllocate): Deleted.

• heap/Subspace.h:

(JSC::Subspace::heapCellType const):
(JSC::Subspace::nextSubspaceInAlignedMemoryAllocator const):
(JSC::Subspace::setNextSubspaceInAlignedMemoryAllocator):
(JSC::Subspace::offsetOfAllocatorForSizeStep): Deleted.
(JSC::Subspace::allocatorForSizeStep): Deleted.
(JSC::Subspace::tryAllocatorFor): Deleted.
(JSC::Subspace::allocatorFor): Deleted.

• jit/AssemblyHelpers.h:

(JSC::AssemblyHelpers::emitAllocateJSObjectWithKnownSize):
(JSC::AssemblyHelpers::emitAllocateVariableSized):
(JSC::AssemblyHelpers::emitAllocateVariableSizedCell):

• jit/JITOpcodes.cpp:

(JSC::JIT::emit_op_new_object):

• runtime/ButterflyInlines.h:

(JSC::Butterfly::createUninitialized):
(JSC::Butterfly::tryCreate):
(JSC::Butterfly::growArrayRight):

• runtime/DirectArguments.cpp:

(JSC::DirectArguments::overrideThings):

• runtime/DirectArguments.h:

(JSC::DirectArguments::subspaceFor):

• runtime/DirectEvalExecutable.h:
• runtime/EvalExecutable.h:
• runtime/ExecutableBase.h:

(JSC::ExecutableBase::subspaceFor):

• runtime/FunctionExecutable.h:
• runtime/GenericArgumentsInlines.h:

(JSC::GenericArguments<Type>::initModifiedArgumentsDescriptor):

• runtime/HashMapImpl.h:

(JSC::HashMapBuffer::create):

• runtime/IndirectEvalExecutable.h:
• runtime/JSArray.cpp:

(JSC::JSArray::tryCreateUninitializedRestricted):
(JSC::JSArray::unshiftCountSlowCase):

• runtime/JSArray.h:

(JSC::JSArray::tryCreate):

• runtime/JSArrayBufferView.cpp:

(JSC::JSArrayBufferView::ConstructionContext::ConstructionContext):

• runtime/JSCell.h:

(JSC::subspaceFor):

• runtime/JSCellInlines.h:

(JSC::JSCell::subspaceFor):
(JSC::tryAllocateCellHelper):
(JSC::allocateCell):
(JSC::tryAllocateCell):

• runtime/JSDestructibleObject.h:

(JSC::JSDestructibleObject::subspaceFor):

• runtime/JSDestructibleObjectHeapCellType.cpp: Copied from Source/JavaScriptCore/runtime/JSDestructibleObjectSubspace.cpp.

(JSC::JSDestructibleObjectHeapCellType::JSDestructibleObjectHeapCellType):
(JSC::JSDestructibleObjectHeapCellType::~JSDestructibleObjectHeapCellType):
(JSC::JSDestructibleObjectHeapCellType::finishSweep):
(JSC::JSDestructibleObjectHeapCellType::destroy):
(JSC::JSDestructibleObjectSubspace::JSDestructibleObjectSubspace): Deleted.
(JSC::JSDestructibleObjectSubspace::~JSDestructibleObjectSubspace): Deleted.
(JSC::JSDestructibleObjectSubspace::finishSweep): Deleted.
(JSC::JSDestructibleObjectSubspace::destroy): Deleted.

• runtime/JSDestructibleObjectHeapCellType.h: Copied from Source/JavaScriptCore/runtime/JSDestructibleObjectSubspace.h.
• runtime/JSDestructibleObjectSubspace.cpp: Removed.
• runtime/JSDestructibleObjectSubspace.h: Removed.
• runtime/JSLexicalEnvironment.h:

(JSC::JSLexicalEnvironment::subspaceFor):

• runtime/JSSegmentedVariableObject.h:

(JSC::JSSegmentedVariableObject::subspaceFor):

• runtime/JSSegmentedVariableObjectHeapCellType.cpp: Copied from Source/JavaScriptCore/runtime/JSSegmentedVariableObjectSubspace.cpp.

(JSC::JSSegmentedVariableObjectHeapCellType::JSSegmentedVariableObjectHeapCellType):
(JSC::JSSegmentedVariableObjectHeapCellType::~JSSegmentedVariableObjectHeapCellType):
(JSC::JSSegmentedVariableObjectHeapCellType::finishSweep):
(JSC::JSSegmentedVariableObjectHeapCellType::destroy):
(JSC::JSSegmentedVariableObjectSubspace::JSSegmentedVariableObjectSubspace): Deleted.
(JSC::JSSegmentedVariableObjectSubspace::~JSSegmentedVariableObjectSubspace): Deleted.
(JSC::JSSegmentedVariableObjectSubspace::finishSweep): Deleted.
(JSC::JSSegmentedVariableObjectSubspace::destroy): Deleted.

• runtime/JSSegmentedVariableObjectHeapCellType.h: Copied from Source/JavaScriptCore/runtime/JSSegmentedVariableObjectSubspace.h.
• runtime/JSSegmentedVariableObjectSubspace.cpp: Removed.
• runtime/JSSegmentedVariableObjectSubspace.h: Removed.
• runtime/JSString.h:

(JSC::JSString::subspaceFor):

• runtime/JSStringHeapCellType.cpp: Copied from Source/JavaScriptCore/runtime/JSStringSubspace.cpp.

(JSC::JSStringHeapCellType::JSStringHeapCellType):
(JSC::JSStringHeapCellType::~JSStringHeapCellType):
(JSC::JSStringHeapCellType::finishSweep):
(JSC::JSStringHeapCellType::destroy):
(JSC::JSStringSubspace::JSStringSubspace): Deleted.
(JSC::JSStringSubspace::~JSStringSubspace): Deleted.
(JSC::JSStringSubspace::finishSweep): Deleted.
(JSC::JSStringSubspace::destroy): Deleted.

• runtime/JSStringHeapCellType.h: Copied from Source/JavaScriptCore/runtime/JSStringSubspace.h.
• runtime/JSStringSubspace.cpp: Removed.
• runtime/JSStringSubspace.h: Removed.
• runtime/ModuleProgramExecutable.h:
• runtime/NativeExecutable.h:
• runtime/ProgramExecutable.h:
• runtime/RegExpMatchesArray.h:

(JSC::tryCreateUninitializedRegExpMatchesArray):

• runtime/ScopedArguments.h:

(JSC::ScopedArguments::subspaceFor):

• runtime/VM.cpp:

(JSC::VM::VM):

• runtime/VM.h:

(JSC::VM::gigacageAuxiliarySpace):

• wasm/js/JSWebAssemblyCodeBlock.h:
• wasm/js/JSWebAssemblyCodeBlockHeapCellType.cpp: Copied from Source/JavaScriptCore/wasm/js/JSWebAssemblyCodeBlockSubspace.cpp.

(JSC::JSWebAssemblyCodeBlockHeapCellType::JSWebAssemblyCodeBlockHeapCellType):
(JSC::JSWebAssemblyCodeBlockHeapCellType::~JSWebAssemblyCodeBlockHeapCellType):
(JSC::JSWebAssemblyCodeBlockHeapCellType::finishSweep):
(JSC::JSWebAssemblyCodeBlockHeapCellType::destroy):
(JSC::JSWebAssemblyCodeBlockSubspace::JSWebAssemblyCodeBlockSubspace): Deleted.
(JSC::JSWebAssemblyCodeBlockSubspace::~JSWebAssemblyCodeBlockSubspace): Deleted.
(JSC::JSWebAssemblyCodeBlockSubspace::finishSweep): Deleted.
(JSC::JSWebAssemblyCodeBlockSubspace::destroy): Deleted.

• wasm/js/JSWebAssemblyCodeBlockHeapCellType.h: Copied from Source/JavaScriptCore/wasm/js/JSWebAssemblyCodeBlockSubspace.h.
• wasm/js/JSWebAssemblyCodeBlockSubspace.cpp: Removed.
• wasm/js/JSWebAssemblyCodeBlockSubspace.h: Removed.
• wasm/js/JSWebAssemblyMemory.h:

(JSC::JSWebAssemblyMemory::subspaceFor):

Source/WebCore:

No new tests because no new behavior.

Adopting changes in JSC Subspace API.

• ForwardingHeaders/runtime/JSDestructibleObjectHeapCellType.h: Added.
• ForwardingHeaders/runtime/JSSegmentedVariableObjectHeapCellType.h: Added.
• bindings/js/JSDOMWrapper.cpp:

(WebCore::outputConstraintSubspaceFor):
(WebCore::globalObjectOutputConstraintSubspaceFor):

• bindings/js/JSDOMWrapper.h:
• bindings/js/WebCoreJSClientData.cpp:

(WebCore::JSVMClientData::JSVMClientData):

• bindings/js/WebCoreJSClientData.h:

(WebCore::JSVMClientData::outputConstraintSpace):
(WebCore::JSVMClientData::globalObjectOutputConstraintSpace):

• bindings/scripts/CodeGeneratorJS.pm:

(GenerateHeader):

Source/WTF:

One of my favorite data structures in the GC is a singly-linked list that knows its tail, so that
things get added to it at the end rather that at the beginning. In this patch, I use this to put
the same node on multiple lists, which our existing linked list templates also don't support.

This adds a new linked list that does those things:

• It supports append(). It could also support prepend(), but currently there is no need for it.

• It supports nodes that are on multiple lists. The GC uses std::mem_fn() to create a lambda that the list uses to set next.

• WTF.xcodeproj/project.pbxproj:
• wtf/SinglyLinkedListWithTail.h: Added.

(WTF::SinglyLinkedListWithTail::isEmpty const):
(WTF::SinglyLinkedListWithTail::append):
(WTF::SinglyLinkedListWithTail::first const):
(WTF::SinglyLinkedListWithTail::last const):

File:

• trunk/Source/JavaScriptCore/runtime/VM.cpp (modified) (9 diffs)

trunk/Source/JavaScriptCore/runtime/VM.cpp

@@ -40 +40 @@
 #include "CustomGetterSetter.h"
 #include "DFGWorklist.h"
+#include "DirectEvalExecutable.h"
 #include "Disassembler.h"
 #include "ErrorInstance.h"
@@ -48 +49 @@
 #include "FunctionCodeBlock.h"
 #include "FunctionConstructor.h"
+#include "FunctionExecutable.h"
 #include "GCActivityCallback.h"
 #include "GetterSetter.h"
@@ -58 +60 @@
 #include "Identifier.h"
 #include "IncrementalSweeper.h"
+#include "IndirectEvalExecutable.h"
 #include "InferredTypeTable.h"
 #include "Interpreter.h"
@@ -65 +68 @@
 #include "JSArray.h"
 #include "JSCInlines.h"
+#include "JSDestructibleObjectHeapCellType.h"
 #include "JSFixedArray.h"
 #include "JSFunction.h"
@@ -74 +78 @@
 #include "JSPromiseDeferred.h"
 #include "JSPropertyNameEnumerator.h"
+#include "JSSegmentedVariableObjectHeapCellType.h"
 #include "JSScriptFetchParameters.h"
 #include "JSScriptFetcher.h"
@@ -79 +84 @@
 #include "JSSetIterator.h"
 #include "JSSourceCode.h"
+#include "JSStringHeapCellType.h"
 #include "JSTemplateRegistryKey.h"
 #include "JSWebAssembly.h"
+#include "JSWebAssemblyCodeBlockHeapCellType.h"
 #include "JSWithScope.h"
 #include "LLIntData.h"
@@ -87 +94 @@
 #include "MinimumReservedZoneSize.h"
 #include "ModuleProgramCodeBlock.h"
+#include "ModuleProgramExecutable.h"
+#include "NativeExecutable.h"
 #include "NativeStdFunctionCell.h"
 #include "Nodes.h"
@@ -92 +101 @@
 #include "ProfilerDatabase.h"
 #include "ProgramCodeBlock.h"
+#include "ProgramExecutable.h"
 #include "PromiseDeferredTimer.h"
 #include "PropertyMapHashTable.h"
@@ -172 +182 @@
     , primitiveGigacageAllocator(std::make_unique<GigacageAlignedMemoryAllocator>(Gigacage::Primitive))
     , jsValueGigacageAllocator(std::make_unique<GigacageAlignedMemoryAllocator>(Gigacage::JSValue))
-    , primitiveGigacageAuxiliarySpace("Primitive Gigacage Auxiliary", heap, AllocatorAttributes(DoesNotNeedDestruction, HeapCell::Auxiliary), primitiveGigacageAllocator.get())
-    , jsValueGigacageAuxiliarySpace("JSValue Gigacage Auxiliary", heap, AllocatorAttributes(DoesNotNeedDestruction, HeapCell::Auxiliary), jsValueGigacageAllocator.get())
-    , cellSpace("JSCell", heap, AllocatorAttributes(DoesNotNeedDestruction, HeapCell::JSCell), fastMallocAllocator.get())
-    , jsValueGigacageCellSpace("JSValue Gigacage JSCell", heap, AllocatorAttributes(DoesNotNeedDestruction, HeapCell::JSCell), jsValueGigacageAllocator.get())
-    , destructibleCellSpace("Destructible JSCell", heap, AllocatorAttributes(NeedsDestruction, HeapCell::JSCell), fastMallocAllocator.get())
-    , stringSpace("JSString", heap, fastMallocAllocator.get())
-    , destructibleObjectSpace("JSDestructibleObject", heap, fastMallocAllocator.get())
-    , eagerlySweptDestructibleObjectSpace("Eagerly Swept JSDestructibleObject", heap, fastMallocAllocator.get())
-    , segmentedVariableObjectSpace("JSSegmentedVariableObjectSpace", heap, fastMallocAllocator.get())
+    , auxiliaryHeapCellType(std::make_unique<HeapCellType>(AllocatorAttributes(DoesNotNeedDestruction, HeapCell::Auxiliary)))
+    , cellHeapCellType(std::make_unique<HeapCellType>(AllocatorAttributes(DoesNotNeedDestruction, HeapCell::JSCell)))
+    , destructibleCellHeapCellType(std::make_unique<HeapCellType>(AllocatorAttributes(NeedsDestruction, HeapCell::JSCell)))
+    , stringHeapCellType(std::make_unique<JSStringHeapCellType>())
+    , destructibleObjectHeapCellType(std::make_unique<JSDestructibleObjectHeapCellType>())
+    , segmentedVariableObjectHeapCellType(std::make_unique<JSSegmentedVariableObjectHeapCellType>())
 #if ENABLE(WEBASSEMBLY)
-    , webAssemblyCodeBlockSpace("JSWebAssemblyCodeBlockSpace", heap, fastMallocAllocator.get())
-#endif
+    , webAssemblyCodeBlockHeapCellType(std::make_unique<JSWebAssemblyCodeBlockHeapCellType>())
+#endif
+    , primitiveGigacageAuxiliarySpace("Primitive Gigacage Auxiliary", heap, auxiliaryHeapCellType.get(), primitiveGigacageAllocator.get())
+    , jsValueGigacageAuxiliarySpace("JSValue Gigacage Auxiliary", heap, auxiliaryHeapCellType.get(), jsValueGigacageAllocator.get())
+    , cellSpace("JSCell", heap, cellHeapCellType.get(), fastMallocAllocator.get())
+    , jsValueGigacageCellSpace("JSValue Gigacage JSCell", heap, cellHeapCellType.get(), jsValueGigacageAllocator.get())
+    , destructibleCellSpace("Destructible JSCell", heap, destructibleCellHeapCellType.get(), fastMallocAllocator.get())
+    , stringSpace("JSString", heap, stringHeapCellType.get(), fastMallocAllocator.get())
+    , destructibleObjectSpace("JSDestructibleObject", heap, destructibleObjectHeapCellType.get(), fastMallocAllocator.get())
+    , eagerlySweptDestructibleObjectSpace("Eagerly Swept JSDestructibleObject", heap, destructibleObjectHeapCellType.get(), fastMallocAllocator.get())
+    , segmentedVariableObjectSpace("JSSegmentedVariableObjectSpace", heap, segmentedVariableObjectHeapCellType.get(), fastMallocAllocator.get())
+#if ENABLE(WEBASSEMBLY)
+    , webAssemblyCodeBlockSpace("JSWebAssemblyCodeBlockSpace", heap, webAssemblyCodeBlockHeapCellType.get(), fastMallocAllocator.get())
+#endif
+    , nativeExecutableSpace ISO_SUBSPACE_INIT(heap, destructibleCellHeapCellType.get(), NativeExecutable)
+    , directEvalExecutableSpace ISO_SUBSPACE_INIT(heap, destructibleCellHeapCellType.get(), DirectEvalExecutable)
+    , indirectEvalExecutableSpace ISO_SUBSPACE_INIT(heap, destructibleCellHeapCellType.get(), IndirectEvalExecutable)
+    , functionExecutableSpace ISO_SUBSPACE_INIT(heap, destructibleCellHeapCellType.get(), FunctionExecutable)
+    , moduleProgramExecutableSpace ISO_SUBSPACE_INIT(heap, destructibleCellHeapCellType.get(), ModuleProgramExecutable)
+    , programExecutableSpace ISO_SUBSPACE_INIT(heap, destructibleCellHeapCellType.get(), ProgramExecutable)
     , vmType(vmType)
     , clientData(0)