Ignore:
Timestamp:
Aug 7, 2017, 2:31:49 PM (8 years ago)
Author:
[email protected]
Message:

Primitive auxiliaries and JSValue auxiliaries should have separate gigacages
​https://bugs.webkit.org/show_bug.cgi?id=174919

Reviewed by Keith Miller.
Source/bmalloc:


This introduces two kinds of Gigacage, Primitive and JSValue. This translates to two kinds of
HeapKind, PrimitiveGigacage and JSValueGigacage.

The new support functionality required turning Inline.h into BInline.h, and INLINE into BINLINE, and
NO_INLINE into BNO_INLINE.

  • bmalloc.xcodeproj/project.pbxproj:
  • bmalloc/Allocator.cpp:

(bmalloc::Allocator::refillAllocatorSlowCase):
(bmalloc::Allocator::refillAllocator):
(bmalloc::Allocator::allocateLarge):
(bmalloc::Allocator::allocateLogSizeClass):

  • bmalloc/AsyncTask.h:
  • bmalloc/BInline.h: Copied from Source/bmalloc/bmalloc/Inline.h.
  • bmalloc/Cache.cpp:

(bmalloc::Cache::tryAllocateSlowCaseNullCache):
(bmalloc::Cache::allocateSlowCaseNullCache):
(bmalloc::Cache::deallocateSlowCaseNullCache):
(bmalloc::Cache::reallocateSlowCaseNullCache):

  • bmalloc/Deallocator.cpp:
  • bmalloc/Gigacage.cpp:

(Gigacage::PrimitiveDisableCallbacks::PrimitiveDisableCallbacks):
(Gigacage::ensureGigacage):
(Gigacage::disablePrimitiveGigacage):
(Gigacage::addPrimitiveDisableCallback):
(Gigacage::removePrimitiveDisableCallback):
(Gigacage::Callbacks::Callbacks): Deleted.
(Gigacage::disableGigacage): Deleted.
(Gigacage::addDisableCallback): Deleted.
(Gigacage::removeDisableCallback): Deleted.

  • bmalloc/Gigacage.h:

(Gigacage::name):
(Gigacage::basePtr):
(Gigacage::forEachKind):
(Gigacage::caged):
(Gigacage::isCaged):

  • bmalloc/Heap.cpp:

(bmalloc::Heap::Heap):
(bmalloc::Heap::usingGigacage):
(bmalloc::Heap::gigacageBasePtr):

  • bmalloc/Heap.h:
  • bmalloc/HeapKind.h:

(bmalloc::isGigacage):
(bmalloc::gigacageKind):
(bmalloc::heapKind):

  • bmalloc/Inline.h: Removed.
  • bmalloc/Map.h:
  • bmalloc/PerProcess.h:

(bmalloc::PerProcess<T>::getFastCase):
(bmalloc::PerProcess<T>::get):
(bmalloc::PerProcess<T>::getSlowCase):

  • bmalloc/PerThread.h:

(bmalloc::PerThread<T>::getFastCase):

  • bmalloc/Vector.h:

(bmalloc::Vector<T>::push):
(bmalloc::Vector<T>::shrinkCapacity):
(bmalloc::Vector<T>::growCapacity):

Source/JavaScriptCore:


This adapts JSC to there being two gigacages.

To make matters simpler, this turns AlignedMemoryAllocators into per-VM instances rather than
singletons. I don't think we were gaining anything by making them be singletons.

This makes it easy to teach GigacageAlignedMemoryAllocator that there are multiple kinds of
gigacages. We'll have one of those allocators per cage.

From there, this change teaches everyone who previously knew about cages that there are two cages.
This means having to specify either Gigacage::Primitive or Gigacage::JSValue. In most places, this is
easy: typed arrays are Primitive and butterflies are JSValue. But there are a few places where it's
not so obvious, so this change introduces some helpers to make it easy to define what cage you want
to use in one place and refer to it abstractly. We do this in DirectArguments and GenericArguments.h

A lot of the magic of this change is due to CagedBarrierPtr, which combines AuxiliaryBarrier and
CagedPtr. This removes one layer of "get()" calls from a bunch of places.

(JSC::AccessCase::generateImpl):

  • dfg/DFGSpeculativeJIT.cpp:

(JSC::DFG::SpeculativeJIT::emitAllocateRawObject):
(JSC::DFG::SpeculativeJIT::compileAllocatePropertyStorage):
(JSC::DFG::SpeculativeJIT::compileReallocatePropertyStorage):
(JSC::DFG::SpeculativeJIT::compileNewTypedArray):
(JSC::DFG::SpeculativeJIT::emitAllocateButterfly):

  • ftl/FTLLowerDFGToB3.cpp:

(JSC::FTL::DFG::LowerDFGToB3::compileGetButterfly):
(JSC::FTL::DFG::LowerDFGToB3::compileGetIndexedPropertyStorage):
(JSC::FTL::DFG::LowerDFGToB3::compileNewTypedArray):
(JSC::FTL::DFG::LowerDFGToB3::compileGetDirectPname):
(JSC::FTL::DFG::LowerDFGToB3::compileMaterializeNewObject):
(JSC::FTL::DFG::LowerDFGToB3::allocatePropertyStorageWithSizeImpl):
(JSC::FTL::DFG::LowerDFGToB3::allocateJSArray):
(JSC::FTL::DFG::LowerDFGToB3::caged):

  • heap/FastMallocAlignedMemoryAllocator.cpp:

(JSC::FastMallocAlignedMemoryAllocator::instance): Deleted.

  • heap/FastMallocAlignedMemoryAllocator.h:
  • heap/GigacageAlignedMemoryAllocator.cpp:

(JSC::GigacageAlignedMemoryAllocator::GigacageAlignedMemoryAllocator):
(JSC::GigacageAlignedMemoryAllocator::tryAllocateAlignedMemory):
(JSC::GigacageAlignedMemoryAllocator::freeAlignedMemory):
(JSC::GigacageAlignedMemoryAllocator::dump const):
(JSC::GigacageAlignedMemoryAllocator::instance): Deleted.

  • heap/GigacageAlignedMemoryAllocator.h:
  • jsc.cpp:

(primitiveGigacageDisabled):
(jscmain):
(gigacageDisabled): Deleted.

  • llint/LowLevelInterpreter64.asm:
  • runtime/ArrayBuffer.cpp:

(JSC::ArrayBufferContents::tryAllocate):
(JSC::ArrayBuffer::createAdopted):
(JSC::ArrayBuffer::createFromBytes):

  • runtime/AuxiliaryBarrier.h:
  • runtime/ButterflyInlines.h:

(JSC::Butterfly::createUninitialized):
(JSC::Butterfly::tryCreate):
(JSC::Butterfly::growArrayRight):

  • runtime/CagedBarrierPtr.h: Added.

(JSC::CagedBarrierPtr::CagedBarrierPtr):
(JSC::CagedBarrierPtr::clear):
(JSC::CagedBarrierPtr::set):
(JSC::CagedBarrierPtr::get const):
(JSC::CagedBarrierPtr::getMayBeNull const):
(JSC::CagedBarrierPtr::operator== const):
(JSC::CagedBarrierPtr::operator!= const):
(JSC::CagedBarrierPtr::operator bool const):
(JSC::CagedBarrierPtr::setWithoutBarrier):
(JSC::CagedBarrierPtr::operator* const):
(JSC::CagedBarrierPtr::operator-> const):
(JSC::CagedBarrierPtr::operator[] const):

  • runtime/DirectArguments.cpp:

(JSC::DirectArguments::overrideThings):
(JSC::DirectArguments::unmapArgument):

  • runtime/DirectArguments.h:

(JSC::DirectArguments::isMappedArgument const):

  • runtime/GenericArguments.h:
  • runtime/GenericArgumentsInlines.h:

(JSC::GenericArguments<Type>::initModifiedArgumentsDescriptor):
(JSC::GenericArguments<Type>::setModifiedArgumentDescriptor):
(JSC::GenericArguments<Type>::isModifiedArgumentDescriptor):

  • runtime/HashMapImpl.cpp:

(JSC::HashMapImpl<HashMapBucket>::visitChildren):

  • runtime/HashMapImpl.h:

(JSC::HashMapBuffer::create):
(JSC::HashMapImpl::buffer const):
(JSC::HashMapImpl::rehash):

  • runtime/JSArray.cpp:

(JSC::JSArray::tryCreateUninitializedRestricted):
(JSC::JSArray::unshiftCountSlowCase):
(JSC::JSArray::setLength):
(JSC::JSArray::pop):
(JSC::JSArray::push):
(JSC::JSArray::fastSlice):
(JSC::JSArray::shiftCountWithArrayStorage):
(JSC::JSArray::shiftCountWithAnyIndexingType):
(JSC::JSArray::unshiftCountWithAnyIndexingType):
(JSC::JSArray::fillArgList):
(JSC::JSArray::copyToArguments):

  • runtime/JSArray.h:

(JSC::JSArray::tryCreate):

  • runtime/JSArrayBufferView.cpp:

(JSC::JSArrayBufferView::ConstructionContext::ConstructionContext):
(JSC::JSArrayBufferView::finalize):

  • runtime/JSLock.cpp:

(JSC::JSLock::didAcquireLock):

  • runtime/JSObject.cpp:

(JSC::JSObject::heapSnapshot):
(JSC::JSObject::getOwnPropertySlotByIndex):
(JSC::JSObject::putByIndex):
(JSC::JSObject::enterDictionaryIndexingMode):
(JSC::JSObject::createInitialIndexedStorage):
(JSC::JSObject::createArrayStorage):
(JSC::JSObject::convertUndecidedToInt32):
(JSC::JSObject::convertUndecidedToDouble):
(JSC::JSObject::convertUndecidedToContiguous):
(JSC::JSObject::constructConvertedArrayStorageWithoutCopyingElements):
(JSC::JSObject::convertUndecidedToArrayStorage):
(JSC::JSObject::convertInt32ToDouble):
(JSC::JSObject::convertInt32ToContiguous):
(JSC::JSObject::convertInt32ToArrayStorage):
(JSC::JSObject::convertDoubleToContiguous):
(JSC::JSObject::convertDoubleToArrayStorage):
(JSC::JSObject::convertContiguousToArrayStorage):
(JSC::JSObject::setIndexQuicklyToUndecided):
(JSC::JSObject::ensureArrayStorageExistsAndEnterDictionaryIndexingMode):
(JSC::JSObject::deletePropertyByIndex):
(JSC::JSObject::getOwnPropertyNames):
(JSC::JSObject::putIndexedDescriptor):
(JSC::JSObject::defineOwnIndexedProperty):
(JSC::JSObject::putByIndexBeyondVectorLengthWithoutAttributes):
(JSC::JSObject::putDirectIndexSlowOrBeyondVectorLength):
(JSC::JSObject::getNewVectorLength):
(JSC::JSObject::ensureLengthSlow):
(JSC::JSObject::reallocateAndShrinkButterfly):
(JSC::JSObject::allocateMoreOutOfLineStorage):
(JSC::JSObject::getEnumerableLength):

  • runtime/JSObject.h:

(JSC::JSObject::getArrayLength const):
(JSC::JSObject::getVectorLength):
(JSC::JSObject::putDirectIndex):
(JSC::JSObject::canGetIndexQuickly):
(JSC::JSObject::getIndexQuickly):
(JSC::JSObject::tryGetIndexQuickly const):
(JSC::JSObject::canSetIndexQuickly):
(JSC::JSObject::setIndexQuickly):
(JSC::JSObject::initializeIndex):
(JSC::JSObject::initializeIndexWithoutBarrier):
(JSC::JSObject::hasSparseMap):
(JSC::JSObject::inSparseIndexingMode):
(JSC::JSObject::butterfly const):
(JSC::JSObject::butterfly):
(JSC::JSObject::outOfLineStorage const):
(JSC::JSObject::outOfLineStorage):
(JSC::JSObject::ensureInt32):
(JSC::JSObject::ensureDouble):
(JSC::JSObject::ensureContiguous):
(JSC::JSObject::ensureArrayStorage):
(JSC::JSObject::arrayStorage):
(JSC::JSObject::arrayStorageOrNull):
(JSC::JSObject::ensureLength):

  • runtime/RegExpMatchesArray.h:

(JSC::tryCreateUninitializedRegExpMatchesArray):

  • runtime/VM.cpp:

(JSC::VM::VM):
(JSC::VM::~VM):
(JSC::VM::primitiveGigacageDisabledCallback):
(JSC::VM::primitiveGigacageDisabled):
(JSC::VM::gigacageDisabledCallback): Deleted.
(JSC::VM::gigacageDisabled): Deleted.

  • runtime/VM.h:

(JSC::VM::gigacageAuxiliarySpace):
(JSC::VM::firePrimitiveGigacageEnabledIfNecessary):
(JSC::VM::primitiveGigacageEnabled):
(JSC::VM::fireGigacageEnabledIfNecessary): Deleted.
(JSC::VM::gigacageEnabled): Deleted.

  • wasm/WasmMemory.cpp:

(JSC::Wasm::Memory::create):
(JSC::Wasm::Memory::~Memory):
(JSC::Wasm::Memory::grow):

Source/WebCore:

No new tests because no change in behavior.

Adapting to API changes - we now specify the AlignedMemoryAllocator differently and we need to be
specific about which Gigacage we're using.

  • bindings/js/WebCoreJSClientData.cpp:

(WebCore::JSVMClientData::JSVMClientData):

  • platform/graphics/cocoa/GPUBufferMetal.mm:

(WebCore::GPUBuffer::GPUBuffer):

Source/WebKit:


The disable callback is all about the primitive gigacage.

  • WebProcess/WebProcess.cpp:

(WebKit::primitiveGigacageDisabled):
(WebKit::m_webSQLiteDatabaseTracker):
(WebKit::gigacageDisabled): Deleted.

Source/WTF:


This mirrors the changes from bmalloc/Gigacage.h.

Also it teaches CagedPtr how to reason about multiple gigacages.

  • wtf/CagedPtr.h:

(WTF::CagedPtr::get const):
(WTF::CagedPtr::operator[] const):

  • wtf/Gigacage.cpp:

(Gigacage::tryMalloc):
(Gigacage::tryAllocateVirtualPages):
(Gigacage::freeVirtualPages):
(Gigacage::tryAlignedMalloc):
(Gigacage::alignedFree):
(Gigacage::free):

  • wtf/Gigacage.h:

(Gigacage::disablePrimitiveGigacage):
(Gigacage::addPrimitiveDisableCallback):
(Gigacage::removePrimitiveDisableCallback):
(Gigacage::name):
(Gigacage::basePtr):
(Gigacage::caged):
(Gigacage::isCaged):
(Gigacage::tryAlignedMalloc):
(Gigacage::alignedFree):
(Gigacage::free):
(Gigacage::disableGigacage): Deleted.
(Gigacage::addDisableCallback): Deleted.
(Gigacage::removeDisableCallback): Deleted.

File:
1 edited

Legend:

Unmodified
Added
Removed

  • trunk/Source/JavaScriptCore/runtime/VM.cpp

    r220322 r220352  
    166166#endif // USE(CF)
    167167    , heap(this, heapType)
    168     , auxiliarySpace("Auxiliary", heap, AllocatorAttributes(DoesNotNeedDestruction, HeapCell::Auxiliary), &GigacageAlignedMemoryAllocator::instance())
    169     , cellSpace("JSCell", heap, AllocatorAttributes(DoesNotNeedDestruction, HeapCell::JSCell), &FastMallocAlignedMemoryAllocator::instance())
    170     , destructibleCellSpace("Destructible JSCell", heap, AllocatorAttributes(NeedsDestruction, HeapCell::JSCell), &FastMallocAlignedMemoryAllocator::instance())
    171     , stringSpace("JSString", heap, &FastMallocAlignedMemoryAllocator::instance())
    172     , destructibleObjectSpace("JSDestructibleObject", heap, &FastMallocAlignedMemoryAllocator::instance())
    173     , eagerlySweptDestructibleObjectSpace("Eagerly Swept JSDestructibleObject", heap, &FastMallocAlignedMemoryAllocator::instance())
    174     , segmentedVariableObjectSpace("JSSegmentedVariableObjectSpace", heap, &FastMallocAlignedMemoryAllocator::instance())
     168    , fastMallocAllocator(std::make_unique<FastMallocAlignedMemoryAllocator>())
     169    , primitiveGigacageAllocator(std::make_unique<GigacageAlignedMemoryAllocator>(Gigacage::Primitive))
     170    , jsValueGigacageAllocator(std::make_unique<GigacageAlignedMemoryAllocator>(Gigacage::JSValue))
     171    , primitiveGigacageAuxiliarySpace("Primitive Gigacage Auxiliary", heap, AllocatorAttributes(DoesNotNeedDestruction, HeapCell::Auxiliary), primitiveGigacageAllocator.get())
     172    , jsValueGigacageAuxiliarySpace("JSValue Gigacage Auxiliary", heap, AllocatorAttributes(DoesNotNeedDestruction, HeapCell::Auxiliary), jsValueGigacageAllocator.get())
     173    , cellSpace("JSCell", heap, AllocatorAttributes(DoesNotNeedDestruction, HeapCell::JSCell), fastMallocAllocator.get())
     174    , destructibleCellSpace("Destructible JSCell", heap, AllocatorAttributes(NeedsDestruction, HeapCell::JSCell), fastMallocAllocator.get())
     175    , stringSpace("JSString", heap, fastMallocAllocator.get())
     176    , destructibleObjectSpace("JSDestructibleObject", heap, fastMallocAllocator.get())
     177    , eagerlySweptDestructibleObjectSpace("Eagerly Swept JSDestructibleObject", heap, fastMallocAllocator.get())
     178    , segmentedVariableObjectSpace("JSSegmentedVariableObjectSpace", heap, fastMallocAllocator.get())
    175179#if ENABLE(WEBASSEMBLY)
    176     , webAssemblyCodeBlockSpace("JSWebAssemblyCodeBlockSpace", heap, &FastMallocAlignedMemoryAllocator::instance())
     180    , webAssemblyCodeBlockSpace("JSWebAssemblyCodeBlockSpace", heap, fastMallocAllocator.get())
    177181#endif
    178182    , vmType(vmType)
    … …  
    212216    , m_builtinExecutables(std::make_unique<BuiltinExecutables>(*this))
    213217    , m_typeProfilerEnabledCount(0)
    214     , m_gigacageEnabled(IsWatched)
     218    , m_primitiveGigacageEnabled(IsWatched)
    215219    , m_controlFlowProfilerEnabledCount(0)
    216220    , m_shadowChicken(std::make_unique<ShadowChicken>())
    … …  
    291295#endif
    292296   
    293     Gigacage::addDisableCallback(gigacageDisabledCallback, this);
     297    Gigacage::addPrimitiveDisableCallback(primitiveGigacageDisabledCallback, this);
    294298
    295299    heap.notifyIsSafeToCollect();
    … …  
    355359    auto destructionLocker = holdLock(s_destructionLock.read());
    356360   
    357     Gigacage::removeDisableCallback(gigacageDisabledCallback, this);
     361    Gigacage::removePrimitiveDisableCallback(primitiveGigacageDisabledCallback, this);
    358362    promiseDeferredTimer->stopRunningTasks();
    359363#if ENABLE(WEBASSEMBLY)
    … …  
    424428}
    425429
    426 void VM::gigacageDisabledCallback(void* argument)
    427 {
    428     static_cast<VM*>(argument)->gigacageDisabled();
    429 }
    430 
    431 void VM::gigacageDisabled()
     430void VM::primitiveGigacageDisabledCallback(void* argument)
     431{
     432    static_cast<VM*>(argument)->primitiveGigacageDisabled();
     433}
     434
     435void VM::primitiveGigacageDisabled()
    432436{
    433437    if (m_apiLock->currentThreadIsHoldingLock()) {
    434         m_gigacageEnabled.fireAll(*this, "Gigacage disabled");
     438        m_primitiveGigacageEnabled.fireAll(*this, "Primitive gigacage disabled");
    435439        return;
    436440    }
    … …  
    438442    // This is totally racy, and that's OK. The point is, it's up to the user to ensure that they pass the
    439443    // uncaged buffer in a nicely synchronized manner.
    440     m_needToFireGigacageEnabled = true;
     444    m_needToFirePrimitiveGigacageEnabled = true;
    441445}
    442446
Note: See TracChangeset for help on using the changeset viewer.