Remove Gigacage from arm64 and use PAC for arm64e instead
https://bugs.webkit.org/show_bug.cgi?id=197110
Reviewed by Saam Barati.
Source/bmalloc:
Stop using gigacage on arm64 and add a new cage function cagedMayBeNull that is the same as
cage but returns a nullptr if the incoming pointer is already null.
(Gigacage::cagedMayBeNull):
Source/JavaScriptCore:
This patch makes a bunch of changes. I'll start with global changes then go over changes to each tier and finish with bug fixes.
Global Changes:
Change CagedBarrierPtr to work with PAC so constructors and accessors now expect to receive a length.
Update assembler helper methods to use do PAC when caging.
LLInt:
Add arm64e.rb backend as we missed that when originally open sourcing our arm64e code.
Add a new optional t6 temporary, which is only used currently on arm64e for GetByVal on a TypedArray.
Refactor caging into two helper macros for Primitive/JSValue cages.
Baseline/DFG:
Add authentication where needed for GetByVal and inline object construction.
FTL:
Add a new ValueRep that allows for a late register use. We want this for the authentication patchpoint since we use the length register at the same time as we are defing the authenticated pointer.
Wasm:
Use the TaggedArrayStoragePtr class for the memory base pointer. In theory we should be caging those pointers but I don't want to risk introducing a performance regression with the rest of this change. I've filed https://bugs.webkit.org/show_bug.cgi?id=197620 to do this later.
As we no longer have the Gigacage using most of our VA memory, we can enable fast memories on iOS.
Using fast memories leads to roughly a 2% JetStream2 speedup.
- assembler/MacroAssemblerARM64E.h:
(JSC::MacroAssemblerARM64E::tagArrayPtr):
(JSC::MacroAssemblerARM64E::untagArrayPtr):
(JSC::MacroAssemblerARM64E::removeArrayPtrTag):
- b3/B3LowerToAir.cpp:
- b3/B3PatchpointSpecial.cpp:
(JSC::B3::PatchpointSpecial::admitsStack):
- b3/B3StackmapSpecial.cpp:
(JSC::B3::StackmapSpecial::forEachArgImpl):
(JSC::B3::StackmapSpecial::isArgValidForRep):
- b3/B3Validate.cpp:
- b3/B3ValueRep.cpp:
(JSC::B3::ValueRep::addUsedRegistersTo const):
(JSC::B3::ValueRep::dump const):
(WTF::printInternal):
(JSC::B3::ValueRep::ValueRep):
(JSC::B3::ValueRep::isReg const):
(JSC::DFG::newTypedArrayWithSize):
- dfg/DFGSpeculativeJIT.cpp:
(JSC::DFG::SpeculativeJIT::jumpForTypedArrayIsNeuteredIfOutOfBounds):
(JSC::DFG::SpeculativeJIT::cageTypedArrayStorage):
(JSC::DFG::SpeculativeJIT::compileGetIndexedPropertyStorage):
(JSC::DFG::SpeculativeJIT::compileGetTypedArrayByteOffset):
(JSC::DFG::SpeculativeJIT::compileNewTypedArrayWithSize):
- dfg/DFGSpeculativeJIT.h:
- dfg/DFGSpeculativeJIT64.cpp:
(JSC::DFG::SpeculativeJIT::compile):
(JSC::FTL::DFG::LowerDFGToB3::compileGetIndexedPropertyStorage):
(JSC::FTL::DFG::LowerDFGToB3::compileGetTypedArrayByteOffset):
(JSC::FTL::DFG::LowerDFGToB3::compileNewTypedArray):
(JSC::FTL::DFG::LowerDFGToB3::compileDataViewGet):
(JSC::FTL::DFG::LowerDFGToB3::compileDataViewSet):
(JSC::FTL::DFG::LowerDFGToB3::untagArrayPtr):
(JSC::FTL::DFG::LowerDFGToB3::caged):
(JSC::FTL::DFG::LowerDFGToB3::speculateTypedArrayIsNotNeutered):
(JSC::AssemblyHelpers::cageConditionally):
- jit/IntrinsicEmitter.cpp:
(JSC::IntrinsicGetterAccessCase::emitIntrinsicGetter):
- jit/JITPropertyAccess.cpp:
(JSC::JIT::emitDirectArgumentsGetByVal):
(JSC::JIT::emitIntTypedArrayGetByVal):
(JSC::JIT::emitFloatTypedArrayGetByVal):
(JSC::JIT::emitIntTypedArrayPutByVal):
(JSC::JIT::emitFloatTypedArrayPutByVal):
- jit/PolymorphicCallStubRoutine.cpp:
(JSC::PolymorphicCallNode::clearCallLinkInfo):
- llint/LowLevelInterpreter64.asm:
- offlineasm/arm64.rb:
- offlineasm/arm64e.rb: Added.
- offlineasm/ast.rb:
- offlineasm/instructions.rb:
- offlineasm/registers.rb:
- offlineasm/x86.rb:
- runtime/ArrayBuffer.cpp:
(JSC::SharedArrayBufferContents::SharedArrayBufferContents):
(JSC::SharedArrayBufferContents::~SharedArrayBufferContents):
(JSC::ArrayBufferContents::ArrayBufferContents):
(JSC::ArrayBufferContents::destroy):
(JSC::ArrayBufferContents::tryAllocate):
(JSC::ArrayBufferContents::makeShared):
(JSC::ArrayBufferContents::copyTo):
(JSC::SharedArrayBufferContents::data const):
(JSC::ArrayBufferContents::data const):
(JSC::ArrayBuffer::data):
(JSC::ArrayBuffer::data const):
(JSC::ArrayBuffer::byteLength const):
- runtime/ArrayBufferView.cpp:
(JSC::ArrayBufferView::ArrayBufferView):
- runtime/ArrayBufferView.h:
(JSC::ArrayBufferView::baseAddress const):
(JSC::ArrayBufferView::byteLength const):
(JSC::ArrayBufferView::setRangeImpl):
(JSC::ArrayBufferView::getRangeImpl):
(JSC::CachedScopedArgumentsTable::encode):
(JSC::CachedScopedArgumentsTable::decode const):
- runtime/CagedBarrierPtr.h:
(JSC::CagedBarrierPtr::CagedBarrierPtr):
(JSC::CagedBarrierPtr::set):
(JSC::CagedBarrierPtr::get const):
(JSC::CagedBarrierPtr::getMayBeNull const):
(JSC::CagedBarrierPtr::getUnsafe const):
(JSC::CagedBarrierPtr::at const):
(JSC::CagedBarrierPtr::operator== const):
(JSC::CagedBarrierPtr::operator bool const):
(JSC::CagedBarrierPtr::setWithoutBarrier):
(JSC::CagedBarrierPtr::operator* const): Deleted.
(JSC::CagedBarrierPtr::operator-> const): Deleted.
(JSC::CagedBarrierPtr::operator[] const): Deleted.
(): Deleted.
(JSC::DataView::DataView):
(JSC::DataView::get):
(JSC::DataView::set):
- runtime/DirectArguments.cpp:
(JSC::DirectArguments::visitChildren):
(JSC::DirectArguments::overrideThings):
(JSC::DirectArguments::unmapArgument):
- runtime/DirectArguments.h:
- runtime/GenericArguments.h:
- runtime/GenericArgumentsInlines.h:
(JSC::GenericArguments<Type>::visitChildren):
(JSC::GenericArguments<Type>::initModifiedArgumentsDescriptor):
(JSC::GenericArguments<Type>::setModifiedArgumentDescriptor):
(JSC::GenericArguments<Type>::isModifiedArgumentDescriptor):
- runtime/GenericTypedArrayView.h:
- runtime/GenericTypedArrayViewInlines.h:
(JSC::GenericTypedArrayView<Adaptor>::GenericTypedArrayView):
- runtime/JSArrayBufferView.cpp:
(JSC::JSArrayBufferView::ConstructionContext::ConstructionContext):
(JSC::JSArrayBufferView::JSArrayBufferView):
(JSC::JSArrayBufferView::finalize):
(JSC::JSArrayBufferView::slowDownAndWasteMemory):
- runtime/JSArrayBufferView.h:
(JSC::JSArrayBufferView::ConstructionContext::vector const):
(JSC::JSArrayBufferView::isNeutered):
(JSC::JSArrayBufferView::hasVector const):
(JSC::JSArrayBufferView::vector const):
- runtime/JSGenericTypedArrayViewInlines.h:
(JSC::JSGenericTypedArrayView<Adaptor>::createUninitialized):
(JSC::JSGenericTypedArrayView<Adaptor>::estimatedSize):
(JSC::JSGenericTypedArrayView<Adaptor>::visitChildren):
- runtime/Options.h:
- runtime/ScopedArgumentsTable.cpp:
(JSC::ScopedArgumentsTable::clone):
(JSC::ScopedArgumentsTable::setLength):
- runtime/ScopedArgumentsTable.h:
- runtime/SymbolTable.h:
- wasm/WasmAirIRGenerator.cpp:
(JSC::Wasm::AirIRGenerator::restoreWebAssemblyGlobalState):
(JSC::Wasm::AirIRGenerator::addCallIndirect):
- wasm/WasmB3IRGenerator.cpp:
(JSC::Wasm::B3IRGenerator::restoreWebAssemblyGlobalState):
(JSC::Wasm::B3IRGenerator::addCallIndirect):
(JSC::Wasm::BBQPlan::complete):
(JSC::Wasm::wasmToWasm):
(JSC::Wasm::Instance::cachedMemory const):
(JSC::Wasm::Instance::updateCachedMemory):
(JSC::Wasm::Memory::Memory):
(JSC::Wasm::Memory::~Memory):
(JSC::Wasm::Memory::grow):
(JSC::Wasm::Memory::dump const):
(JSC::Wasm::Memory::memory const):
(JSC::Wasm::createJSToWasmWrapper):
- wasm/js/WebAssemblyFunction.cpp:
(JSC::WebAssemblyFunction::jsCallEntrypointSlow):
Source/WTF:
This patch changes the Gigacage to use PAC on arm64e. As part of
this process all platforms must provide their length when
materializing the caged pointer. Since it would be somewhat
confusing to have two parameters for an operator [] those methods
have been removed. Lastly, this patch removes the specializations
for void* caged pointers, instead opting to use enable_if on the
methods that would normally fail on void* e.g. anything that
returns a T&.
- WTF.xcodeproj/project.pbxproj:
- wtf/CMakeLists.txt:
- wtf/CagedPtr.h:
(WTF::CagedPtr::CagedPtr):
(WTF::CagedPtr::get const):
(WTF::CagedPtr::getMayBeNull const):
(WTF::CagedPtr::getUnsafe const):
(WTF::CagedPtr::at const):
(WTF::CagedPtr::reauthenticate):
(WTF::CagedPtr::operator=):
(WTF::CagedPtr::operator== const):
(WTF::CagedPtr::operator bool const):
(WTF::CagedPtr::operator* const): Deleted.
(WTF::CagedPtr::operator-> const): Deleted.
(WTF::CagedPtr::operator[] const): Deleted.
(): Deleted.
(WTF::CagedUniquePtr::CagedUniquePtr):
(WTF::CagedUniquePtr::create):
(WTF::CagedUniquePtr::~CagedUniquePtr):
(WTF::CagedUniquePtr::destroy):
(): Deleted.
(Gigacage::cagedMayBeNull):
(WTF::tagArrayPtr):
(WTF::untagArrayPtr):
(WTF::removeArrayPtrTag):
(WTF::retagArrayPtr):
- wtf/TaggedArrayStoragePtr.h: Copied from Source/JavaScriptCore/runtime/ArrayBufferView.cpp.
(WTF::TaggedArrayStoragePtr::TaggedArrayStoragePtr):
(WTF::TaggedArrayStoragePtr::get const):
(WTF::TaggedArrayStoragePtr::getUnsafe const):
(WTF::TaggedArrayStoragePtr::resize):
(WTF::TaggedArrayStoragePtr::operator bool const):
