Ignore:
Timestamp:
Dec 9, 2016, 11:32:38 PM (9 years ago)
Author:
[email protected]
Message:

JSVALUE64: Pass arguments in platform argument registers when making JavaScript calls
​https://bugs.webkit.org/show_bug.cgi?id=160355

Reviewed by Filip Pizlo.

JSTests:

New microbenchmarks to measure call type performance.

  • microbenchmarks/calling-computed-args.js: Added.
  • microbenchmarks/calling-many-callees.js: Added.
  • microbenchmarks/calling-one-callee-fixed.js: Added.
  • microbenchmarks/calling-one-callee.js: Added.
  • microbenchmarks/calling-poly-callees.js: Added.
  • microbenchmarks/calling-poly-extra-arity-callees.js: Added.
  • microbenchmarks/calling-tailcall.js: Added.
  • microbenchmarks/calling-virtual-arity-fixup-callees.js: Added.
  • microbenchmarks/calling-virtual-arity-fixup-stackargs.js: Added.
  • microbenchmarks/calling-virtual-callees.js: Added.
  • microbenchmarks/calling-virtual-extra-arity-callees.js: Added.

Source/JavaScriptCore:

This patch implements passing JavaScript function arguments in registers for 64 bit platforms.

The implemented convention follows the ABI conventions for the associated platform.
The first two arguments are the callee and argument count, the rest of the argument registers
contain "this" and following argument until all platform argument registers are exhausted.
Arguments beyond what fit in registers are placed on the stack in the same location as
before this patch.

For X86-64 non-Windows platforms, there are 6 argument registers specified in the related ABI.
ARM64 has had argument registers. This allows for 4 or 6 parameter values to be placed in
registers on these respective platforms. This patch doesn't implement passing arguments in
registers for 32 bit platform, since most platforms have at most 4 argument registers
specified and 32 bit platforms use two 32 bit registers/memory locations to store one JSValue.

The call frame on the stack in unchanged in format and the arguments that are passed in
registers use the corresponding call frame location as a spill location. Arguments can
also be passed on the stack. The LLInt, baseline JIT'ed code as well as the initial entry
from C++ code base arguments on the stack. DFG s and FTL generated code pass arguments
via registers. All callees can accept arguments either in registers or on the stack.
The callee is responsible for moving argument to its preferred location.

The multiple entry points to JavaSCript code is now handled via the JITEntryPoints class and
related code. That class now has entries for StackArgsArityCheckNotRequired,
StackArgsMustCheckArity and for platforms that support registers arguments,
RegisterArgsArityCheckNotRequired, RegisterArgsMustCheckArity as well as and additional
RegisterArgsPossibleExtraArgs entry point when extra registers argument are passed.
This last case is needed to spill those extra arguments to the corresponding call frame
slots.

  • JavaScriptCore.xcodeproj/project.pbxproj:
  • b3/B3ArgumentRegValue.h:
  • b3/B3Validate.cpp:
  • bytecode/CallLinkInfo.cpp:

(JSC::CallLinkInfo::CallLinkInfo):

  • bytecode/CallLinkInfo.h:

(JSC::CallLinkInfo::setUpCall):
(JSC::CallLinkInfo::argumentsLocation):
(JSC::CallLinkInfo::argumentsInRegisters):

  • bytecode/PolymorphicAccess.cpp:

(JSC::AccessCase::generateImpl):

  • dfg/DFGAbstractInterpreterInlines.h:

(JSC::DFG::AbstractInterpreter<AbstractStateType>::executeEffects):

  • dfg/DFGByteCodeParser.cpp:

(JSC::DFG::ByteCodeParser::parseBlock):

  • dfg/DFGCPSRethreadingPhase.cpp:

(JSC::DFG::CPSRethreadingPhase::canonicalizeLocalsInBlock):
(JSC::DFG::CPSRethreadingPhase::specialCaseArguments):
(JSC::DFG::CPSRethreadingPhase::computeIsFlushed):

  • dfg/DFGClobberize.h:

(JSC::DFG::clobberize):

  • dfg/DFGCommon.h:
  • dfg/DFGDCEPhase.cpp:

(JSC::DFG::DCEPhase::run):

  • dfg/DFGDoesGC.cpp:

(JSC::DFG::doesGC):

  • dfg/DFGDriver.cpp:

(JSC::DFG::compileImpl):

  • dfg/DFGFixupPhase.cpp:

(JSC::DFG::FixupPhase::fixupNode):

  • dfg/DFGGenerationInfo.h:

(JSC::DFG::GenerationInfo::initArgumentRegisterValue):

  • dfg/DFGGraph.cpp:

(JSC::DFG::Graph::dump):
(JSC::DFG::Graph::methodOfGettingAValueProfileFor):

  • dfg/DFGGraph.h:

(JSC::DFG::Graph::needsFlushedThis):
(JSC::DFG::Graph::addImmediateShouldSpeculateInt32):

  • dfg/DFGInPlaceAbstractState.cpp:

(JSC::DFG::InPlaceAbstractState::initialize):

  • dfg/DFGJITCompiler.cpp:

(JSC::DFG::JITCompiler::link):
(JSC::DFG::JITCompiler::compile):
(JSC::DFG::JITCompiler::compileFunction):
(JSC::DFG::JITCompiler::compileEntry): Deleted.

  • dfg/DFGJITCompiler.h:

(JSC::DFG::JITCompiler::addJSDirectCall):
(JSC::DFG::JITCompiler::JSDirectCallRecord::JSDirectCallRecord):
(JSC::DFG::JITCompiler::JSDirectCallRecord::hasSlowCall):

  • dfg/DFGJITFinalizer.cpp:

(JSC::DFG::JITFinalizer::JITFinalizer):
(JSC::DFG::JITFinalizer::finalize):
(JSC::DFG::JITFinalizer::finalizeFunction):

  • dfg/DFGJITFinalizer.h:
  • dfg/DFGLiveCatchVariablePreservationPhase.cpp:

(JSC::DFG::LiveCatchVariablePreservationPhase::handleBlock):

  • dfg/DFGMaximalFlushInsertionPhase.cpp:

(JSC::DFG::MaximalFlushInsertionPhase::treatRegularBlock):
(JSC::DFG::MaximalFlushInsertionPhase::treatRootBlock):

  • dfg/DFGMayExit.cpp:
  • dfg/DFGMinifiedNode.cpp:

(JSC::DFG::MinifiedNode::fromNode):

  • dfg/DFGMinifiedNode.h:

(JSC::DFG::belongsInMinifiedGraph):

  • dfg/DFGNode.cpp:

(JSC::DFG::Node::hasVariableAccessData):

  • dfg/DFGNode.h:

(JSC::DFG::Node::accessesStack):
(JSC::DFG::Node::setVariableAccessData):
(JSC::DFG::Node::hasArgumentRegisterIndex):
(JSC::DFG::Node::argumentRegisterIndex):

  • dfg/DFGNodeType.h:
  • dfg/DFGOSRAvailabilityAnalysisPhase.cpp:

(JSC::DFG::LocalOSRAvailabilityCalculator::executeNode):

  • dfg/DFGOSREntrypointCreationPhase.cpp:

(JSC::DFG::OSREntrypointCreationPhase::run):

  • dfg/DFGPlan.cpp:

(JSC::DFG::Plan::compileInThreadImpl):

  • dfg/DFGPreciseLocalClobberize.h:

(JSC::DFG::PreciseLocalClobberizeAdaptor::readTop):

  • dfg/DFGPredictionInjectionPhase.cpp:

(JSC::DFG::PredictionInjectionPhase::run):

  • dfg/DFGPredictionPropagationPhase.cpp:
  • dfg/DFGPutStackSinkingPhase.cpp:
  • dfg/DFGRegisterBank.h:

(JSC::DFG::RegisterBank::iterator::unlock):
(JSC::DFG::RegisterBank::unlockAtIndex):

  • dfg/DFGSSAConversionPhase.cpp:

(JSC::DFG::SSAConversionPhase::run):

  • dfg/DFGSafeToExecute.h:

(JSC::DFG::safeToExecute):

  • dfg/DFGSpeculativeJIT.cpp:

(JSC::DFG::SpeculativeJIT::SpeculativeJIT):
(JSC::DFG::SpeculativeJIT::clearGenerationInfo):
(JSC::DFG::dumpRegisterInfo):
(JSC::DFG::SpeculativeJIT::dump):
(JSC::DFG::SpeculativeJIT::compileCurrentBlock):
(JSC::DFG::SpeculativeJIT::checkArgumentTypes):
(JSC::DFG::SpeculativeJIT::setupArgumentRegistersForEntry):
(JSC::DFG::SpeculativeJIT::compile):

  • dfg/DFGSpeculativeJIT.h:

(JSC::DFG::SpeculativeJIT::allocate):
(JSC::DFG::SpeculativeJIT::spill):
(JSC::DFG::SpeculativeJIT::generationInfoFromVirtualRegister):
(JSC::DFG::JSValueOperand::JSValueOperand):
(JSC::DFG::JSValueOperand::gprUseSpecific):

  • dfg/DFGSpeculativeJIT32_64.cpp:

(JSC::DFG::SpeculativeJIT::emitCall):
(JSC::DFG::SpeculativeJIT::compile):

  • dfg/DFGSpeculativeJIT64.cpp:

(JSC::DFG::SpeculativeJIT::fillJSValue):
(JSC::DFG::SpeculativeJIT::emitCall):
(JSC::DFG::SpeculativeJIT::compile):

  • dfg/DFGStrengthReductionPhase.cpp:

(JSC::DFG::StrengthReductionPhase::handleNode):

  • dfg/DFGThunks.cpp:

(JSC::DFG::osrEntryThunkGenerator):

  • dfg/DFGVariableEventStream.cpp:

(JSC::DFG::VariableEventStream::reconstruct):

  • dfg/DFGVirtualRegisterAllocationPhase.cpp:

(JSC::DFG::VirtualRegisterAllocationPhase::allocateRegister):
(JSC::DFG::VirtualRegisterAllocationPhase::run):

  • ftl/FTLCapabilities.cpp:

(JSC::FTL::canCompile):

  • ftl/FTLJITCode.cpp:

(JSC::FTL::JITCode::~JITCode):
(JSC::FTL::JITCode::initializeEntrypointThunk):
(JSC::FTL::JITCode::setEntryFor):
(JSC::FTL::JITCode::addressForCall):
(JSC::FTL::JITCode::executableAddressAtOffset):
(JSC::FTL::JITCode::initializeAddressForCall): Deleted.
(JSC::FTL::JITCode::initializeArityCheckEntrypoint): Deleted.

  • ftl/FTLJITCode.h:
  • ftl/FTLJITFinalizer.cpp:

(JSC::FTL::JITFinalizer::finalizeFunction):

  • ftl/FTLLink.cpp:

(JSC::FTL::link):

  • ftl/FTLLowerDFGToB3.cpp:

(JSC::FTL::DFG::LowerDFGToB3::lower):
(JSC::FTL::DFG::LowerDFGToB3::compileNode):
(JSC::FTL::DFG::LowerDFGToB3::compileGetArgumentRegister):
(JSC::FTL::DFG::LowerDFGToB3::compileCallOrConstruct):
(JSC::FTL::DFG::LowerDFGToB3::compileDirectCallOrConstruct):
(JSC::FTL::DFG::LowerDFGToB3::compileTailCall):
(JSC::FTL::DFG::LowerDFGToB3::compileCallOrConstructVarargsSpread):
(JSC::FTL::DFG::LowerDFGToB3::compileCallOrConstructVarargs):
(JSC::FTL::DFG::LowerDFGToB3::compileCallEval):

  • ftl/FTLOSREntry.cpp:

(JSC::FTL::prepareOSREntry):

  • ftl/FTLOutput.cpp:

(JSC::FTL::Output::argumentRegister):
(JSC::FTL::Output::argumentRegisterInt32):

  • ftl/FTLOutput.h:
  • interpreter/ShadowChicken.cpp:

(JSC::ShadowChicken::update):

  • jit/AssemblyHelpers.cpp:

(JSC::AssemblyHelpers::emitDumbVirtualCall):

  • jit/AssemblyHelpers.h:

(JSC::AssemblyHelpers::spillArgumentRegistersToFrameBeforePrologue):
(JSC::AssemblyHelpers::spillArgumentRegistersToFrame):
(JSC::AssemblyHelpers::fillArgumentRegistersFromFrameBeforePrologue):
(JSC::AssemblyHelpers::emitPutArgumentToCallFrameBeforePrologue):
(JSC::AssemblyHelpers::emitPutArgumentToCallFrame):
(JSC::AssemblyHelpers::emitGetFromCallFrameHeaderBeforePrologue):
(JSC::AssemblyHelpers::emitGetFromCallFrameArgumentBeforePrologue):
(JSC::AssemblyHelpers::emitGetPayloadFromCallFrameHeaderBeforePrologue):
(JSC::AssemblyHelpers::incrementCounter):

  • jit/CachedRecovery.cpp:

(JSC::CachedRecovery::addTargetJSValueRegs):

  • jit/CachedRecovery.h:

(JSC::CachedRecovery::gprTargets):
(JSC::CachedRecovery::setWantedFPR):
(JSC::CachedRecovery::wantedJSValueRegs):
(JSC::CachedRecovery::setWantedJSValueRegs): Deleted.

  • jit/CallFrameShuffleData.h:
  • jit/CallFrameShuffler.cpp:

(JSC::CallFrameShuffler::CallFrameShuffler):
(JSC::CallFrameShuffler::dump):
(JSC::CallFrameShuffler::tryWrites):
(JSC::CallFrameShuffler::prepareAny):

  • jit/CallFrameShuffler.h:

(JSC::CallFrameShuffler::snapshot):
(JSC::CallFrameShuffler::addNew):
(JSC::CallFrameShuffler::initDangerFrontier):
(JSC::CallFrameShuffler::updateDangerFrontier):
(JSC::CallFrameShuffler::findDangerFrontierFrom):

  • jit/CallFrameShuffler64.cpp:

(JSC::CallFrameShuffler::emitDisplace):

  • jit/GPRInfo.h:

(JSC::JSValueRegs::operator==):
(JSC::JSValueRegs::operator!=):
(JSC::GPRInfo::toArgumentIndex):
(JSC::argumentRegisterFor):
(JSC::argumentRegisterForCallee):
(JSC::argumentRegisterForArgumentCount):
(JSC::argumentRegisterIndexForJSFunctionArgument):
(JSC::jsFunctionArgumentForArgumentRegister):
(JSC::argumentRegisterForFunctionArgument):
(JSC::numberOfRegisterArgumentsFor):

  • jit/JIT.cpp:

(JSC::JIT::compileWithoutLinking):
(JSC::JIT::link):
(JSC::JIT::compileCTINativeCall): Deleted.

  • jit/JIT.h:

(JSC::JIT::compileNativeCallEntryPoints):

  • jit/JITCall.cpp:

(JSC::JIT::compileSetupVarargsFrame):
(JSC::JIT::compileCallEval):
(JSC::JIT::compileCallEvalSlowCase):
(JSC::JIT::compileOpCall):
(JSC::JIT::compileOpCallSlowCase):

  • jit/JITCall32_64.cpp:

(JSC::JIT::compileCallEvalSlowCase):
(JSC::JIT::compileOpCall):
(JSC::JIT::compileOpCallSlowCase):

  • jit/JITCode.cpp:

(JSC::JITCode::execute):
(JSC::DirectJITCode::DirectJITCode):
(JSC::DirectJITCode::initializeEntryPoints):
(JSC::DirectJITCode::addressForCall):
(JSC::NativeJITCode::addressForCall):
(JSC::DirectJITCode::initializeCodeRef): Deleted.

  • jit/JITCode.h:

(JSC::JITCode::executableAddress): Deleted.

  • jit/JITEntryPoints.h: Added.

(JSC::JITEntryPoints::JITEntryPoints):
(JSC::JITEntryPoints::entryFor):
(JSC::JITEntryPoints::setEntryFor):
(JSC::JITEntryPoints::offsetOfEntryFor):
(JSC::JITEntryPoints::registerEntryTypeForArgumentCount):
(JSC::JITEntryPoints::registerEntryTypeForArgumentType):
(JSC::JITEntryPoints::clearEntries):
(JSC::JITEntryPoints::operator=):
(JSC::JITEntryPointsWithRef::JITEntryPointsWithRef):
(JSC::JITEntryPointsWithRef::codeRef):
(JSC::argumentsLocationFor):
(JSC::registerEntryPointTypeFor):
(JSC::entryPointTypeFor):
(JSC::thunkEntryPointTypeFor):
(JSC::JITJSCallThunkEntryPointsWithRef::JITJSCallThunkEntryPointsWithRef):
(JSC::JITJSCallThunkEntryPointsWithRef::entryFor):
(JSC::JITJSCallThunkEntryPointsWithRef::setEntryFor):
(JSC::JITJSCallThunkEntryPointsWithRef::offsetOfEntryFor):
(JSC::JITJSCallThunkEntryPointsWithRef::clearEntries):
(JSC::JITJSCallThunkEntryPointsWithRef::codeRef):
(JSC::JITJSCallThunkEntryPointsWithRef::operator=):

  • jit/JITOpcodes.cpp:

(JSC::JIT::privateCompileJITEntryNativeCall):
(JSC::JIT::privateCompileCTINativeCall): Deleted.

  • jit/JITOpcodes32_64.cpp:

(JSC::JIT::privateCompileJITEntryNativeCall):
(JSC::JIT::privateCompileCTINativeCall): Deleted.

  • jit/JITOperations.cpp:
  • jit/JITThunks.cpp:

(JSC::JITThunks::jitEntryNativeCall):
(JSC::JITThunks::jitEntryNativeConstruct):
(JSC::JITThunks::jitEntryStub):
(JSC::JITThunks::jitCallThunkEntryStub):
(JSC::JITThunks::hostFunctionStub):
(JSC::JITThunks::ctiNativeCall): Deleted.
(JSC::JITThunks::ctiNativeConstruct): Deleted.

  • jit/JITThunks.h:
  • jit/JSInterfaceJIT.h:

(JSC::JSInterfaceJIT::emitJumpIfNotInt32):
(JSC::JSInterfaceJIT::emitLoadInt32):

  • jit/RegisterSet.cpp:

(JSC::RegisterSet::argumentRegisters):

  • jit/RegisterSet.h:
  • jit/Repatch.cpp:

(JSC::linkSlowFor):
(JSC::revertCall):
(JSC::unlinkFor):
(JSC::linkVirtualFor):
(JSC::linkPolymorphicCall):

  • jit/SpecializedThunkJIT.h:

(JSC::SpecializedThunkJIT::SpecializedThunkJIT):
(JSC::SpecializedThunkJIT::checkJSStringArgument):
(JSC::SpecializedThunkJIT::linkFailureHere):
(JSC::SpecializedThunkJIT::finalize):

  • jit/ThunkGenerator.h:
  • jit/ThunkGenerators.cpp:

(JSC::createRegisterArgumentsSpillEntry):
(JSC::slowPathFor):
(JSC::linkCallThunkGenerator):
(JSC::linkDirectCallThunkGenerator):
(JSC::linkPolymorphicCallThunkGenerator):
(JSC::virtualThunkFor):
(JSC::nativeForGenerator):
(JSC::nativeCallGenerator):
(JSC::nativeTailCallGenerator):
(JSC::nativeTailCallWithoutSavedTagsGenerator):
(JSC::nativeConstructGenerator):
(JSC::stringCharLoadRegCall):
(JSC::charCodeAtThunkGenerator):
(JSC::charAtThunkGenerator):
(JSC::fromCharCodeThunkGenerator):
(JSC::clz32ThunkGenerator):
(JSC::sqrtThunkGenerator):
(JSC::floorThunkGenerator):
(JSC::ceilThunkGenerator):
(JSC::truncThunkGenerator):
(JSC::roundThunkGenerator):
(JSC::expThunkGenerator):
(JSC::logThunkGenerator):
(JSC::absThunkGenerator):
(JSC::imulThunkGenerator):
(JSC::randomThunkGenerator):
(JSC::boundThisNoArgsFunctionCallGenerator):

  • jit/ThunkGenerators.h:
  • jsc.cpp:

(jscmain):

  • llint/LLIntEntrypoint.cpp:

(JSC::LLInt::setFunctionEntrypoint):
(JSC::LLInt::setEvalEntrypoint):
(JSC::LLInt::setProgramEntrypoint):
(JSC::LLInt::setModuleProgramEntrypoint):

  • llint/LLIntSlowPaths.cpp:

(JSC::LLInt::entryOSR):
(JSC::LLInt::setUpCall):

  • llint/LLIntThunks.cpp:

(JSC::LLInt::generateThunkWithJumpTo):
(JSC::LLInt::functionForRegisterCallEntryThunkGenerator):
(JSC::LLInt::functionForStackCallEntryThunkGenerator):
(JSC::LLInt::functionForRegisterConstructEntryThunkGenerator):
(JSC::LLInt::functionForStackConstructEntryThunkGenerator):
(JSC::LLInt::functionForRegisterCallArityCheckThunkGenerator):
(JSC::LLInt::functionForStackCallArityCheckThunkGenerator):
(JSC::LLInt::functionForRegisterConstructArityCheckThunkGenerator):
(JSC::LLInt::functionForStackConstructArityCheckThunkGenerator):
(JSC::LLInt::functionForCallEntryThunkGenerator): Deleted.
(JSC::LLInt::functionForConstructEntryThunkGenerator): Deleted.
(JSC::LLInt::functionForCallArityCheckThunkGenerator): Deleted.
(JSC::LLInt::functionForConstructArityCheckThunkGenerator): Deleted.

  • llint/LLIntThunks.h:
  • runtime/ArityCheckMode.h:
  • runtime/ExecutableBase.cpp:

(JSC::ExecutableBase::clearCode):

  • runtime/ExecutableBase.h:

(JSC::ExecutableBase::entrypointFor):
(JSC::ExecutableBase::offsetOfEntryFor):
(JSC::ExecutableBase::offsetOfJITCodeWithArityCheckFor): Deleted.

  • runtime/JSBoundFunction.cpp:

(JSC::boundThisNoArgsFunctionCall):

  • runtime/NativeExecutable.cpp:

(JSC::NativeExecutable::finishCreation):

  • runtime/ScriptExecutable.cpp:

(JSC::ScriptExecutable::installCode):

  • runtime/VM.cpp:

(JSC::VM::VM):
(JSC::thunkGeneratorForIntrinsic):
(JSC::VM::clearCounters):
(JSC::VM::dumpCounters):

  • runtime/VM.h:

(JSC::VM::getJITEntryStub):
(JSC::VM::getJITCallThunkEntryStub):
(JSC::VM::addressOfCounter):
(JSC::VM::counterFor):

  • wasm/WasmBinding.cpp:

(JSC::Wasm::importStubGenerator):

Source/WTF:

Added a new build option ENABLE_VM_COUNTERS to enable JIT'able counters.
The default is for the option to be off.

  • wtf/Platform.h:

Added ENABLE_VM_COUNTERS

File:
1 edited

Legend:

Unmodified
Added
Removed

  • trunk/Source/JavaScriptCore/ftl/FTLLowerDFGToB3.cpp

    r209638 r209653  
    197197        m_proc.addFastConstant(m_tagMask->key());
    198198       
     199        // Store out callee and argument count for possible OSR exit.
     200        m_out.store64(m_out.argumentRegister(argumentRegisterForCallee()), addressFor(CallFrameSlot::callee));
     201        m_out.store32(m_out.argumentRegisterInt32(argumentRegisterForArgumentCount()), payloadFor(CallFrameSlot::argumentCount));
     202
    199203        m_out.storePtr(m_out.constIntPtr(codeBlock()), addressFor(CallFrameSlot::codeBlock));
    200204
    … …  
    248252        availabilityMap().clear();
    249253        availabilityMap().m_locals = Operands<Availability>(codeBlock()->numParameters(), 0);
     254
     255        Vector<Node*, 8> argumentNodes;
     256        Vector<LValue, 8> argumentValues;
     257
     258        argumentNodes.resize(codeBlock()->numParameters());
     259        argumentValues.resize(codeBlock()->numParameters());
     260
     261        m_highBlock = m_graph.block(0);
     262
    250263        for (unsigned i = codeBlock()->numParameters(); i--;) {
    251             availabilityMap().m_locals.argument(i) =
    252                 Availability(FlushedAt(FlushedJSValue, virtualRegisterForArgument(i)));
    253         }
    254         m_node = nullptr;
    255         m_origin = NodeOrigin(CodeOrigin(0), CodeOrigin(0), true);
    256         for (unsigned i = codeBlock()->numParameters(); i--;) {
    257             Node* node = m_graph.m_arguments[i];
     264            Node* node = m_graph.m_argumentsForChecking[i];
    258265            VirtualRegister operand = virtualRegisterForArgument(i);
    259266           
    260             LValue jsValue = m_out.load64(addressFor(operand));
    261            
     267            LValue jsValue = nullptr;
     268
    262269            if (node) {
    263                 DFG_ASSERT(m_graph, node, operand == node->stackAccessData()->machineLocal);
     270                if (i < NUMBER_OF_JS_FUNCTION_ARGUMENT_REGISTERS) {
     271                    availabilityMap().m_locals.argument(i) = Availability(node);
     272                    jsValue = m_out.argumentRegister(GPRInfo::toArgumentRegister(node->argumentRegisterIndex()));
     273
     274                    setJSValue(node, jsValue);
     275                } else {
     276                    availabilityMap().m_locals.argument(i) =
     277                        Availability(FlushedAt(FlushedJSValue, operand));
     278                    jsValue = m_out.load64(addressFor(virtualRegisterForArgument(i)));
     279                }
     280           
     281                DFG_ASSERT(m_graph, node, node->hasArgumentRegisterIndex() || operand == node->stackAccessData()->machineLocal);
    264282               
    265283                // This is a hack, but it's an effective one. It allows us to do CSE on the
    … …  
    269287                m_loadedArgumentValues.add(node, jsValue);
    270288            }
    271            
     289
     290            argumentNodes[i] = node;
     291            argumentValues[i] = jsValue;
     292        }
     293
     294        m_node = nullptr;
     295        m_origin = NodeOrigin(CodeOrigin(0), CodeOrigin(0), true);
     296        for (unsigned i = codeBlock()->numParameters(); i--;) {
     297            Node* node = argumentNodes[i];
     298           
     299            if (!node)
     300                continue;
     301
     302            LValue jsValue = argumentValues[i];
     303
    272304            switch (m_graph.m_argumentFormats[i]) {
    273305            case FlushedInt32:
    … …  
    813845        case GetArgumentCountIncludingThis:
    814846            compileGetArgumentCountIncludingThis();
     847            break;
     848        case GetArgumentRegister:
     849            compileGetArgumentRegister();
    815850            break;
    816851        case GetScope:
    … …  
    54035438    }
    54045439   
     5440    void compileGetArgumentRegister()
     5441    {
     5442        // We might have already have a value for this node.
     5443        if (LValue value = m_loadedArgumentValues.get(m_node)) {
     5444            setJSValue(value);
     5445            return;
     5446        }
     5447        setJSValue(m_out.argumentRegister(GPRInfo::toArgumentRegister(m_node->argumentRegisterIndex())));
     5448    }
     5449   
    54055450    void compileGetScope()
    54065451    {
    … …  
    58155860        Vector<ConstrainedValue> arguments;
    58165861
    5817         // Make sure that the callee goes into GPR0 because that's where the slow path thunks expect the
    5818         // callee to be.
    5819         arguments.append(ConstrainedValue(jsCallee, ValueRep::reg(GPRInfo::regT0)));
     5862        // Make sure that the callee goes into argumentRegisterForCallee() because that's where
     5863        // the slow path thunks expect the callee to be.
     5864        GPRReg calleeReg = argumentRegisterForCallee();
     5865        arguments.append(ConstrainedValue(jsCallee, ValueRep::reg(calleeReg)));
    58205866
    58215867        auto addArgument = [&] (LValue value, VirtualRegister reg, int offset) {
    … …  
    58255871        };
    58265872
    5827         addArgument(jsCallee, VirtualRegister(CallFrameSlot::callee), 0);
    5828         addArgument(m_out.constInt32(numArgs), VirtualRegister(CallFrameSlot::argumentCount), PayloadOffset);
    5829         for (unsigned i = 0; i < numArgs; ++i)
    5830             addArgument(lowJSValue(m_graph.varArgChild(node, 1 + i)), virtualRegisterForArgument(i), 0);
     5873        ArgumentsLocation argLocation = argumentsLocationFor(numArgs);
     5874        arguments.append(ConstrainedValue(jsCallee, ValueRep::reg(calleeReg)));
     5875        arguments.append(ConstrainedValue(m_out.constInt32(numArgs), ValueRep::reg(argumentRegisterForArgumentCount())));
     5876
     5877        for (unsigned i = 0; i < numArgs; ++i) {
     5878            if (i < NUMBER_OF_JS_FUNCTION_ARGUMENT_REGISTERS)
     5879                arguments.append(ConstrainedValue(lowJSValue(m_graph.varArgChild(node, 1 + i)), ValueRep::reg(argumentRegisterForFunctionArgument(i))));
     5880            else
     5881                addArgument(lowJSValue(m_graph.varArgChild(node, 1 + i)), virtualRegisterForArgument(i), 0);
     5882        }
    58315883
    58325884        PatchpointValue* patchpoint = m_out.patchpoint(Int64);
    … …  
    58575909                CallLinkInfo* callLinkInfo = jit.codeBlock()->addCallLinkInfo();
    58585910
     5911                incrementCounter(&jit, VM::FTLCaller);
     5912
    58595913                CCallHelpers::DataLabelPtr targetToCheck;
    58605914                CCallHelpers::Jump slowPath = jit.branchPtrWithPatch(
    5861                     CCallHelpers::NotEqual, GPRInfo::regT0, targetToCheck,
     5915                    CCallHelpers::NotEqual, calleeReg, targetToCheck,
    58625916                    CCallHelpers::TrustedImmPtr(0));
    58635917
    … …  
    58675921                slowPath.link(&jit);
    58685922
    5869                 jit.move(CCallHelpers::TrustedImmPtr(callLinkInfo), GPRInfo::regT2);
     5923                jit.move(CCallHelpers::TrustedImmPtr(callLinkInfo), GPRInfo::nonArgGPR0);
    58705924                CCallHelpers::Call slowCall = jit.nearCall();
    58715925                done.link(&jit);
    … …  
    58735927                callLinkInfo->setUpCall(
    58745928                    node->op() == Construct ? CallLinkInfo::Construct : CallLinkInfo::Call,
    5875                     node->origin.semantic, GPRInfo::regT0);
     5929                    argLocation, node->origin.semantic, argumentRegisterForCallee());
    58765930
    58775931                jit.addPtr(
    … …  
    58825936                    [=] (LinkBuffer& linkBuffer) {
    58835937                        MacroAssemblerCodePtr linkCall =
    5884                             linkBuffer.vm().getCTIStub(linkCallThunkGenerator).code();
     5938                            linkBuffer.vm().getJITCallThunkEntryStub(linkCallThunkGenerator).entryFor(callLinkInfo->argumentsLocation());
    58855939                        linkBuffer.link(slowCall, FunctionPtr(linkCall.executableAddress()));
    58865940
    … …  
    59265980        Vector<ConstrainedValue> arguments;
    59275981       
    5928         arguments.append(ConstrainedValue(jsCallee, ValueRep::SomeRegister));
     5982        // Make sure that the callee goes into argumentRegisterForCallee() because that's where
     5983        // the slow path thunks expect the callee to be.
     5984        GPRReg calleeReg = argumentRegisterForCallee();
     5985        arguments.append(ConstrainedValue(jsCallee, ValueRep::reg(calleeReg)));
    59295986        if (!isTail) {
    59305987            auto addArgument = [&] (LValue value, VirtualRegister reg, int offset) {
    … …  
    59335990                arguments.append(ConstrainedValue(value, ValueRep::stackArgument(offsetFromSP)));
    59345991            };
    5935            
     5992
     5993            arguments.append(ConstrainedValue(jsCallee, ValueRep::reg(calleeReg)));
     5994#if ENABLE(CALLER_SPILLS_CALLEE)
    59365995            addArgument(jsCallee, VirtualRegister(CallFrameSlot::callee), 0);
     5996#endif
     5997            arguments.append(ConstrainedValue(m_out.constInt32(numPassedArgs), ValueRep::reg(argumentRegisterForArgumentCount())));
     5998#if ENABLE(CALLER_SPILLS_ARGCOUNT)
    59375999            addArgument(m_out.constInt32(numPassedArgs), VirtualRegister(CallFrameSlot::argumentCount), PayloadOffset);
    5938             for (unsigned i = 0; i < numPassedArgs; ++i)
    5939                 addArgument(lowJSValue(m_graph.varArgChild(node, 1 + i)), virtualRegisterForArgument(i), 0);
    5940             for (unsigned i = numPassedArgs; i < numAllocatedArgs; ++i)
    5941                 addArgument(m_out.constInt64(JSValue::encode(jsUndefined())), virtualRegisterForArgument(i), 0);
     6000#endif
     6001           
     6002            for (unsigned i = 0; i < numPassedArgs; ++i) {
     6003                if (i < NUMBER_OF_JS_FUNCTION_ARGUMENT_REGISTERS)
     6004                    arguments.append(ConstrainedValue(lowJSValue(m_graph.varArgChild(node, 1 + i)), ValueRep::reg(argumentRegisterForFunctionArgument(i))));
     6005                else
     6006                    addArgument(lowJSValue(m_graph.varArgChild(node, 1 + i)), virtualRegisterForArgument(i), 0);
     6007            }
     6008            for (unsigned i = numPassedArgs; i < numAllocatedArgs; ++i) {
     6009                if (i < NUMBER_OF_JS_FUNCTION_ARGUMENT_REGISTERS)
     6010                    arguments.append(ConstrainedValue(m_out.constInt64(JSValue::encode(jsUndefined())), ValueRep::reg(argumentRegisterForFunctionArgument(i))));
     6011                else
     6012                    addArgument(m_out.constInt64(JSValue::encode(jsUndefined())), virtualRegisterForArgument(i), 0);
     6013            }
    59426014        } else {
    59436015            for (unsigned i = 0; i < numPassedArgs; ++i)
    … …  
    59816053                   
    59826054                    RegisterSet toSave = params.unavailableRegisters();
     6055                    shuffleData.argumentsInRegisters = true;
    59836056                    shuffleData.callee = ValueRecovery::inGPR(calleeGPR, DataFormatCell);
    59846057                    toSave.set(calleeGPR);
    … …  
    59996072                    CCallHelpers::PatchableJump patchableJump = jit.patchableJump();
    60006073                    CCallHelpers::Label mainPath = jit.label();
    6001                    
     6074
     6075                    incrementCounter(&jit, VM::FTLCaller);
     6076                    incrementCounter(&jit, VM::TailCall);
     6077                    incrementCounter(&jit, VM::DirectCall);
     6078
    60026079                    jit.store32(
    60036080                        CCallHelpers::TrustedImm32(callSiteIndex.bits()),
    … …  
    60206097                   
    60216098                    callLinkInfo->setUpCall(
    6022                         CallLinkInfo::DirectTailCall, node->origin.semantic, InvalidGPRReg);
     6099                        CallLinkInfo::DirectTailCall, argumentsLocationFor(numPassedArgs), node->origin.semantic, InvalidGPRReg);
    60236100                    callLinkInfo->setExecutableDuringCompilation(executable);
    60246101                    if (numAllocatedArgs > numPassedArgs)
    … …  
    60436120                CCallHelpers::Label mainPath = jit.label();
    60446121
     6122                incrementCounter(&jit, VM::FTLCaller);
     6123                incrementCounter(&jit, VM::DirectCall);
     6124
    60456125                jit.store32(
    60466126                    CCallHelpers::TrustedImm32(callSiteIndex.bits()),
    … …  
    60546134                callLinkInfo->setUpCall(
    60556135                    isConstruct ? CallLinkInfo::DirectConstruct : CallLinkInfo::DirectCall,
    6056                     node->origin.semantic, InvalidGPRReg);
     6136                    argumentsLocationFor(numPassedArgs), node->origin.semantic, InvalidGPRReg);
    60576137                callLinkInfo->setExecutableDuringCompilation(executable);
    60586138                if (numAllocatedArgs > numPassedArgs)
    … …  
    60656145                        CCallHelpers::Label slowPath = jit.label();
    60666146                        if (isX86())
    6067                             jit.pop(CCallHelpers::selectScratchGPR(calleeGPR));
    6068                        
    6069                         callOperation(
    6070                             *state, params.unavailableRegisters(), jit,
    6071                             node->origin.semantic, exceptions.get(), operationLinkDirectCall,
    6072                             InvalidGPRReg, CCallHelpers::TrustedImmPtr(callLinkInfo),
    6073                             calleeGPR).call();
     6147                            jit.pop(GPRInfo::nonArgGPR0);
     6148
     6149                        jit.move(CCallHelpers::TrustedImmPtr(callLinkInfo), GPRInfo::nonArgGPR0); // Link info needs to be in nonArgGPR0
     6150                        CCallHelpers::Call slowCall = jit.nearCall();
     6151                        exceptions->append(jit.emitExceptionCheck(AssemblyHelpers::NormalExceptionCheck, AssemblyHelpers::FarJumpWidth));
    60746152                        jit.jump().linkTo(mainPath, &jit);
    60756153                       
    … …  
    60806158                               
    60816159                                linkBuffer.link(call, slowPathLocation);
     6160                                MacroAssemblerCodePtr linkCall =
     6161                                    linkBuffer.vm().getJITCallThunkEntryStub(linkDirectCallThunkGenerator).entryFor(callLinkInfo->argumentsLocation());
     6162                                linkBuffer.link(slowCall, FunctionPtr(linkCall.executableAddress()));
    60826163                               
    60836164                                callLinkInfo->setCallLocations(
    … …  
    61116192        Vector<ConstrainedValue> arguments;
    61126193
    6113         arguments.append(ConstrainedValue(jsCallee, ValueRep::reg(GPRInfo::regT0)));
     6194        GPRReg calleeReg = argumentRegisterForCallee();
     6195        arguments.append(ConstrainedValue(jsCallee, ValueRep::reg(calleeReg)));
    61146196
    61156197        for (unsigned i = 0; i < numArgs; ++i) {
    … …  
    61456227                CallSiteIndex callSiteIndex = state->jitCode->common.addUniqueCallSiteIndex(codeOrigin);
    61466228
     6229                incrementCounter(&jit, VM::FTLCaller);
     6230                incrementCounter(&jit, VM::TailCall);
     6231
    61476232                CallFrameShuffleData shuffleData;
     6233                shuffleData.argumentsInRegisters = true;
    61486234                shuffleData.numLocals = state->jitCode->common.frameRegisterCount;
    6149                 shuffleData.callee = ValueRecovery::inGPR(GPRInfo::regT0, DataFormatJS);
     6235                shuffleData.callee = ValueRecovery::inGPR(calleeReg, DataFormatJS);
    61506236
    61516237                for (unsigned i = 0; i < numArgs; ++i)
    … …  
    61586244                CCallHelpers::DataLabelPtr targetToCheck;
    61596245                CCallHelpers::Jump slowPath = jit.branchPtrWithPatch(
    6160                     CCallHelpers::NotEqual, GPRInfo::regT0, targetToCheck,
     6246                    CCallHelpers::NotEqual, calleeReg, targetToCheck,
    61616247                    CCallHelpers::TrustedImmPtr(0));
    61626248
    … …  
    61766262
    61776263                CallFrameShuffler slowPathShuffler(jit, shuffleData);
    6178                 slowPathShuffler.setCalleeJSValueRegs(JSValueRegs(GPRInfo::regT0));
    61796264                slowPathShuffler.prepareForSlowPath();
    61806265
    6181                 jit.move(CCallHelpers::TrustedImmPtr(callLinkInfo), GPRInfo::regT2);
     6266                jit.move(CCallHelpers::TrustedImmPtr(callLinkInfo), GPRInfo::nonArgGPR0);
    61826267                CCallHelpers::Call slowCall = jit.nearCall();
    61836268
    61846269                jit.abortWithReason(JITDidReturnFromTailCall);
    61856270
    6186                 callLinkInfo->setUpCall(CallLinkInfo::TailCall, codeOrigin, GPRInfo::regT0);
     6271                callLinkInfo->setUpCall(CallLinkInfo::TailCall, argumentsLocationFor(numArgs), codeOrigin, calleeReg);
    61876272
    61886273                jit.addLinkTask(
    61896274                    [=] (LinkBuffer& linkBuffer) {
    61906275                        MacroAssemblerCodePtr linkCall =
    6191                             linkBuffer.vm().getCTIStub(linkCallThunkGenerator).code();
     6276                            linkBuffer.vm().getJITCallThunkEntryStub(linkCallThunkGenerator).entryFor(callLinkInfo->argumentsLocation());
    61926277                        linkBuffer.link(slowCall, FunctionPtr(linkCall.executableAddress()));
    61936278
    … …  
    62796364
    62806365                CallLinkInfo* callLinkInfo = jit.codeBlock()->addCallLinkInfo();
     6366                ArgumentsLocation argumentsLocation = StackArgs;
    62816367
    62826368                RegisterSet usedRegisters = RegisterSet::allRegisters();
    … …  
    64286514                    jit.emitRestoreCalleeSaves();
    64296515                ASSERT(!usedRegisters.get(GPRInfo::regT2));
    6430                 jit.move(CCallHelpers::TrustedImmPtr(callLinkInfo), GPRInfo::regT2);
     6516                jit.move(CCallHelpers::TrustedImmPtr(callLinkInfo), GPRInfo::nonArgGPR0);
    64316517                CCallHelpers::Call slowCall = jit.nearCall();
    64326518               
    … …  
    64366522                    done.link(&jit);
    64376523               
    6438                 callLinkInfo->setUpCall(callType, node->origin.semantic, GPRInfo::regT0);
     6524                callLinkInfo->setUpCall(callType, argumentsLocation, node->origin.semantic, GPRInfo::regT0);
    64396525
    64406526                jit.addPtr(
    … …  
    64456531                    [=] (LinkBuffer& linkBuffer) {
    64466532                        MacroAssemblerCodePtr linkCall =
    6447                             linkBuffer.vm().getCTIStub(linkCallThunkGenerator).code();
     6533                            linkBuffer.vm().getJITCallThunkEntryStub(linkCallThunkGenerator).entryFor(StackArgs);
    64486534                        linkBuffer.link(slowCall, FunctionPtr(linkCall.executableAddress()));
    64496535                       
    … …  
    65466632                exceptionHandle->scheduleExitCreationForUnwind(params, callSiteIndex);
    65476633
     6634                incrementCounter(&jit, VM::FTLCaller);
     6635                incrementCounter(&jit, VM::CallVarargs);
     6636               
    65486637                jit.store32(
    65496638                    CCallHelpers::TrustedImm32(callSiteIndex.bits()),
    … …  
    65516640
    65526641                CallLinkInfo* callLinkInfo = jit.codeBlock()->addCallLinkInfo();
     6642                ArgumentsLocation argumentsLocation = StackArgs;
    65536643                CallVarargsData* data = node->callVarargsData();
    65546644
    … …  
    67116801                if (isTailCall)
    67126802                    jit.emitRestoreCalleeSaves();
    6713                 jit.move(CCallHelpers::TrustedImmPtr(callLinkInfo), GPRInfo::regT2);
     6803                jit.move(CCallHelpers::TrustedImmPtr(callLinkInfo), GPRInfo::nonArgGPR0);
    67146804                CCallHelpers::Call slowCall = jit.nearCall();
    67156805               
    … …  
    67196809                    done.link(&jit);
    67206810               
    6721                 callLinkInfo->setUpCall(callType, node->origin.semantic, GPRInfo::regT0);
     6811                callLinkInfo->setUpCall(callType, argumentsLocation, node->origin.semantic, GPRInfo::regT0);
    67226812               
    67236813                jit.addPtr(
    … …  
    67286818                    [=] (LinkBuffer& linkBuffer) {
    67296819                        MacroAssemblerCodePtr linkCall =
    6730                             linkBuffer.vm().getCTIStub(linkCallThunkGenerator).code();
     6820                            linkBuffer.vm().getJITCallThunkEntryStub(linkCallThunkGenerator).entryFor(StackArgs);
    67316821                        linkBuffer.link(slowCall, FunctionPtr(linkCall.executableAddress()));
    67326822                       
    … …  
    67976887               
    67986888                exceptionHandle->scheduleExitCreationForUnwind(params, callSiteIndex);
    6799                
     6889
     6890                incrementCounter(&jit, VM::FTLCaller);
     6891                incrementCounter(&jit, VM::CallEval);
     6892
    68006893                jit.store32(
    68016894                    CCallHelpers::TrustedImm32(callSiteIndex.bits()),
    … …  
    68036896               
    68046897                CallLinkInfo* callLinkInfo = jit.codeBlock()->addCallLinkInfo();
    6805                 callLinkInfo->setUpCall(CallLinkInfo::Call, node->origin.semantic, GPRInfo::regT0);
     6898                callLinkInfo->setUpCall(CallLinkInfo::Call, StackArgs, node->origin.semantic, GPRInfo::regT0);
    68066899               
    68076900                jit.addPtr(CCallHelpers::TrustedImm32(-static_cast<ptrdiff_t>(sizeof(CallerFrameAndPC))), CCallHelpers::stackPointerRegister, GPRInfo::regT1);
Note: See TracChangeset for help on using the changeset viewer.