Ignore:
Timestamp:
Oct 18, 2016, 11:30:05 AM (9 years ago)
Author:
[email protected]
Message:

DFG and FTL should be able to use DirectCall ICs when they proved the callee or its executable
​https://bugs.webkit.org/show_bug.cgi?id=163371

Reviewed by Geoffrey Garen and Saam Barati.

JSTests:

Add microbenchmarks for all of the cases that this patch optimizes.

  • microbenchmarks/direct-call-arity-mismatch.js: Added.

(foo):
(bar):

  • microbenchmarks/direct-call.js: Added.

(foo):
(bar):

  • microbenchmarks/direct-construct-arity-mismatch.js: Added.

(Foo):
(bar):

  • microbenchmarks/direct-construct.js: Added.

(Foo):
(bar):

  • microbenchmarks/direct-tail-call-arity-mismatch.js: Added.

(foo):
(bar):

  • microbenchmarks/direct-tail-call-inlined-caller-arity-mismatch.js: Added.

(foo):
(bar):
(baz):

  • microbenchmarks/direct-tail-call-inlined-caller.js: Added.

(foo):
(bar):
(baz):

  • microbenchmarks/direct-tail-call.js: Added.

(foo):
(bar):

Source/JavaScriptCore:

This adds a new kind of call inline cache for when the DFG can prove what the callee
executable is. In those cases, we can skip some of the things that the traditional call IC
would do:

  • No need to check who the callee is.
  • No need to do arity checks.


This case isn't as simple as just emitting a call instruction since the callee may not be
compiled at the time that the caller is compiled. So, we need lazy resolution. Also, the
callee may be jettisoned independently of the caller, so we need to be able to revert the
call to an unlinked state. This means that we need almost all of the things that
CallLinkInfo has. CallLinkInfo already knows about different kinds of calls. This patch
teaches it about new "Direct" call types.

The direct non-tail call IC looks like this:

set up arguments

FastPath:

call _SlowPath
lea -FrameSize(%rbp), %rsp


SlowPath:

pop
call operationLinkDirectCall
check exception
jmp FastPath


The job of operationLinkDirectCall is to link the fast path's call entrypoint of the callee.
This means that in steady state, a call is just that: a call. There are no extra branches or
checks.

The direct tail call IC is a bit more complicated because the act of setting up arguments
destroys our frame, which would prevent us from being able to throw an exception if we
failed to compile the callee. So, direct tail call ICs look like this:

jmp _SlowPath

FastPath:

set up arguments
jmp 0 patch to jump to callee


SlowPath:

silent spill
call operationLinkDirectCall
silent fill
check exception
jmp FastPath


The jmp to the slow path is patched to be a fall-through jmp when we link the call.

Direct calls mean less code at call sites, fewer checks on the steady state call fast path,
and no need for arity fixup. This looks like a slight speed-up (~0.8%) on both Octane and
AsmBench.

  • assembler/ARM64Assembler.h:

(JSC::ARM64Assembler::relinkJumpToNop):

  • assembler/ARMv7Assembler.h:

(JSC::ARMv7Assembler::relinkJumpToNop):
(JSC::ARMv7Assembler::relinkJump): Deleted.

  • assembler/AbstractMacroAssembler.h:

(JSC::AbstractMacroAssembler::repatchJumpToNop):
(JSC::AbstractMacroAssembler::repatchJump): Deleted.

  • assembler/X86Assembler.h:

(JSC::X86Assembler::relinkJumpToNop):

  • bytecode/CallLinkInfo.cpp:

(JSC::CallLinkInfo::CallLinkInfo):
(JSC::CallLinkInfo::callReturnLocation):
(JSC::CallLinkInfo::patchableJump):
(JSC::CallLinkInfo::hotPathBegin):
(JSC::CallLinkInfo::slowPathStart):
(JSC::CallLinkInfo::setCallee):
(JSC::CallLinkInfo::clearCallee):
(JSC::CallLinkInfo::callee):
(JSC::CallLinkInfo::setCodeBlock):
(JSC::CallLinkInfo::clearCodeBlock):
(JSC::CallLinkInfo::codeBlock):
(JSC::CallLinkInfo::setLastSeenCallee):
(JSC::CallLinkInfo::clearLastSeenCallee):
(JSC::CallLinkInfo::lastSeenCallee):
(JSC::CallLinkInfo::haveLastSeenCallee):
(JSC::CallLinkInfo::setExecutableDuringCompilation):
(JSC::CallLinkInfo::executable):
(JSC::CallLinkInfo::setMaxNumArguments):
(JSC::CallLinkInfo::visitWeak):

  • bytecode/CallLinkInfo.h:

(JSC::CallLinkInfo::specializationKindFor):
(JSC::CallLinkInfo::callModeFor):
(JSC::CallLinkInfo::isDirect):
(JSC::CallLinkInfo::nearCallMode):
(JSC::CallLinkInfo::isLinked):
(JSC::CallLinkInfo::setCallLocations):
(JSC::CallLinkInfo::addressOfMaxNumArguments):
(JSC::CallLinkInfo::maxNumArguments):
(JSC::CallLinkInfo::isTailCall): Deleted.
(JSC::CallLinkInfo::setUpCallFromFTL): Deleted.
(JSC::CallLinkInfo::callReturnLocation): Deleted.
(JSC::CallLinkInfo::hotPathBegin): Deleted.
(JSC::CallLinkInfo::callee): Deleted.
(JSC::CallLinkInfo::setLastSeenCallee): Deleted.
(JSC::CallLinkInfo::clearLastSeenCallee): Deleted.
(JSC::CallLinkInfo::lastSeenCallee): Deleted.
(JSC::CallLinkInfo::haveLastSeenCallee): Deleted.

  • bytecode/CallLinkStatus.cpp:

(JSC::CallLinkStatus::computeDFGStatuses):

  • bytecode/PolymorphicAccess.cpp:

(JSC::AccessCase::generateImpl):

  • bytecode/UnlinkedFunctionExecutable.h:
  • bytecode/ValueRecovery.h:

(JSC::ValueRecovery::forEachReg):

  • dfg/DFGAbstractInterpreterInlines.h:

(JSC::DFG::AbstractInterpreter<AbstractStateType>::executeEffects):

  • dfg/DFGBasicBlock.h:

(JSC::DFG::BasicBlock::findTerminal):

  • dfg/DFGByteCodeParser.cpp:

(JSC::DFG::ByteCodeParser::addCallWithoutSettingResult):
(JSC::DFG::ByteCodeParser::handleCall):

  • dfg/DFGClobberize.h:

(JSC::DFG::clobberize):

  • dfg/DFGDoesGC.cpp:

(JSC::DFG::doesGC):

  • dfg/DFGFixupPhase.cpp:

(JSC::DFG::FixupPhase::fixupNode):

  • dfg/DFGGraph.cpp:

(JSC::DFG::Graph::parameterSlotsForArgCount):

  • dfg/DFGGraph.h:
  • dfg/DFGInPlaceAbstractState.cpp:

(JSC::DFG::InPlaceAbstractState::mergeToSuccessors):

  • dfg/DFGJITCompiler.cpp:

(JSC::DFG::JITCompiler::link):

  • dfg/DFGJITCompiler.h:

(JSC::DFG::JITCompiler::addJSDirectCall):
(JSC::DFG::JITCompiler::addJSDirectTailCall):
(JSC::DFG::JITCompiler::JSCallRecord::JSCallRecord):
(JSC::DFG::JITCompiler::JSDirectCallRecord::JSDirectCallRecord):
(JSC::DFG::JITCompiler::JSDirectTailCallRecord::JSDirectTailCallRecord):
(JSC::DFG::JITCompiler::currentJSCallIndex): Deleted.

  • dfg/DFGNode.cpp:

(JSC::DFG::Node::convertToDirectCall):

  • dfg/DFGNode.h:

(JSC::DFG::Node::isTerminal):
(JSC::DFG::Node::hasHeapPrediction):
(JSC::DFG::Node::hasCellOperand):

  • dfg/DFGNodeType.h:
  • dfg/DFGPredictionPropagationPhase.cpp:
  • dfg/DFGSafeToExecute.h:

(JSC::DFG::safeToExecute):

  • dfg/DFGSpeculativeJIT.h:

(JSC::DFG::SpeculativeJIT::callOperation):

  • dfg/DFGSpeculativeJIT64.cpp:

(JSC::DFG::SpeculativeJIT::emitCall):
(JSC::DFG::SpeculativeJIT::compile):

  • dfg/DFGStrengthReductionPhase.cpp:

(JSC::DFG::StrengthReductionPhase::handleNode):

  • ftl/FTLCapabilities.cpp:

(JSC::FTL::canCompile):

  • ftl/FTLLowerDFGToB3.cpp:

(JSC::FTL::DFG::LowerDFGToB3::compileNode):
(JSC::FTL::DFG::LowerDFGToB3::compileCallOrConstruct):
(JSC::FTL::DFG::LowerDFGToB3::compileDirectCallOrConstruct):
(JSC::FTL::DFG::LowerDFGToB3::compileTailCall):
(JSC::FTL::DFG::LowerDFGToB3::compileCallOrConstructVarargs):

  • interpreter/Interpreter.cpp:

(JSC::Interpreter::execute):
(JSC::Interpreter::executeCall):
(JSC::Interpreter::executeConstruct):
(JSC::Interpreter::prepareForRepeatCall):

  • jit/JIT.cpp:

(JSC::JIT::link):

  • jit/JITCall.cpp:

(JSC::JIT::compileSetupVarargsFrame):

  • jit/JITCall32_64.cpp:

(JSC::JIT::compileSetupVarargsFrame):

  • jit/JITOperations.cpp:
  • jit/JITOperations.h:
  • jit/Repatch.cpp:

(JSC::linkDirectFor):
(JSC::revertCall):

  • jit/Repatch.h:
  • llint/LLIntSlowPaths.cpp:

(JSC::LLInt::setUpCall):

  • runtime/Executable.cpp:

(JSC::ScriptExecutable::prepareForExecutionImpl):

  • runtime/Executable.h:

(JSC::ScriptExecutable::prepareForExecution):

  • runtime/Options.h:
File:
1 edited

Legend:

Unmodified
Added
Removed

  • trunk/Source/JavaScriptCore/bytecode/CallLinkInfo.h

    r207222 r207475  
    2929#include "CodeLocation.h"
    3030#include "CodeSpecializationKind.h"
    31 #include "JSFunction.h"
    3231#include "PolymorphicCallStubRoutine.h"
    3332#include "WriteBarrier.h"
    … …  
    3837#if ENABLE(JIT)
    3938
     39class FunctionCodeBlock;
     40class JSFunction;
    4041enum OpcodeID : unsigned;
    4142struct CallFrameShuffleData;
    … …  
    4344class CallLinkInfo : public BasicRawSentinelNode<CallLinkInfo> {
    4445public:
    45     enum CallType { None, Call, CallVarargs, Construct, ConstructVarargs, TailCall, TailCallVarargs };
     46    enum CallType {
     47        None,
     48        Call,
     49        CallVarargs,
     50        Construct,
     51        ConstructVarargs,
     52        TailCall,
     53        TailCallVarargs,
     54        DirectCall,
     55        DirectConstruct,
     56        DirectTailCall
     57    };
     58   
    4659    static CallType callTypeFor(OpcodeID opcodeID);
    4760
    … …  
    6578    static CodeSpecializationKind specializationKindFor(CallType callType)
    6679    {
    67         return specializationFromIsConstruct(callType == Construct || callType == ConstructVarargs);
     80        return specializationFromIsConstruct(callType == Construct || callType == ConstructVarargs || callType == DirectConstruct);
    6881    }
    6982    CodeSpecializationKind specializationKind() const
    … …  
    7184        return specializationKindFor(static_cast<CallType>(m_callType));
    7285    }
    73 
     86   
    7487    static CallMode callModeFor(CallType callType)
    7588    {
    … …  
    7790        case Call:
    7891        case CallVarargs:
     92        case DirectCall:
    7993            return CallMode::Regular;
    8094        case TailCall:
    8195        case TailCallVarargs:
     96        case DirectTailCall:
    8297            return CallMode::Tail;
    8398        case Construct:
    8499        case ConstructVarargs:
     100        case DirectConstruct:
    85101            return CallMode::Construct;
    86102        case None:
    … …  
    90106        RELEASE_ASSERT_NOT_REACHED();
    91107    }
    92 
     108   
     109    static bool isDirect(CallType callType)
     110    {
     111        switch (callType) {
     112        case DirectCall:
     113        case DirectTailCall:
     114        case DirectConstruct:
     115            return true;
     116        case Call:
     117        case CallVarargs:
     118        case TailCall:
     119        case TailCallVarargs:
     120        case Construct:
     121        case ConstructVarargs:
     122            return false;
     123        case None:
     124            RELEASE_ASSERT_NOT_REACHED();
     125            return false;
     126        }
     127
     128        RELEASE_ASSERT_NOT_REACHED();
     129        return false;
     130    }
     131   
    93132    CallMode callMode() const
    94133    {
    … …  
    96135    }
    97136
     137    bool isDirect()
     138    {
     139        return isDirect(static_cast<CallType>(m_callType));
     140    }
     141
    98142    bool isTailCall() const
    99143    {
    100144        return callMode() == CallMode::Tail;
    101145    }
     146   
     147    NearCallMode nearCallMode() const
     148    {
     149        return isTailCall() ? Tail : Regular;
     150    }
    102151
    103152    bool isVarargs() const
    … …  
    106155    }
    107156
    108     bool isLinked() { return m_stub || m_callee; }
     157    bool isLinked() { return m_stub || m_calleeOrCodeBlock; }
    109158    void unlink(VM&);
    110159
    … …  
    116165    }
    117166
    118     void setCallLocations(CodeLocationNearCall callReturnLocation, CodeLocationDataLabelPtr hotPathBegin,
     167    void setCallLocations(
     168        CodeLocationLabel callReturnLocationOrPatchableJump,
     169        CodeLocationLabel hotPathBeginOrSlowPathStart,
    119170        CodeLocationNearCall hotPathOther)
    120171    {
    121         m_callReturnLocation = callReturnLocation;
    122         m_hotPathBegin = hotPathBegin;
     172        m_callReturnLocationOrPatchableJump = callReturnLocationOrPatchableJump;
     173        m_hotPathBeginOrSlowPathStart = hotPathBeginOrSlowPathStart;
    123174        m_hotPathOther = hotPathOther;
    124175    }
    … …  
    131182    }
    132183
    133     void setUpCallFromFTL(CallType callType, CodeOrigin codeOrigin,
    134         CodeLocationNearCall callReturnLocation, CodeLocationDataLabelPtr hotPathBegin,
    135         CodeLocationNearCall hotPathOther, unsigned calleeGPR)
    136     {
    137         m_callType = callType;
    138         m_codeOrigin = codeOrigin;
    139         m_callReturnLocation = callReturnLocation;
    140         m_hotPathBegin = hotPathBegin;
    141         m_hotPathOther = hotPathOther;
    142         m_calleeGPR = calleeGPR;
    143     }
    144 
    145     CodeLocationNearCall callReturnLocation()
    146     {
    147         return m_callReturnLocation;
    148     }
    149 
    150     CodeLocationDataLabelPtr hotPathBegin()
    151     {
    152         return m_hotPathBegin;
    153     }
     184    CodeLocationNearCall callReturnLocation();
     185    CodeLocationJump patchableJump();
     186    CodeLocationDataLabelPtr hotPathBegin();
     187    CodeLocationLabel slowPathStart();
    154188
    155189    CodeLocationNearCall hotPathOther()
    … …  
    159193
    160194    void setCallee(VM&, JSCell*, JSFunction* callee);
    161 
    162195    void clearCallee();
    163 
    164     JSFunction* callee()
    165     {
    166         return m_callee.get();
    167     }
    168 
    169     void setLastSeenCallee(VM& vm, const JSCell* owner, JSFunction* callee)
    170     {
    171         m_lastSeenCallee.set(vm, owner, callee);
    172     }
    173 
    174     void clearLastSeenCallee()
    175     {
    176         m_lastSeenCallee.clear();
    177     }
    178 
    179     JSFunction* lastSeenCallee()
    180     {
    181         return m_lastSeenCallee.get();
    182     }
    183 
    184     bool haveLastSeenCallee()
    185     {
    186         return !!m_lastSeenCallee;
    187     }
    188 
     196    JSFunction* callee();
     197
     198    void setCodeBlock(VM&, JSCell*, FunctionCodeBlock*);
     199    void clearCodeBlock();
     200    FunctionCodeBlock* codeBlock();
     201
     202    void setLastSeenCallee(VM& vm, const JSCell* owner, JSFunction* callee);
     203    void clearLastSeenCallee();
     204    JSFunction* lastSeenCallee();
     205    bool haveLastSeenCallee();
     206   
     207    void setExecutableDuringCompilation(ExecutableBase*);
     208    ExecutableBase* executable();
     209   
    189210    void setStub(PassRefPtr<PolymorphicCallStubRoutine> newStub)
    190211    {
    … …  
    255276    }
    256277
    257     uint8_t* addressOfMaxNumArguments()
     278    uint32_t* addressOfMaxNumArguments()
    258279    {
    259280        return &m_maxNumArguments;
    260281    }
    261282
    262     uint8_t maxNumArguments()
     283    uint32_t maxNumArguments()
    263284    {
    264285        return m_maxNumArguments;
    265286    }
     287   
     288    void setMaxNumArguments(unsigned);
    266289
    267290    static ptrdiff_t offsetOfSlowPathCount()
    … …  
    305328
    306329private:
    307     CodeLocationNearCall m_callReturnLocation;
    308     CodeLocationDataLabelPtr m_hotPathBegin;
     330    CodeLocationLabel m_callReturnLocationOrPatchableJump;
     331    CodeLocationLabel m_hotPathBeginOrSlowPathStart;
    309332    CodeLocationNearCall m_hotPathOther;
    310     WriteBarrier<JSFunction> m_callee;
    311     WriteBarrier<JSFunction> m_lastSeenCallee;
     333    WriteBarrier<JSCell> m_calleeOrCodeBlock;
     334    WriteBarrier<JSCell> m_lastSeenCalleeOrExecutable;
    312335    RefPtr<PolymorphicCallStubRoutine> m_stub;
    313336    RefPtr<JITStubRoutine> m_slowStub;
    … …  
    317340    bool m_clearedByGC : 1;
    318341    bool m_allowStubs : 1;
     342    bool m_isLinked : 1;
    319343    unsigned m_callType : 4; // CallType
    320344    unsigned m_calleeGPR : 8;
    321     uint8_t m_maxNumArguments; // Only used for varargs calls.
     345    uint32_t m_maxNumArguments; // For varargs: the profiled maximum number of arguments. For direct: the number of stack slots allocated for arguments.
    322346    uint32_t m_slowPathCount;
    323347    CodeOrigin m_codeOrigin;
Note: See TracChangeset for help on using the changeset viewer.