Changeset 172940 in webkit for trunk/Source/JavaScriptCore


Ignore:
Timestamp:
Aug 25, 2014, 3:35:40 PM (11 years ago)
Author:
[email protected]
Message:

FTL should be able to do polymorphic call inlining
​https://bugs.webkit.org/show_bug.cgi?id=135145

Reviewed by Geoffrey Garen.
Source/JavaScriptCore:


Added a log-based high-fidelity call edge profiler that runs in DFG JIT (and optionally
baseline JIT) code. Used it to do precise polymorphic inlining in the FTL. Potential
inlining sites use the call edge profile if it is available, but they will still fall back
on the call inline cache and rare case counts if it's not. Polymorphic inlining means that
multiple possible callees can be inlined with a switch to guard them. The slow path may
either be an OSR exit or a virtual call.

The call edge profiling added in this patch is very precise - it will tell you about every
call that has ever happened. It took some effort to reduce the overhead of this profiling.
This mostly involved ensuring that we don't do it unnecessarily. For example, we avoid it
in the baseline JIT (you can conditionally enable it but it's off by default) and we only do
it in the DFG JIT if we know that the regular inline cache profiling wasn't precise enough.
I also experimented with reducing the precision of the profiling. This led to a significant
reduction in the speed-up, so I avoided this approach. I also explored making log processing
concurrent, but that didn't help. Also, I tested the overhead of the log processing and
found that most of the overhead of this profiling is actually in putting things into the log
rather than in processing the log - that part appears to be surprisingly cheap.

Polymorphic inlining could be enabled in the DFG if we enabled baseline call edge profiling,
and if we guarded such inlining sites with some profiling mechanism to detect
polyvariant monomorphisation opportunities (where the callsite being inlined reveals that
it's actually monomorphic).

This is a ~28% speed-up on deltablue and a ~7% speed-up on richards, with small speed-ups on
other programs as well. It's about a 2% speed-up on Octane version 2, and never a regression
on anything we care about. Some aggregates, like V8Spider, see a regression. This is
highlighting the increase in profiling overhead. But since this doesn't show up on any major
score (code-load or SunSpider), it's probably not relevant.

(JSC::CallEdge::dump):

  • bytecode/CallEdge.h: Added.

(JSC::CallEdge::operator!):
(JSC::CallEdge::callee):
(JSC::CallEdge::count):
(JSC::CallEdge::despecifiedClosure):
(JSC::CallEdge::CallEdge):

  • bytecode/CallEdgeProfile.cpp: Added.

(JSC::CallEdgeProfile::callEdges):
(JSC::CallEdgeProfile::numCallsToKnownCells):
(JSC::worthDespecifying):
(JSC::CallEdgeProfile::worthDespecifying):
(JSC::CallEdgeProfile::visitWeak):
(JSC::CallEdgeProfile::addSlow):
(JSC::CallEdgeProfile::mergeBack):
(JSC::CallEdgeProfile::fadeByHalf):
(JSC::CallEdgeLog::CallEdgeLog):
(JSC::CallEdgeLog::~CallEdgeLog):
(JSC::CallEdgeLog::isEnabled):
(JSC::operationProcessCallEdgeLog):
(JSC::CallEdgeLog::emitLogCode):
(JSC::CallEdgeLog::processLog):

  • bytecode/CallEdgeProfile.h: Added.

(JSC::CallEdgeProfile::numCallsToNotCell):
(JSC::CallEdgeProfile::numCallsToUnknownCell):
(JSC::CallEdgeProfile::totalCalls):

  • bytecode/CallEdgeProfileInlines.h: Added.

(JSC::CallEdgeProfile::CallEdgeProfile):
(JSC::CallEdgeProfile::add):

  • bytecode/CallLinkInfo.cpp:

(JSC::CallLinkInfo::visitWeak):

  • bytecode/CallLinkInfo.h:
  • bytecode/CallLinkStatus.cpp:

(JSC::CallLinkStatus::CallLinkStatus):
(JSC::CallLinkStatus::computeFromLLInt):
(JSC::CallLinkStatus::computeFor):
(JSC::CallLinkStatus::computeExitSiteData):
(JSC::CallLinkStatus::computeFromCallLinkInfo):
(JSC::CallLinkStatus::computeFromCallEdgeProfile):
(JSC::CallLinkStatus::computeDFGStatuses):
(JSC::CallLinkStatus::isClosureCall):
(JSC::CallLinkStatus::makeClosureCall):
(JSC::CallLinkStatus::dump):
(JSC::CallLinkStatus::function): Deleted.
(JSC::CallLinkStatus::internalFunction): Deleted.
(JSC::CallLinkStatus::intrinsicFor): Deleted.

  • bytecode/CallLinkStatus.h:

(JSC::CallLinkStatus::CallLinkStatus):
(JSC::CallLinkStatus::isSet):
(JSC::CallLinkStatus::couldTakeSlowPath):
(JSC::CallLinkStatus::edges):
(JSC::CallLinkStatus::size):
(JSC::CallLinkStatus::at):
(JSC::CallLinkStatus::operator[]):
(JSC::CallLinkStatus::canOptimize):
(JSC::CallLinkStatus::canTrustCounts):
(JSC::CallLinkStatus::isClosureCall): Deleted.
(JSC::CallLinkStatus::callTarget): Deleted.
(JSC::CallLinkStatus::executable): Deleted.
(JSC::CallLinkStatus::makeClosureCall): Deleted.

  • bytecode/CallVariant.cpp: Added.

(JSC::CallVariant::dump):

  • bytecode/CallVariant.h: Added.

(JSC::CallVariant::CallVariant):
(JSC::CallVariant::operator!):
(JSC::CallVariant::despecifiedClosure):
(JSC::CallVariant::rawCalleeCell):
(JSC::CallVariant::internalFunction):
(JSC::CallVariant::function):
(JSC::CallVariant::isClosureCall):
(JSC::CallVariant::executable):
(JSC::CallVariant::nonExecutableCallee):
(JSC::CallVariant::intrinsicFor):
(JSC::CallVariant::functionExecutable):
(JSC::CallVariant::isHashTableDeletedValue):
(JSC::CallVariant::operator==):
(JSC::CallVariant::operator!=):
(JSC::CallVariant::operator<):
(JSC::CallVariant::operator>):
(JSC::CallVariant::operator<=):
(JSC::CallVariant::operator>=):
(JSC::CallVariant::hash):
(JSC::CallVariant::deletedToken):
(JSC::CallVariantHash::hash):
(JSC::CallVariantHash::equal):

  • bytecode/CodeOrigin.h:

(JSC::InlineCallFrame::isNormalCall):

  • bytecode/ExitKind.cpp:

(JSC::exitKindToString):

  • bytecode/ExitKind.h:
  • bytecode/GetByIdStatus.cpp:

(JSC::GetByIdStatus::computeForStubInfo):

  • bytecode/PutByIdStatus.cpp:

(JSC::PutByIdStatus::computeForStubInfo):

  • dfg/DFGAbstractInterpreterInlines.h:

(JSC::DFG::AbstractInterpreter<AbstractStateType>::executeEffects):

  • dfg/DFGBackwardsPropagationPhase.cpp:

(JSC::DFG::BackwardsPropagationPhase::propagate):

  • dfg/DFGBasicBlock.cpp:

(JSC::DFG::BasicBlock::~BasicBlock):

  • dfg/DFGBasicBlock.h:

(JSC::DFG::BasicBlock::takeLast):
(JSC::DFG::BasicBlock::didLink):

  • dfg/DFGByteCodeParser.cpp:

(JSC::DFG::ByteCodeParser::processSetLocalQueue):
(JSC::DFG::ByteCodeParser::removeLastNodeFromGraph):
(JSC::DFG::ByteCodeParser::addCallWithoutSettingResult):
(JSC::DFG::ByteCodeParser::addCall):
(JSC::DFG::ByteCodeParser::handleCall):
(JSC::DFG::ByteCodeParser::emitFunctionChecks):
(JSC::DFG::ByteCodeParser::undoFunctionChecks):
(JSC::DFG::ByteCodeParser::inliningCost):
(JSC::DFG::ByteCodeParser::inlineCall):
(JSC::DFG::ByteCodeParser::cancelLinkingForBlock):
(JSC::DFG::ByteCodeParser::attemptToInlineCall):
(JSC::DFG::ByteCodeParser::handleInlining):
(JSC::DFG::ByteCodeParser::handleConstantInternalFunction):
(JSC::DFG::ByteCodeParser::prepareToParseBlock):
(JSC::DFG::ByteCodeParser::clearCaches):
(JSC::DFG::ByteCodeParser::parseBlock):
(JSC::DFG::ByteCodeParser::linkBlock):
(JSC::DFG::ByteCodeParser::linkBlocks):
(JSC::DFG::ByteCodeParser::parseCodeBlock):

  • dfg/DFGCPSRethreadingPhase.cpp:

(JSC::DFG::CPSRethreadingPhase::freeUnnecessaryNodes):

  • dfg/DFGClobberize.h:

(JSC::DFG::clobberize):

  • dfg/DFGCommon.h:
  • dfg/DFGConstantFoldingPhase.cpp:

(JSC::DFG::ConstantFoldingPhase::foldConstants):

  • dfg/DFGDoesGC.cpp:

(JSC::DFG::doesGC):

  • dfg/DFGDriver.cpp:

(JSC::DFG::compileImpl):

  • dfg/DFGFixupPhase.cpp:

(JSC::DFG::FixupPhase::fixupNode):

  • dfg/DFGGraph.cpp:

(JSC::DFG::Graph::dump):
(JSC::DFG::Graph::visitChildren):

  • dfg/DFGJITCompiler.cpp:

(JSC::DFG::JITCompiler::link):

  • dfg/DFGLazyJSValue.cpp:

(JSC::DFG::LazyJSValue::switchLookupValue):

  • dfg/DFGLazyJSValue.h:

(JSC::DFG::LazyJSValue::switchLookupValue): Deleted.

  • dfg/DFGNode.cpp:

(WTF::printInternal):

  • dfg/DFGNode.h:

(JSC::DFG::OpInfo::OpInfo):
(JSC::DFG::Node::hasHeapPrediction):
(JSC::DFG::Node::hasCellOperand):
(JSC::DFG::Node::cellOperand):
(JSC::DFG::Node::setCellOperand):
(JSC::DFG::Node::canBeKnownFunction): Deleted.
(JSC::DFG::Node::hasKnownFunction): Deleted.
(JSC::DFG::Node::knownFunction): Deleted.
(JSC::DFG::Node::giveKnownFunction): Deleted.
(JSC::DFG::Node::hasFunction): Deleted.
(JSC::DFG::Node::function): Deleted.
(JSC::DFG::Node::hasExecutable): Deleted.
(JSC::DFG::Node::executable): Deleted.

  • dfg/DFGNodeType.h:
  • dfg/DFGPhantomCanonicalizationPhase.cpp:

(JSC::DFG::PhantomCanonicalizationPhase::run):

  • dfg/DFGPhantomRemovalPhase.cpp:

(JSC::DFG::PhantomRemovalPhase::run):

  • dfg/DFGPredictionPropagationPhase.cpp:

(JSC::DFG::PredictionPropagationPhase::propagate):

  • dfg/DFGSafeToExecute.h:

(JSC::DFG::safeToExecute):

  • dfg/DFGSpeculativeJIT.cpp:

(JSC::DFG::SpeculativeJIT::emitSwitch):

  • dfg/DFGSpeculativeJIT32_64.cpp:

(JSC::DFG::SpeculativeJIT::emitCall):
(JSC::DFG::SpeculativeJIT::compile):

  • dfg/DFGSpeculativeJIT64.cpp:

(JSC::DFG::SpeculativeJIT::emitCall):
(JSC::DFG::SpeculativeJIT::compile):

  • dfg/DFGStructureRegistrationPhase.cpp:

(JSC::DFG::StructureRegistrationPhase::run):

  • dfg/DFGTierUpCheckInjectionPhase.cpp:

(JSC::DFG::TierUpCheckInjectionPhase::run):
(JSC::DFG::TierUpCheckInjectionPhase::removeFTLProfiling):

  • dfg/DFGValidate.cpp:

(JSC::DFG::Validate::validate):

  • dfg/DFGWatchpointCollectionPhase.cpp:

(JSC::DFG::WatchpointCollectionPhase::handle):

  • ftl/FTLCapabilities.cpp:

(JSC::FTL::canCompile):

  • ftl/FTLLowerDFGToLLVM.cpp:

(JSC::FTL::ftlUnreachable):
(JSC::FTL::LowerDFGToLLVM::lower):
(JSC::FTL::LowerDFGToLLVM::compileNode):
(JSC::FTL::LowerDFGToLLVM::compileCheckCell):
(JSC::FTL::LowerDFGToLLVM::compileCheckBadCell):
(JSC::FTL::LowerDFGToLLVM::compileGetExecutable):
(JSC::FTL::LowerDFGToLLVM::compileNativeCallOrConstruct):
(JSC::FTL::LowerDFGToLLVM::compileSwitch):
(JSC::FTL::LowerDFGToLLVM::buildSwitch):
(JSC::FTL::LowerDFGToLLVM::compileCheckFunction): Deleted.
(JSC::FTL::LowerDFGToLLVM::compileCheckExecutable): Deleted.

  • heap/Heap.cpp:

(JSC::Heap::collect):

  • jit/AssemblyHelpers.h:

(JSC::AssemblyHelpers::storeValue):
(JSC::AssemblyHelpers::loadValue):

  • jit/CCallHelpers.h:

(JSC::CCallHelpers::setupArguments):

  • jit/GPRInfo.h:

(JSC::JSValueRegs::uses):

  • jit/JITCall.cpp:

(JSC::JIT::compileOpCall):

  • jit/JITCall32_64.cpp:

(JSC::JIT::compileOpCall):

  • runtime/Options.h:
  • runtime/VM.cpp:

(JSC::VM::ensureCallEdgeLog):

  • runtime/VM.h:
  • tests/stress/new-array-then-exit.js: Added.

(foo):

  • tests/stress/poly-call-exit-this.js: Added.
  • tests/stress/poly-call-exit.js: Added.

Source/WTF:


Add some power that I need for call edge profiling.

  • wtf/OwnPtr.h:

(WTF::OwnPtr<T>::createTransactionally):

  • wtf/Spectrum.h:

(WTF::Spectrum::add):
(WTF::Spectrum::addAll):
(WTF::Spectrum::get):
(WTF::Spectrum::size):
(WTF::Spectrum::KeyAndCount::KeyAndCount):
(WTF::Spectrum::clear):
(WTF::Spectrum::removeIf):

LayoutTests:

  • js/regress/script-tests/simple-poly-call-nested.js: Added.
  • js/regress/script-tests/simple-poly-call.js: Added.
  • js/regress/simple-poly-call-expected.txt: Added.
  • js/regress/simple-poly-call-nested-expected.txt: Added.
  • js/regress/simple-poly-call-nested.html: Added.
  • js/regress/simple-poly-call.html: Added.
Location:
trunk/Source/JavaScriptCore
Files:
10 added
54 edited

Legend:

Unmodified
Added
Removed

  • trunk/Source/JavaScriptCore/CMakeLists.txt

    r172930 r172940  
    6666    bytecode/BytecodeBasicBlock.cpp
    6767    bytecode/BytecodeLivenessAnalysis.cpp
     68    bytecode/CallEdge.cpp
     69    bytecode/CallEdgeProfile.cpp
    6870    bytecode/CallLinkInfo.cpp
    6971    bytecode/CallLinkStatus.cpp
     72    bytecode/CallVariant.cpp
    7073    bytecode/CodeBlock.cpp
    7174    bytecode/CodeBlockHash.cpp

  • trunk/Source/JavaScriptCore/ChangeLog

    r172932 r172940  
     12014-08-24  Filip Pizlo  <[email protected]>
     2
     3        FTL should be able to do polymorphic call inlining
     4        https://bugs.webkit.org/show_bug.cgi?id=135145
     5
     6        Reviewed by Geoffrey Garen.
     7       
     8        Added a log-based high-fidelity call edge profiler that runs in DFG JIT (and optionally
     9        baseline JIT) code. Used it to do precise polymorphic inlining in the FTL. Potential
     10        inlining sites use the call edge profile if it is available, but they will still fall back
     11        on the call inline cache and rare case counts if it's not. Polymorphic inlining means that
     12        multiple possible callees can be inlined with a switch to guard them. The slow path may
     13        either be an OSR exit or a virtual call.
     14       
     15        The call edge profiling added in this patch is very precise - it will tell you about every
     16        call that has ever happened. It took some effort to reduce the overhead of this profiling.
     17        This mostly involved ensuring that we don't do it unnecessarily. For example, we avoid it
     18        in the baseline JIT (you can conditionally enable it but it's off by default) and we only do
     19        it in the DFG JIT if we know that the regular inline cache profiling wasn't precise enough.
     20        I also experimented with reducing the precision of the profiling. This led to a significant
     21        reduction in the speed-up, so I avoided this approach. I also explored making log processing
     22        concurrent, but that didn't help. Also, I tested the overhead of the log processing and
     23        found that most of the overhead of this profiling is actually in putting things into the log
     24        rather than in processing the log - that part appears to be surprisingly cheap.
     25       
     26        Polymorphic inlining could be enabled in the DFG if we enabled baseline call edge profiling,
     27        and if we guarded such inlining sites with some profiling mechanism to detect
     28        polyvariant monomorphisation opportunities (where the callsite being inlined reveals that
     29        it's actually monomorphic).
     30       
     31        This is a ~28% speed-up on deltablue and a ~7% speed-up on richards, with small speed-ups on
     32        other programs as well. It's about a 2% speed-up on Octane version 2, and never a regression
     33        on anything we care about. Some aggregates, like V8Spider, see a regression. This is
     34        highlighting the increase in profiling overhead. But since this doesn't show up on any major
     35        score (code-load or SunSpider), it's probably not relevant.
     36       
     37        * CMakeLists.txt:
     38        * JavaScriptCore.vcxproj/JavaScriptCore.vcxproj:
     39        * JavaScriptCore.xcodeproj/project.pbxproj:
     40        * bytecode/CallEdge.cpp: Added.
     41        (JSC::CallEdge::dump):
     42        * bytecode/CallEdge.h: Added.
     43        (JSC::CallEdge::operator!):
     44        (JSC::CallEdge::callee):
     45        (JSC::CallEdge::count):
     46        (JSC::CallEdge::despecifiedClosure):
     47        (JSC::CallEdge::CallEdge):
     48        * bytecode/CallEdgeProfile.cpp: Added.
     49        (JSC::CallEdgeProfile::callEdges):
     50        (JSC::CallEdgeProfile::numCallsToKnownCells):
     51        (JSC::worthDespecifying):
     52        (JSC::CallEdgeProfile::worthDespecifying):
     53        (JSC::CallEdgeProfile::visitWeak):
     54        (JSC::CallEdgeProfile::addSlow):
     55        (JSC::CallEdgeProfile::mergeBack):
     56        (JSC::CallEdgeProfile::fadeByHalf):
     57        (JSC::CallEdgeLog::CallEdgeLog):
     58        (JSC::CallEdgeLog::~CallEdgeLog):
     59        (JSC::CallEdgeLog::isEnabled):
     60        (JSC::operationProcessCallEdgeLog):
     61        (JSC::CallEdgeLog::emitLogCode):
     62        (JSC::CallEdgeLog::processLog):
     63        * bytecode/CallEdgeProfile.h: Added.
     64        (JSC::CallEdgeProfile::numCallsToNotCell):
     65        (JSC::CallEdgeProfile::numCallsToUnknownCell):
     66        (JSC::CallEdgeProfile::totalCalls):
     67        * bytecode/CallEdgeProfileInlines.h: Added.
     68        (JSC::CallEdgeProfile::CallEdgeProfile):
     69        (JSC::CallEdgeProfile::add):
     70        * bytecode/CallLinkInfo.cpp:
     71        (JSC::CallLinkInfo::visitWeak):
     72        * bytecode/CallLinkInfo.h:
     73        * bytecode/CallLinkStatus.cpp:
     74        (JSC::CallLinkStatus::CallLinkStatus):
     75        (JSC::CallLinkStatus::computeFromLLInt):
     76        (JSC::CallLinkStatus::computeFor):
     77        (JSC::CallLinkStatus::computeExitSiteData):
     78        (JSC::CallLinkStatus::computeFromCallLinkInfo):
     79        (JSC::CallLinkStatus::computeFromCallEdgeProfile):
     80        (JSC::CallLinkStatus::computeDFGStatuses):
     81        (JSC::CallLinkStatus::isClosureCall):
     82        (JSC::CallLinkStatus::makeClosureCall):
     83        (JSC::CallLinkStatus::dump):
     84        (JSC::CallLinkStatus::function): Deleted.
     85        (JSC::CallLinkStatus::internalFunction): Deleted.
     86        (JSC::CallLinkStatus::intrinsicFor): Deleted.
     87        * bytecode/CallLinkStatus.h:
     88        (JSC::CallLinkStatus::CallLinkStatus):
     89        (JSC::CallLinkStatus::isSet):
     90        (JSC::CallLinkStatus::couldTakeSlowPath):
     91        (JSC::CallLinkStatus::edges):
     92        (JSC::CallLinkStatus::size):
     93        (JSC::CallLinkStatus::at):
     94        (JSC::CallLinkStatus::operator[]):
     95        (JSC::CallLinkStatus::canOptimize):
     96        (JSC::CallLinkStatus::canTrustCounts):
     97        (JSC::CallLinkStatus::isClosureCall): Deleted.
     98        (JSC::CallLinkStatus::callTarget): Deleted.
     99        (JSC::CallLinkStatus::executable): Deleted.
     100        (JSC::CallLinkStatus::makeClosureCall): Deleted.
     101        * bytecode/CallVariant.cpp: Added.
     102        (JSC::CallVariant::dump):
     103        * bytecode/CallVariant.h: Added.
     104        (JSC::CallVariant::CallVariant):
     105        (JSC::CallVariant::operator!):
     106        (JSC::CallVariant::despecifiedClosure):
     107        (JSC::CallVariant::rawCalleeCell):
     108        (JSC::CallVariant::internalFunction):
     109        (JSC::CallVariant::function):
     110        (JSC::CallVariant::isClosureCall):
     111        (JSC::CallVariant::executable):
     112        (JSC::CallVariant::nonExecutableCallee):
     113        (JSC::CallVariant::intrinsicFor):
     114        (JSC::CallVariant::functionExecutable):
     115        (JSC::CallVariant::isHashTableDeletedValue):
     116        (JSC::CallVariant::operator==):
     117        (JSC::CallVariant::operator!=):
     118        (JSC::CallVariant::operator<):
     119        (JSC::CallVariant::operator>):
     120        (JSC::CallVariant::operator<=):
     121        (JSC::CallVariant::operator>=):
     122        (JSC::CallVariant::hash):
     123        (JSC::CallVariant::deletedToken):
     124        (JSC::CallVariantHash::hash):
     125        (JSC::CallVariantHash::equal):
     126        * bytecode/CodeOrigin.h:
     127        (JSC::InlineCallFrame::isNormalCall):
     128        * bytecode/ExitKind.cpp:
     129        (JSC::exitKindToString):
     130        * bytecode/ExitKind.h:
     131        * bytecode/GetByIdStatus.cpp:
     132        (JSC::GetByIdStatus::computeForStubInfo):
     133        * bytecode/PutByIdStatus.cpp:
     134        (JSC::PutByIdStatus::computeForStubInfo):
     135        * dfg/DFGAbstractInterpreterInlines.h:
     136        (JSC::DFG::AbstractInterpreter<AbstractStateType>::executeEffects):
     137        * dfg/DFGBackwardsPropagationPhase.cpp:
     138        (JSC::DFG::BackwardsPropagationPhase::propagate):
     139        * dfg/DFGBasicBlock.cpp:
     140        (JSC::DFG::BasicBlock::~BasicBlock):
     141        * dfg/DFGBasicBlock.h:
     142        (JSC::DFG::BasicBlock::takeLast):
     143        (JSC::DFG::BasicBlock::didLink):
     144        * dfg/DFGByteCodeParser.cpp:
     145        (JSC::DFG::ByteCodeParser::processSetLocalQueue):
     146        (JSC::DFG::ByteCodeParser::removeLastNodeFromGraph):
     147        (JSC::DFG::ByteCodeParser::addCallWithoutSettingResult):
     148        (JSC::DFG::ByteCodeParser::addCall):
     149        (JSC::DFG::ByteCodeParser::handleCall):
     150        (JSC::DFG::ByteCodeParser::emitFunctionChecks):
     151        (JSC::DFG::ByteCodeParser::undoFunctionChecks):
     152        (JSC::DFG::ByteCodeParser::inliningCost):
     153        (JSC::DFG::ByteCodeParser::inlineCall):
     154        (JSC::DFG::ByteCodeParser::cancelLinkingForBlock):
     155        (JSC::DFG::ByteCodeParser::attemptToInlineCall):
     156        (JSC::DFG::ByteCodeParser::handleInlining):
     157        (JSC::DFG::ByteCodeParser::handleConstantInternalFunction):
     158        (JSC::DFG::ByteCodeParser::prepareToParseBlock):
     159        (JSC::DFG::ByteCodeParser::clearCaches):
     160        (JSC::DFG::ByteCodeParser::parseBlock):
     161        (JSC::DFG::ByteCodeParser::linkBlock):
     162        (JSC::DFG::ByteCodeParser::linkBlocks):
     163        (JSC::DFG::ByteCodeParser::parseCodeBlock):
     164        * dfg/DFGCPSRethreadingPhase.cpp:
     165        (JSC::DFG::CPSRethreadingPhase::freeUnnecessaryNodes):
     166        * dfg/DFGClobberize.h:
     167        (JSC::DFG::clobberize):
     168        * dfg/DFGCommon.h:
     169        * dfg/DFGConstantFoldingPhase.cpp:
     170        (JSC::DFG::ConstantFoldingPhase::foldConstants):
     171        * dfg/DFGDoesGC.cpp:
     172        (JSC::DFG::doesGC):
     173        * dfg/DFGDriver.cpp:
     174        (JSC::DFG::compileImpl):
     175        * dfg/DFGFixupPhase.cpp:
     176        (JSC::DFG::FixupPhase::fixupNode):
     177        * dfg/DFGGraph.cpp:
     178        (JSC::DFG::Graph::dump):
     179        (JSC::DFG::Graph::visitChildren):
     180        * dfg/DFGJITCompiler.cpp:
     181        (JSC::DFG::JITCompiler::link):
     182        * dfg/DFGLazyJSValue.cpp:
     183        (JSC::DFG::LazyJSValue::switchLookupValue):
     184        * dfg/DFGLazyJSValue.h:
     185        (JSC::DFG::LazyJSValue::switchLookupValue): Deleted.
     186        * dfg/DFGNode.cpp:
     187        (WTF::printInternal):
     188        * dfg/DFGNode.h:
     189        (JSC::DFG::OpInfo::OpInfo):
     190        (JSC::DFG::Node::hasHeapPrediction):
     191        (JSC::DFG::Node::hasCellOperand):
     192        (JSC::DFG::Node::cellOperand):
     193        (JSC::DFG::Node::setCellOperand):
     194        (JSC::DFG::Node::canBeKnownFunction): Deleted.
     195        (JSC::DFG::Node::hasKnownFunction): Deleted.
     196        (JSC::DFG::Node::knownFunction): Deleted.
     197        (JSC::DFG::Node::giveKnownFunction): Deleted.
     198        (JSC::DFG::Node::hasFunction): Deleted.
     199        (JSC::DFG::Node::function): Deleted.
     200        (JSC::DFG::Node::hasExecutable): Deleted.
     201        (JSC::DFG::Node::executable): Deleted.
     202        * dfg/DFGNodeType.h:
     203        * dfg/DFGPhantomCanonicalizationPhase.cpp:
     204        (JSC::DFG::PhantomCanonicalizationPhase::run):
     205        * dfg/DFGPhantomRemovalPhase.cpp:
     206        (JSC::DFG::PhantomRemovalPhase::run):
     207        * dfg/DFGPredictionPropagationPhase.cpp:
     208        (JSC::DFG::PredictionPropagationPhase::propagate):
     209        * dfg/DFGSafeToExecute.h:
     210        (JSC::DFG::safeToExecute):
     211        * dfg/DFGSpeculativeJIT.cpp:
     212        (JSC::DFG::SpeculativeJIT::emitSwitch):
     213        * dfg/DFGSpeculativeJIT32_64.cpp:
     214        (JSC::DFG::SpeculativeJIT::emitCall):
     215        (JSC::DFG::SpeculativeJIT::compile):
     216        * dfg/DFGSpeculativeJIT64.cpp:
     217        (JSC::DFG::SpeculativeJIT::emitCall):
     218        (JSC::DFG::SpeculativeJIT::compile):
     219        * dfg/DFGStructureRegistrationPhase.cpp:
     220        (JSC::DFG::StructureRegistrationPhase::run):
     221        * dfg/DFGTierUpCheckInjectionPhase.cpp:
     222        (JSC::DFG::TierUpCheckInjectionPhase::run):
     223        (JSC::DFG::TierUpCheckInjectionPhase::removeFTLProfiling):
     224        * dfg/DFGValidate.cpp:
     225        (JSC::DFG::Validate::validate):
     226        * dfg/DFGWatchpointCollectionPhase.cpp:
     227        (JSC::DFG::WatchpointCollectionPhase::handle):
     228        * ftl/FTLCapabilities.cpp:
     229        (JSC::FTL::canCompile):
     230        * ftl/FTLLowerDFGToLLVM.cpp:
     231        (JSC::FTL::ftlUnreachable):
     232        (JSC::FTL::LowerDFGToLLVM::lower):
     233        (JSC::FTL::LowerDFGToLLVM::compileNode):
     234        (JSC::FTL::LowerDFGToLLVM::compileCheckCell):
     235        (JSC::FTL::LowerDFGToLLVM::compileCheckBadCell):
     236        (JSC::FTL::LowerDFGToLLVM::compileGetExecutable):
     237        (JSC::FTL::LowerDFGToLLVM::compileNativeCallOrConstruct):
     238        (JSC::FTL::LowerDFGToLLVM::compileSwitch):
     239        (JSC::FTL::LowerDFGToLLVM::buildSwitch):
     240        (JSC::FTL::LowerDFGToLLVM::compileCheckFunction): Deleted.
     241        (JSC::FTL::LowerDFGToLLVM::compileCheckExecutable): Deleted.
     242        * heap/Heap.cpp:
     243        (JSC::Heap::collect):
     244        * jit/AssemblyHelpers.h:
     245        (JSC::AssemblyHelpers::storeValue):
     246        (JSC::AssemblyHelpers::loadValue):
     247        * jit/CCallHelpers.h:
     248        (JSC::CCallHelpers::setupArguments):
     249        * jit/GPRInfo.h:
     250        (JSC::JSValueRegs::uses):
     251        * jit/JITCall.cpp:
     252        (JSC::JIT::compileOpCall):
     253        * jit/JITCall32_64.cpp:
     254        (JSC::JIT::compileOpCall):
     255        * runtime/Options.h:
     256        * runtime/VM.cpp:
     257        (JSC::VM::ensureCallEdgeLog):
     258        * runtime/VM.h:
     259        * tests/stress/new-array-then-exit.js: Added.
     260        (foo):
     261        * tests/stress/poly-call-exit-this.js: Added.
     262        * tests/stress/poly-call-exit.js: Added.
     263
    12642014-08-22  Michael Saboff  <[email protected]>
    2265

  • trunk/Source/JavaScriptCore/JavaScriptCore.vcxproj/JavaScriptCore.vcxproj

    r172930 r172940  
    315315    <ClCompile Include="..\bytecode\BytecodeBasicBlock.cpp" />
    316316    <ClCompile Include="..\bytecode\BytecodeLivenessAnalysis.cpp" />
     317    <ClCompile Include="..\bytecode\CallEdge.cpp" />
     318    <ClCompile Include="..\bytecode\CallEdgeProfile.cpp" />
    317319    <ClCompile Include="..\bytecode\CallLinkInfo.cpp" />
    318320    <ClCompile Include="..\bytecode\CallLinkStatus.cpp" />
     321    <ClCompile Include="..\bytecode\CallVariant.cpp" />
    319322    <ClCompile Include="..\bytecode\CodeBlock.cpp" />
    320323    <ClCompile Include="..\bytecode\CodeBlockHash.cpp" />
    … …  
    906909    <ClInclude Include="..\bytecode\BytecodeLivenessAnalysis.h" />
    907910    <ClInclude Include="..\bytecode\BytecodeUseDef.h" />
     911    <ClInclude Include="..\bytecode\CallEdge.h" />
     912    <ClInclude Include="..\bytecode\CallEdgeProfile.h" />
     913    <ClInclude Include="..\bytecode\CallEdgeProfileInlines.h" />
    908914    <ClInclude Include="..\bytecode\CallLinkInfo.h" />
    909915    <ClInclude Include="..\bytecode\CallLinkStatus.h" />
    910916    <ClInclude Include="..\bytecode\CallReturnOffsetToBytecodeOffset.h" />
     917    <ClInclude Include="..\bytecode\CallVariant.h" />
    911918    <ClInclude Include="..\bytecode\CodeBlock.h" />
    912919    <ClInclude Include="..\bytecode\CodeBlockHash.h" />

  • trunk/Source/JavaScriptCore/JavaScriptCore.xcodeproj/project.pbxproj

    r172930 r172940  
    264264                0F3B3A2B15475000003ED0FF /* DFGValidate.cpp in Sources */ = {isa = PBXBuildFile; fileRef = 0F3B3A2915474FF4003ED0FF /* DFGValidate.cpp */; };
    265265                0F3B3A2C15475002003ED0FF /* DFGValidate.h in Headers */ = {isa = PBXBuildFile; fileRef = 0F3B3A2A15474FF4003ED0FF /* DFGValidate.h */; settings = {ATTRIBUTES = (Private, ); }; };
     266                0F3B7E2619A11B8000D9BC56 /* CallEdge.h in Headers */ = {isa = PBXBuildFile; fileRef = 0F3B7E2019A11B8000D9BC56 /* CallEdge.h */; settings = {ATTRIBUTES = (Private, ); }; };
     267                0F3B7E2719A11B8000D9BC56 /* CallEdgeProfile.cpp in Sources */ = {isa = PBXBuildFile; fileRef = 0F3B7E2119A11B8000D9BC56 /* CallEdgeProfile.cpp */; };
     268                0F3B7E2819A11B8000D9BC56 /* CallEdgeProfile.h in Headers */ = {isa = PBXBuildFile; fileRef = 0F3B7E2219A11B8000D9BC56 /* CallEdgeProfile.h */; settings = {ATTRIBUTES = (Private, ); }; };
     269                0F3B7E2919A11B8000D9BC56 /* CallEdgeProfileInlines.h in Headers */ = {isa = PBXBuildFile; fileRef = 0F3B7E2319A11B8000D9BC56 /* CallEdgeProfileInlines.h */; settings = {ATTRIBUTES = (Private, ); }; };
     270                0F3B7E2A19A11B8000D9BC56 /* CallVariant.cpp in Sources */ = {isa = PBXBuildFile; fileRef = 0F3B7E2419A11B8000D9BC56 /* CallVariant.cpp */; };
     271                0F3B7E2B19A11B8000D9BC56 /* CallVariant.h in Headers */ = {isa = PBXBuildFile; fileRef = 0F3B7E2519A11B8000D9BC56 /* CallVariant.h */; settings = {ATTRIBUTES = (Private, ); }; };
     272                0F3B7E2D19A12AAE00D9BC56 /* CallEdge.cpp in Sources */ = {isa = PBXBuildFile; fileRef = 0F3B7E2C19A12AAE00D9BC56 /* CallEdge.cpp */; };
    266273                0F3D0BBC194A414300FC9CF9 /* ConstantStructureCheck.cpp in Sources */ = {isa = PBXBuildFile; fileRef = 0F3D0BBA194A414300FC9CF9 /* ConstantStructureCheck.cpp */; };
    267274                0F3D0BBD194A414300FC9CF9 /* ConstantStructureCheck.h in Headers */ = {isa = PBXBuildFile; fileRef = 0F3D0BBB194A414300FC9CF9 /* ConstantStructureCheck.h */; settings = {ATTRIBUTES = (Private, ); }; };
    … …  
    21702177                0F3B3A2915474FF4003ED0FF /* DFGValidate.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; name = DFGValidate.cpp; path = dfg/DFGValidate.cpp; sourceTree = "<group>"; };
    21712178                0F3B3A2A15474FF4003ED0FF /* DFGValidate.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; name = DFGValidate.h; path = dfg/DFGValidate.h; sourceTree = "<group>"; };
     2179                0F3B7E2019A11B8000D9BC56 /* CallEdge.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = CallEdge.h; sourceTree = "<group>"; };
     2180                0F3B7E2119A11B8000D9BC56 /* CallEdgeProfile.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = CallEdgeProfile.cpp; sourceTree = "<group>"; };
     2181                0F3B7E2219A11B8000D9BC56 /* CallEdgeProfile.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = CallEdgeProfile.h; sourceTree = "<group>"; };
     2182                0F3B7E2319A11B8000D9BC56 /* CallEdgeProfileInlines.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = CallEdgeProfileInlines.h; sourceTree = "<group>"; };
     2183                0F3B7E2419A11B8000D9BC56 /* CallVariant.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = CallVariant.cpp; sourceTree = "<group>"; };
     2184                0F3B7E2519A11B8000D9BC56 /* CallVariant.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = CallVariant.h; sourceTree = "<group>"; };
     2185                0F3B7E2C19A12AAE00D9BC56 /* CallEdge.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = CallEdge.cpp; sourceTree = "<group>"; };
    21722186                0F3D0BBA194A414300FC9CF9 /* ConstantStructureCheck.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = ConstantStructureCheck.cpp; sourceTree = "<group>"; };
    21732187                0F3D0BBB194A414300FC9CF9 /* ConstantStructureCheck.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = ConstantStructureCheck.h; sourceTree = "<group>"; };
    … …  
    51475161                                0F885E101849A3BE00F1E3FA /* BytecodeUseDef.h */,
    51485162                                0F8023E91613832300A0BA45 /* ByValInfo.h */,
     5163                                0F3B7E2C19A12AAE00D9BC56 /* CallEdge.cpp */,
     5164                                0F3B7E2019A11B8000D9BC56 /* CallEdge.h */,
     5165                                0F3B7E2119A11B8000D9BC56 /* CallEdgeProfile.cpp */,
     5166                                0F3B7E2219A11B8000D9BC56 /* CallEdgeProfile.h */,
     5167                                0F3B7E2319A11B8000D9BC56 /* CallEdgeProfileInlines.h */,
    51495168                                0F0B83AE14BCF71400885B4F /* CallLinkInfo.cpp */,
    51505169                                0F0B83AF14BCF71400885B4F /* CallLinkInfo.h */,
    … …  
    51525171                                0F93329414CA7DC10085F3C6 /* CallLinkStatus.h */,
    51535172                                0F0B83B814BCF95B00885B4F /* CallReturnOffsetToBytecodeOffset.h */,
     5173                                0F3B7E2419A11B8000D9BC56 /* CallVariant.cpp */,
     5174                                0F3B7E2519A11B8000D9BC56 /* CallVariant.h */,
    51545175                                969A07900ED1D3AE00F1F681 /* CodeBlock.cpp */,
    51555176                                969A07910ED1D3AE00F1F681 /* CodeBlock.h */,
    … …  
    58805901                                0F2FC77316E12F740038D976 /* DFGDCEPhase.h in Headers */,
    58815902                                0F8F2B9A172F0501007DBDA5 /* DFGDesiredIdentifiers.h in Headers */,
     5903                                0F3B7E2819A11B8000D9BC56 /* CallEdgeProfile.h in Headers */,
    58825904                                C2C0F7CE17BBFC5B00464FE4 /* DFGDesiredTransitions.h in Headers */,
    58835905                                0FE8534C1723CDA500B618F5 /* DFGDesiredWatchpoints.h in Headers */,
    … …  
    61486170                                0F766D2C15A8CC3A008F363E /* JITStubRoutineSet.h in Headers */,
    61496171                                14C5242B0F5355E900BA3D04 /* JITStubs.h in Headers */,
     6172                                0F3B7E2B19A11B8000D9BC56 /* CallVariant.h in Headers */,
    61506173                                FEF6835E174343CC00A32E25 /* JITStubsARM.h in Headers */,
    61516174                                FEF6835F174343CC00A32E25 /* JITStubsARMv7.h in Headers */,
    … …  
    61596182                                BC18C4160E16F5CD00B34460 /* JSActivation.h in Headers */,
    61606183                                840480131021A1D9008E7F01 /* JSAPIValueWrapper.h in Headers */,
     6184                                0F3B7E2919A11B8000D9BC56 /* CallEdgeProfileInlines.h in Headers */,
    61616185                                C2CF39C216E15A8100DD69BE /* JSAPIWrapperObject.h in Headers */,
    61626186                                A76140D2182982CB00750624 /* JSArgumentsIterator.h in Headers */,
    … …  
    64746498                                E49DC16D12EF295300184A1F /* SourceProviderCacheItem.h in Headers */,
    64756499                                0FB7F39E15ED8E4600F167B2 /* SparseArrayValueMap.h in Headers */,
     6500                                0F3B7E2619A11B8000D9BC56 /* CallEdge.h in Headers */,
    64766501                                A7386554118697B400540279 /* SpecializedThunkJIT.h in Headers */,
    64776502                                0F5541B21613C1FB00CE3E25 /* SpecialPointer.h in Headers */,
    … …  
    73287353                                0F235BD817178E1C00690C7F /* FTLExitThunkGenerator.cpp in Sources */,
    73297354                                0F235BDA17178E1C00690C7F /* FTLExitValue.cpp in Sources */,
     7355                                0F3B7E2719A11B8000D9BC56 /* CallEdgeProfile.cpp in Sources */,
    73307356                                A7F2996B17A0BB670010417A /* FTLFail.cpp in Sources */,
    73317357                                0FD8A31917D51F2200CA2C40 /* FTLForOSREntryJITCode.cpp in Sources */,
    … …  
    75147540                                0F38B01117CF078000B144D3 /* LLIntEntrypoint.cpp in Sources */,
    75157541                                0F4680A814BA7FAB00BFE272 /* LLIntExceptions.cpp in Sources */,
     7542                                0F3B7E2D19A12AAE00D9BC56 /* CallEdge.cpp in Sources */,
    75167543                                0F4680A414BA7F8D00BFE272 /* LLIntSlowPaths.cpp in Sources */,
    75177544                                0F0B839C14BCF46300885B4F /* LLIntThunks.cpp in Sources */,
    … …  
    76757702                                2A4EC90B1860D6C20094F782 /* WriteBarrierBuffer.cpp in Sources */,
    76767703                                0FC8150B14043C0E00CFA603 /* WriteBarrierSupport.cpp in Sources */,
     7704                                0F3B7E2A19A11B8000D9BC56 /* CallVariant.cpp in Sources */,
    76777705                                A7E5AB3A1799E4B200D2833D /* X86Disassembler.cpp in Sources */,
    76787706                                863C6D9C1521111A00585E4E /* YarrCanonicalizeUCS2.cpp in Sources */,

  • trunk/Source/JavaScriptCore/bytecode/CallLinkInfo.cpp

    r172176 r172940  
    8484    if (!!lastSeenCallee && !Heap::isMarked(lastSeenCallee.get()))
    8585        lastSeenCallee.clear();
     86   
     87    if (callEdgeProfile) {
     88        WTF::loadLoadFence();
     89        callEdgeProfile->visitWeak();
     90    }
    8691}
    8792

  • trunk/Source/JavaScriptCore/bytecode/CallLinkInfo.h

    r166392 r172940  
    2727#define CallLinkInfo_h
    2828
     29#include "CallEdgeProfile.h"
    2930#include "ClosureCallStubRoutine.h"
    3031#include "CodeLocation.h"
    … …  
    3435#include "Opcode.h"
    3536#include "WriteBarrier.h"
     37#include <wtf/OwnPtr.h>
    3638#include <wtf/SentinelLinkedList.h>
    3739
    … …  
    8991    unsigned slowPathCount;
    9092    CodeOrigin codeOrigin;
     93    OwnPtr<CallEdgeProfile> callEdgeProfile;
    9194
    9295    bool isLinked() { return stub || callee; }

  • trunk/Source/JavaScriptCore/bytecode/CallLinkStatus.cpp

    r172176 r172940  
    3333#include "JSCInlines.h"
    3434#include <wtf/CommaPrinter.h>
     35#include <wtf/ListDump.h>
    3536
    3637namespace JSC {
    … …  
    3940
    4041CallLinkStatus::CallLinkStatus(JSValue value)
    41     : m_callTarget(value)
    42     , m_executable(0)
    43     , m_couldTakeSlowPath(false)
     42    : m_couldTakeSlowPath(false)
    4443    , m_isProved(false)
    4544{
    46     if (!value || !value.isCell())
     45    if (!value || !value.isCell()) {
     46        m_couldTakeSlowPath = true;
    4747        return;
    48    
    49     if (!value.asCell()->inherits(JSFunction::info()))
    50         return;
    51    
    52     m_executable = jsCast<JSFunction*>(value.asCell())->executable();
    53 }
    54 
    55 JSFunction* CallLinkStatus::function() const
    56 {
    57     if (!m_callTarget || !m_callTarget.isCell())
    58         return 0;
    59    
    60     if (!m_callTarget.asCell()->inherits(JSFunction::info()))
    61         return 0;
    62    
    63     return jsCast<JSFunction*>(m_callTarget.asCell());
    64 }
    65 
    66 InternalFunction* CallLinkStatus::internalFunction() const
    67 {
    68     if (!m_callTarget || !m_callTarget.isCell())
    69         return 0;
    70    
    71     if (!m_callTarget.asCell()->inherits(InternalFunction::info()))
    72         return 0;
    73    
    74     return jsCast<InternalFunction*>(m_callTarget.asCell());
    75 }
    76 
    77 Intrinsic CallLinkStatus::intrinsicFor(CodeSpecializationKind kind) const
    78 {
    79     if (!m_executable)
    80         return NoIntrinsic;
    81    
    82     return m_executable->intrinsicFor(kind);
     48    }
     49   
     50    m_edges.append(CallEdge(CallVariant(value.asCell()), 1));
    8351}
    8452
    … …  
    8856    UNUSED_PARAM(bytecodeIndex);
    8957#if ENABLE(DFG_JIT)
    90     if (profiledBlock->hasExitSite(locker, DFG::FrequentExitSite(bytecodeIndex, BadFunction))) {
     58    if (profiledBlock->hasExitSite(locker, DFG::FrequentExitSite(bytecodeIndex, BadCell))) {
    9159        // We could force this to be a closure call, but instead we'll just assume that it
    9260        // takes slow path.
    … …  
    12694        return computeFromLLInt(locker, profiledBlock, bytecodeIndex);
    12795   
    128     return computeFor(locker, *callLinkInfo, exitSiteData);
     96    return computeFor(locker, profiledBlock, *callLinkInfo, exitSiteData);
    12997#else
    13098    return CallLinkStatus();
    … …  
    140108#if ENABLE(DFG_JIT)
    141109    exitSiteData.m_takesSlowPath =
    142         profiledBlock->hasExitSite(locker, DFG::FrequentExitSite(bytecodeIndex, BadCache, exitingJITType))
     110        profiledBlock->hasExitSite(locker, DFG::FrequentExitSite(bytecodeIndex, BadType, exitingJITType))
    143111        || profiledBlock->hasExitSite(locker, DFG::FrequentExitSite(bytecodeIndex, BadExecutable, exitingJITType));
    144112    exitSiteData.m_badFunction =
    145         profiledBlock->hasExitSite(locker, DFG::FrequentExitSite(bytecodeIndex, BadFunction, exitingJITType));
     113        profiledBlock->hasExitSite(locker, DFG::FrequentExitSite(bytecodeIndex, BadCell, exitingJITType));
    146114#else
    147115    UNUSED_PARAM(locker);
    … …  
    155123
    156124#if ENABLE(JIT)
    157 CallLinkStatus CallLinkStatus::computeFor(const ConcurrentJITLocker&, CallLinkInfo& callLinkInfo)
     125CallLinkStatus CallLinkStatus::computeFor(
     126    const ConcurrentJITLocker& locker, CodeBlock* profiledBlock, CallLinkInfo& callLinkInfo)
     127{
     128    // We don't really need this, but anytime we have to debug this code, it becomes indispensable.
     129    UNUSED_PARAM(profiledBlock);
     130   
     131    if (Options::callStatusShouldUseCallEdgeProfile()) {
     132        // Always trust the call edge profile over anything else since this has precise counts.
     133        // It can make the best possible decision because it never "forgets" what happened for any
     134        // call, with the exception of fading out the counts of old calls (for example if the
     135        // counter type is 16-bit then calls that happened more than 2^16 calls ago are given half
     136        // weight, and this compounds for every 2^15 [sic] calls after that). The combination of
     137        // high fidelity for recent calls and fading for older calls makes this the most useful
     138        // mechamism of choosing how to optimize future calls.
     139        CallEdgeProfile* edgeProfile = callLinkInfo.callEdgeProfile.get();
     140        WTF::loadLoadFence();
     141        if (edgeProfile) {
     142            CallLinkStatus result = computeFromCallEdgeProfile(edgeProfile);
     143            if (!!result)
     144                return result;
     145        }
     146    }
     147   
     148    return computeFromCallLinkInfo(locker, callLinkInfo);
     149}
     150
     151CallLinkStatus CallLinkStatus::computeFromCallLinkInfo(
     152    const ConcurrentJITLocker&, CallLinkInfo& callLinkInfo)
    158153{
    159154    // Note that despite requiring that the locker is held, this code is racy with respect
    … …  
    178173    JSFunction* target = callLinkInfo.lastSeenCallee.get();
    179174    if (!target)
    180         return CallLinkStatus();
     175        return takesSlowPath();
    181176   
    182177    if (callLinkInfo.hasSeenClosure)
    … …  
    186181}
    187182
     183CallLinkStatus CallLinkStatus::computeFromCallEdgeProfile(CallEdgeProfile* edgeProfile)
     184{
     185    // In cases where the call edge profile saw nothing, use the CallLinkInfo instead.
     186    if (!edgeProfile->totalCalls())
     187        return CallLinkStatus();
     188   
     189    // To do anything meaningful, we require that the majority of calls are to something we
     190    // know how to handle.
     191    unsigned numCallsToKnown = edgeProfile->numCallsToKnownCells();
     192    unsigned numCallsToUnknown = edgeProfile->numCallsToNotCell() + edgeProfile->numCallsToUnknownCell();
     193   
     194    // We require that the majority of calls were to something that we could possibly inline.
     195    if (numCallsToKnown <= numCallsToUnknown)
     196        return takesSlowPath();
     197   
     198    // We require that the number of such calls is greater than some minimal threshold, so that we
     199    // avoid inlining completely cold calls.
     200    if (numCallsToKnown < Options::frequentCallThreshold())
     201        return takesSlowPath();
     202   
     203    CallLinkStatus result;
     204    result.m_edges = edgeProfile->callEdges();
     205    result.m_couldTakeSlowPath = !!numCallsToUnknown;
     206    result.m_canTrustCounts = true;
     207   
     208    return result;
     209}
     210
    188211CallLinkStatus CallLinkStatus::computeFor(
    189     const ConcurrentJITLocker& locker, CallLinkInfo& callLinkInfo, ExitSiteData exitSiteData)
    190 {
    191     if (exitSiteData.m_takesSlowPath)
    192         return takesSlowPath();
    193    
    194     CallLinkStatus result = computeFor(locker, callLinkInfo);
     212    const ConcurrentJITLocker& locker, CodeBlock* profiledBlock, CallLinkInfo& callLinkInfo,
     213    ExitSiteData exitSiteData)
     214{
     215    CallLinkStatus result = computeFor(locker, profiledBlock, callLinkInfo);
    195216    if (exitSiteData.m_badFunction)
    196217        result.makeClosureCall();
     218    if (exitSiteData.m_takesSlowPath)
     219        result.m_couldTakeSlowPath = true;
    197220   
    198221    return result;
    … …  
    228251        {
    229252            ConcurrentJITLocker locker(dfgCodeBlock->m_lock);
    230             map.add(info.codeOrigin, computeFor(locker, info, exitSiteData));
     253            map.add(info.codeOrigin, computeFor(locker, dfgCodeBlock, info, exitSiteData));
    231254        }
    232255    }
    … …  
    257280}
    258281
     282bool CallLinkStatus::isClosureCall() const
     283{
     284    for (unsigned i = m_edges.size(); i--;) {
     285        if (m_edges[i].callee().isClosureCall())
     286            return true;
     287    }
     288    return false;
     289}
     290
     291void CallLinkStatus::makeClosureCall()
     292{
     293    ASSERT(!m_isProved);
     294    for (unsigned i = m_edges.size(); i--;)
     295        m_edges[i] = m_edges[i].despecifiedClosure();
     296   
     297    if (!ASSERT_DISABLED) {
     298        // Doing this should not have created duplicates, because the CallEdgeProfile
     299        // should despecify closures if doing so would reduce the number of known callees.
     300        for (unsigned i = 0; i < m_edges.size(); ++i) {
     301            for (unsigned j = i + 1; j < m_edges.size(); ++j)
     302                ASSERT(m_edges[i].callee() != m_edges[j].callee());
     303        }
     304    }
     305}
     306
    259307void CallLinkStatus::dump(PrintStream& out) const
    260308{
    … …  
    272320        out.print(comma, "Could Take Slow Path");
    273321   
    274     if (m_callTarget)
    275         out.print(comma, "Known target: ", m_callTarget);
    276    
    277     if (m_executable) {
    278         out.print(comma, "Executable/CallHash: ", RawPointer(m_executable));
    279         if (!isCompilationThread())
    280             out.print("/", m_executable->hashFor(CodeForCall));
    281     }
     322    out.print(listDump(m_edges));
    282323}
    283324

  • trunk/Source/JavaScriptCore/bytecode/CallLinkStatus.h

    r172176 r172940  
    4747public:
    4848    CallLinkStatus()
    49         : m_executable(0)
    50         , m_couldTakeSlowPath(false)
     49        : m_couldTakeSlowPath(false)
    5150        , m_isProved(false)
     51        , m_canTrustCounts(false)
    5252    {
    5353    }
    … …  
    6262    explicit CallLinkStatus(JSValue);
    6363   
    64     CallLinkStatus(ExecutableBase* executable)
    65         : m_executable(executable)
     64    CallLinkStatus(CallVariant variant)
     65        : m_edges(1, CallEdge(variant, 1))
    6666        , m_couldTakeSlowPath(false)
    6767        , m_isProved(false)
     68        , m_canTrustCounts(false)
    6869    {
    6970    }
    … …  
    9394    // Computes the status assuming that we never took slow path and never previously
    9495    // exited.
    95     static CallLinkStatus computeFor(const ConcurrentJITLocker&, CallLinkInfo&);
    96     static CallLinkStatus computeFor(const ConcurrentJITLocker&, CallLinkInfo&, ExitSiteData);
     96    static CallLinkStatus computeFor(const ConcurrentJITLocker&, CodeBlock*, CallLinkInfo&);
     97    static CallLinkStatus computeFor(
     98        const ConcurrentJITLocker&, CodeBlock*, CallLinkInfo&, ExitSiteData);
    9799#endif
    98100   
    … …  
    108110        CodeBlock*, CodeOrigin, const CallLinkInfoMap&, const ContextMap&);
    109111   
    110     bool isSet() const { return m_callTarget || m_executable || m_couldTakeSlowPath; }
     112    bool isSet() const { return !m_edges.isEmpty() || m_couldTakeSlowPath; }
    111113   
    112114    bool operator!() const { return !isSet(); }
    113115   
    114116    bool couldTakeSlowPath() const { return m_couldTakeSlowPath; }
    115     bool isClosureCall() const { return m_executable && !m_callTarget; }
    116117   
    117     JSValue callTarget() const { return m_callTarget; }
    118     JSFunction* function() const;
    119     InternalFunction* internalFunction() const;
    120     Intrinsic intrinsicFor(CodeSpecializationKind) const;
    121     ExecutableBase* executable() const { return m_executable; }
     118    CallEdgeList edges() const { return m_edges; }
     119    unsigned size() const { return m_edges.size(); }
     120    CallEdge at(unsigned i) const { return m_edges[i]; }
     121    CallEdge operator[](unsigned i) const { return at(i); }
    122122    bool isProved() const { return m_isProved; }
    123     bool canOptimize() const { return (m_callTarget || m_executable) && !m_couldTakeSlowPath; }
     123    bool canOptimize() const { return !m_edges.isEmpty(); }
     124    bool canTrustCounts() const { return m_canTrustCounts; }
     125   
     126    bool isClosureCall() const; // Returns true if any callee is a closure call.
    124127   
    125128    void dump(PrintStream&) const;
    126129   
    127130private:
    128     void makeClosureCall()
    129     {
    130         ASSERT(!m_isProved);
    131         // Turn this into a closure call.
    132         m_callTarget = JSValue();
    133     }
     131    void makeClosureCall();
    134132   
    135133    static CallLinkStatus computeFromLLInt(const ConcurrentJITLocker&, CodeBlock*, unsigned bytecodeIndex);
     134#if ENABLE(JIT)
     135    static CallLinkStatus computeFromCallEdgeProfile(CallEdgeProfile*);
     136    static CallLinkStatus computeFromCallLinkInfo(
     137        const ConcurrentJITLocker&, CallLinkInfo&);
     138#endif
    136139   
    137     JSValue m_callTarget;
    138     ExecutableBase* m_executable;
     140    CallEdgeList m_edges;
    139141    bool m_couldTakeSlowPath;
    140142    bool m_isProved;
     143    bool m_canTrustCounts;
    141144};
    142145

  • trunk/Source/JavaScriptCore/bytecode/CodeOrigin.h

    r172853 r172940  
    155155    }
    156156   
     157    static bool isNormalCall(Kind kind)
     158    {
     159        switch (kind) {
     160        case Call:
     161        case Construct:
     162            return true;
     163        default:
     164            return false;
     165        }
     166    }
     167   
    157168    Vector<ValueRecovery> arguments; // Includes 'this'.
    158169    WriteBarrier<ScriptExecutable> executable;

  • trunk/Source/JavaScriptCore/bytecode/ExitKind.cpp

    r171613 r172940  
    3939    case BadType:
    4040        return "BadType";
    41     case BadFunction:
    42         return "BadFunction";
     41    case BadCell:
     42        return "BadCell";
    4343    case BadExecutable:
    4444        return "BadExecutable";

  • trunk/Source/JavaScriptCore/bytecode/ExitKind.h

    r171613 r172940  
    3232    ExitKindUnset,
    3333    BadType, // We exited because a type prediction was wrong.
    34     BadFunction, // We exited because we made an incorrect assumption about what function we would see.
     34    BadCell, // We exited because we made an incorrect assumption about what cell we would see. Usually used for function checks.
    3535    BadExecutable, // We exited because we made an incorrect assumption about what executable we would see.
    3636    BadCache, // We exited because an inline cache was wrong.

  • trunk/Source/JavaScriptCore/bytecode/GetByIdStatus.cpp

    r172129 r172940  
    188188                        list->at(listIndex).stubRoutine());
    189189                    callLinkStatus = std::make_unique<CallLinkStatus>(
    190                         CallLinkStatus::computeFor(locker, *stub->m_callLinkInfo, callExitSiteData));
     190                        CallLinkStatus::computeFor(
     191                            locker, profiledBlock, *stub->m_callLinkInfo, callExitSiteData));
    191192                    break;
    192193                }

  • trunk/Source/JavaScriptCore/bytecode/PutByIdStatus.cpp

    r172129 r172940  
    248248                        std::make_unique<CallLinkStatus>(
    249249                            CallLinkStatus::computeFor(
    250                                 locker, *stub->m_callLinkInfo, callExitSiteData));
     250                                locker, profiledBlock, *stub->m_callLinkInfo, callExitSiteData));
    251251                   
    252252                    variant = PutByIdVariant::setter(

  • trunk/Source/JavaScriptCore/dfg/DFGAbstractInterpreterInlines.h

    r172808 r172940  
    14601460        break;
    14611461       
    1462     case CheckExecutable: {
    1463         // FIXME: We could track executables in AbstractValue, which would allow us to get rid of these checks
    1464         // more thoroughly. https://bugs.webkit.org/show_bug.cgi?id=106200
    1465         // FIXME: We could eliminate these entirely if we know the exact value that flows into this.
    1466         // https://bugs.webkit.org/show_bug.cgi?id=106201
    1467         break;
    1468     }
    1469 
    14701462    case CheckStructure: {
    14711463        // FIXME: We should be able to propagate the structure sets of constants (i.e. prototypes).
    … …  
    17271719        break;
    17281720    }
     1721       
     1722    case GetExecutable: {
     1723        JSValue value = forNode(node->child1()).value();
     1724        if (value) {
     1725            JSFunction* function = jsDynamicCast<JSFunction*>(value);
     1726            if (function) {
     1727                setConstant(node, *m_graph.freeze(function->executable()));
     1728                break;
     1729            }
     1730        }
     1731        forNode(node).setType(SpecCellOther);
     1732        break;
     1733    }
    17291734   
    1730     case CheckFunction: {
     1735    case CheckCell: {
    17311736        JSValue value = forNode(node->child1()).value();
    1732         if (value == node->function()->value()) {
     1737        if (value == node->cellOperand()->value()) {
    17331738            m_state.setFoundConstants(true);
    17341739            ASSERT(value);
    … …  
    17361741        }
    17371742       
    1738         filterByValue(node->child1(), *node->function());
     1743        filterByValue(node->child1(), *node->cellOperand());
    17391744        break;
    17401745    }
    … …  
    18601865    case VariableWatchpoint:
    18611866    case VarInjectionWatchpoint:
    1862         break;
    1863            
    18641867    case PutGlobalVar:
    18651868    case NotifyWrite:
    … …  
    19011904        break;
    19021905
     1906    case ProfiledCall:
     1907    case ProfiledConstruct:
     1908        if (forNode(m_graph.varArgChild(node, 0)).m_value)
     1909            m_state.setFoundConstants(true);
     1910        clobberWorld(node->origin.semantic, clobberLimit);
     1911        forNode(node).makeHeapTop();
     1912        break;
     1913
    19031914    case ForceOSRExit:
     1915    case CheckBadCell:
    19041916        m_state.setIsValid(false);
    19051917        break;
    … …  
    19561968    case ArithIMul:
    19571969    case FiatInt52:
    1958         RELEASE_ASSERT_NOT_REACHED();
     1970    case BottomValue:
     1971        DFG_CRASH(m_graph, node, "Unexpected node type");
    19591972        break;
    19601973    }

  • trunk/Source/JavaScriptCore/dfg/DFGBackwardsPropagationPhase.cpp

    r171613 r172940  
    390390                node->child1()->mergeFlags(NodeBytecodeUsesAsNumber | NodeBytecodeUsesAsOther);
    391391                break;
     392            case SwitchCell:
     393                // There is currently no point to being clever here since this is used for switching
     394                // on objects.
     395                mergeDefaultFlags(node);
     396                break;
    392397            }
    393398            break;

  • trunk/Source/JavaScriptCore/dfg/DFGBasicBlock.cpp

    r171613 r172940  
    5959}
    6060
    61 BasicBlock::~BasicBlock() { }
     61BasicBlock::~BasicBlock()
     62{
     63}
    6264
    6365void BasicBlock::ensureLocals(unsigned newNumLocals)

  • trunk/Source/JavaScriptCore/dfg/DFGBasicBlock.h

    r172129 r172940  
    6363    Node* operator[](size_t i) const { return at(i); }
    6464    Node* last() const { return at(size() - 1); }
     65    Node* takeLast() { return m_nodes.takeLast(); }
    6566    void resize(size_t size) { m_nodes.resize(size); }
    6667    void grow(size_t size) { m_nodes.grow(size); }
    … …  
    106107   
    107108    void dump(PrintStream& out) const;
     109   
     110    void didLink()
     111    {
     112#if !ASSERT_DISABLED
     113        isLinked = true;
     114#endif
     115    }
    108116   
    109117    // This value is used internally for block linking and OSR entry. It is mostly meaningless

  • trunk/Source/JavaScriptCore/dfg/DFGByteCodeParser.cpp

    r172853 r172940  
    5151namespace JSC { namespace DFG {
    5252
     53static const bool verbose = false;
     54
    5355class ConstantBufferKey {
    5456public:
    … …  
    179181    void handleCall(int result, NodeType op, CodeSpecializationKind, unsigned instructionSize, int callee, int argCount, int registerOffset);
    180182    void handleCall(Instruction* pc, NodeType op, CodeSpecializationKind);
    181     void emitFunctionChecks(const CallLinkStatus&, Node* callTarget, int registerOffset, CodeSpecializationKind);
     183    void emitFunctionChecks(CallVariant, Node* callTarget, int registerOffset, CodeSpecializationKind);
     184    void undoFunctionChecks(CallVariant);
    182185    void emitArgumentPhantoms(int registerOffset, int argumentCountIncludingThis, CodeSpecializationKind);
     186    unsigned inliningCost(CallVariant, int argumentCountIncludingThis, CodeSpecializationKind); // Return UINT_MAX if it's not an inlining candidate. By convention, intrinsics have a cost of 1.
    183187    // Handle inlining. Return true if it succeeded, false if we need to plant a call.
    184     bool handleInlining(Node* callTargetNode, int resultOperand, const CallLinkStatus&, int registerOffset, int argumentCountIncludingThis, unsigned nextOffset, InlineCallFrame::Kind);
     188    bool handleInlining(Node* callTargetNode, int resultOperand, const CallLinkStatus&, int registerOffset, int argumentCountIncludingThis, unsigned nextOffset, NodeType callOp, InlineCallFrame::Kind, SpeculatedType prediction);
     189    enum CallerLinkability { CallerDoesNormalLinking, CallerLinksManually };
     190    bool attemptToInlineCall(Node* callTargetNode, int resultOperand, CallVariant, int registerOffset, int argumentCountIncludingThis, unsigned nextOffset, InlineCallFrame::Kind, CallerLinkability, SpeculatedType prediction, unsigned& inliningBalance);
     191    void inlineCall(Node* callTargetNode, int resultOperand, CallVariant, int registerOffset, int argumentCountIncludingThis, unsigned nextOffset, InlineCallFrame::Kind, CallerLinkability);
     192    void cancelLinkingForBlock(InlineStackEntry*, BasicBlock*); // Only works when the given block is the last one to have been added for that inline stack entry.
    185193    // Handle intrinsic functions. Return true if it succeeded, false if we need to plant a call.
    186194    bool handleIntrinsic(int resultOperand, Intrinsic, int registerOffset, int argumentCountIncludingThis, SpeculatedType prediction);
    187195    bool handleTypedArrayConstructor(int resultOperand, InternalFunction*, int registerOffset, int argumentCountIncludingThis, TypedArrayType);
    188     bool handleConstantInternalFunction(int resultOperand, InternalFunction*, int registerOffset, int argumentCountIncludingThis, SpeculatedType prediction, CodeSpecializationKind);
     196    bool handleConstantInternalFunction(int resultOperand, InternalFunction*, int registerOffset, int argumentCountIncludingThis, CodeSpecializationKind);
    189197    Node* handlePutByOffset(Node* base, unsigned identifier, PropertyOffset, Node* value);
    190198    Node* handleGetByOffset(SpeculatedType, Node* base, const StructureSet&, unsigned identifierNumber, PropertyOffset, NodeType op = GetByOffset);
    … …  
    201209    Node* getScope(unsigned skipCount);
    202210   
    203     // Prepare to parse a block.
    204211    void prepareToParseBlock();
     212    void clearCaches();
     213
    205214    // Parse a single basic block of bytecode instructions.
    206215    bool parseBlock(unsigned limit);
    … …  
    297306        return delayed.execute(this, setMode);
    298307    }
     308   
     309    void processSetLocalQueue()
     310    {
     311        for (unsigned i = 0; i < m_setLocalQueue.size(); ++i)
     312            m_setLocalQueue[i].execute(this);
     313        m_setLocalQueue.resize(0);
     314    }
    299315
    300316    Node* set(VirtualRegister operand, Node* value, SetMode setMode = NormalSet)
    … …  
    638654        return result;
    639655    }
     656   
     657    void removeLastNodeFromGraph(NodeType expectedNodeType)
     658    {
     659        Node* node = m_currentBlock->takeLast();
     660        RELEASE_ASSERT(node->op() == expectedNodeType);
     661        m_graph.m_allocator.free(node);
     662    }
    640663
    641664    void addVarArgChild(Node* child)
    … …  
    646669   
    647670    Node* addCallWithoutSettingResult(
    648         NodeType op, Node* callee, int argCount, int registerOffset,
     671        NodeType op, OpInfo opInfo, Node* callee, int argCount, int registerOffset,
    649672        SpeculatedType prediction)
    650673    {
    … …  
    654677            m_parameterSlots = parameterSlots;
    655678
    656         int dummyThisArgument = op == Call || op == NativeCall ? 0 : 1;
     679        int dummyThisArgument = op == Call || op == NativeCall || op == ProfiledCall ? 0 : 1;
    657680        for (int i = 0 + dummyThisArgument; i < argCount; ++i)
    658681            addVarArgChild(get(virtualRegisterForArgument(i, registerOffset)));
    659682
    660         return addToGraph(Node::VarArg, op, OpInfo(0), OpInfo(prediction));
     683        return addToGraph(Node::VarArg, op, opInfo, OpInfo(prediction));
    661684    }
    662685   
    663686    Node* addCall(
    664         int result, NodeType op, Node* callee, int argCount, int registerOffset,
     687        int result, NodeType op, OpInfo opInfo, Node* callee, int argCount, int registerOffset,
    665688        SpeculatedType prediction)
    666689    {
    667690        Node* call = addCallWithoutSettingResult(
    668             op, callee, argCount, registerOffset, prediction);
     691            op, opInfo, callee, argCount, registerOffset, prediction);
    669692        VirtualRegister resultReg(result);
    670693        if (resultReg.isValid())
    … …  
    872895       
    873896        // Potential block linking targets. Must be sorted by bytecodeBegin, and
    874         // cannot have two blocks that have the same bytecodeBegin. For this very
    875         // reason, this is not equivalent to
     897        // cannot have two blocks that have the same bytecodeBegin.
    876898        Vector<BasicBlock*> m_blockLinkingTargets;
    877899       
    … …  
    10201042{
    10211043    ASSERT(registerOffset <= 0);
    1022     CodeSpecializationKind specializationKind = InlineCallFrame::specializationKindFor(kind);
    10231044   
    10241045    if (callTarget->hasConstant())
    10251046        callLinkStatus = CallLinkStatus(callTarget->asJSValue()).setIsProved(true);
     1047   
     1048    if ((!callLinkStatus.canOptimize() || callLinkStatus.size() != 1)
     1049        && !isFTL(m_graph.m_plan.mode) && Options::useFTLJIT()
     1050        && InlineCallFrame::isNormalCall(kind)
     1051        && CallEdgeLog::isEnabled()
     1052        && Options::dfgDoesCallEdgeProfiling()) {
     1053        ASSERT(op == Call || op == Construct);
     1054        if (op == Call)
     1055            op = ProfiledCall;
     1056        else
     1057            op = ProfiledConstruct;
     1058    }
    10261059   
    10271060    if (!callLinkStatus.canOptimize()) {
    … …  
    10291062        // that we cannot optimize them.
    10301063       
    1031         addCall(result, op, callTarget, argumentCountIncludingThis, registerOffset, prediction);
     1064        addCall(result, op, OpInfo(), callTarget, argumentCountIncludingThis, registerOffset, prediction);
    10321065        return;
    10331066    }
    10341067   
    10351068    unsigned nextOffset = m_currentIndex + instructionSize;
    1036 
    1037     if (InternalFunction* function = callLinkStatus.internalFunction()) {
    1038         if (handleConstantInternalFunction(result, function, registerOffset, argumentCountIncludingThis, prediction, specializationKind)) {
    1039             // This phantoming has to be *after* the code for the intrinsic, to signify that
    1040             // the inputs must be kept alive whatever exits the intrinsic may do.
    1041             addToGraph(Phantom, callTarget);
    1042             emitArgumentPhantoms(registerOffset, argumentCountIncludingThis, specializationKind);
    1043             return;
    1044         }
    1045        
    1046         // Can only handle this using the generic call handler.
    1047         addCall(result, op, callTarget, argumentCountIncludingThis, registerOffset, prediction);
    1048         return;
    1049     }
    1050        
    1051     Intrinsic intrinsic = callLinkStatus.intrinsicFor(specializationKind);
    1052 
    1053     JSFunction* knownFunction = nullptr;
    1054     if (intrinsic != NoIntrinsic) {
    1055         emitFunctionChecks(callLinkStatus, callTarget, registerOffset, specializationKind);
    1056            
    1057         if (handleIntrinsic(result, intrinsic, registerOffset, argumentCountIncludingThis, prediction)) {
    1058             // This phantoming has to be *after* the code for the intrinsic, to signify that
    1059             // the inputs must be kept alive whatever exits the intrinsic may do.
    1060             addToGraph(Phantom, callTarget);
    1061             emitArgumentPhantoms(registerOffset, argumentCountIncludingThis, specializationKind);
    1062             if (m_graph.compilation())
    1063                 m_graph.compilation()->noticeInlinedCall();
    1064             return;
    1065         }
    1066     } else if (handleInlining(callTarget, result, callLinkStatus, registerOffset, argumentCountIncludingThis, nextOffset, kind)) {
     1069   
     1070    OpInfo callOpInfo;
     1071   
     1072    if (handleInlining(callTarget, result, callLinkStatus, registerOffset, argumentCountIncludingThis, nextOffset, op, kind, prediction)) {
    10671073        if (m_graph.compilation())
    10681074            m_graph.compilation()->noticeInlinedCall();
    10691075        return;
     1076    }
     1077   
    10701078#if ENABLE(FTL_NATIVE_CALL_INLINING)
    1071     } else if (isFTL(m_graph.m_plan.mode) && Options::optimizeNativeCalls()) {
    1072         JSFunction* function = callLinkStatus.function();
     1079    if (isFTL(m_graph.m_plan.mode) && Options::optimizeNativeCalls() && callLinkStatus.size() == 1 && !callLinkStatus.couldTakeSlowPath()) {
     1080        CallVariant callee = callLinkStatus[0].callee();
     1081        JSFunction* function = callee.function();
     1082        CodeSpecializationKind specializationKind = InlineCallFrame::specializationKindFor(kind);
    10731083        if (function && function->isHostFunction()) {
    1074             emitFunctionChecks(callLinkStatus, callTarget, registerOffset, specializationKind);
    1075             knownFunction = function;
    1076 
    1077             if (op == Call)
     1084            emitFunctionChecks(callee, callTarget, registerOffset, specializationKind);
     1085            callOpInfo = OpInfo(m_graph.freeze(function));
     1086
     1087            if (op == Call || op == ProfiledCall)
    10781088                op = NativeCall;
    10791089            else {
    1080                 ASSERT(op == Construct);
     1090                ASSERT(op == Construct || op == ProfiledConstruct);
    10811091                op = NativeConstruct;
    10821092            }
    10831093        }
     1094    }
    10841095#endif
    1085     }
    1086     Node* call = addCall(result, op, callTarget, argumentCountIncludingThis, registerOffset, prediction);
    1087 
    1088     if (knownFunction)
    1089         call->giveKnownFunction(knownFunction);
     1096   
     1097    addCall(result, op, callOpInfo, callTarget, argumentCountIncludingThis, registerOffset, prediction);
    10901098}
    10911099
    1092 void ByteCodeParser::emitFunctionChecks(const CallLinkStatus& callLinkStatus, Node* callTarget, int registerOffset, CodeSpecializationKind kind)
     1100void ByteCodeParser::emitFunctionChecks(CallVariant callee, Node* callTarget, int registerOffset, CodeSpecializationKind kind)
    10931101{
    10941102    Node* thisArgument;
    … …  
    10981106        thisArgument = 0;
    10991107
    1100     if (callLinkStatus.isProved()) {
    1101         addToGraph(Phantom, callTarget, thisArgument);
    1102         return;
    1103     }
    1104    
    1105     ASSERT(callLinkStatus.canOptimize());
    1106    
    1107     if (JSFunction* function = callLinkStatus.function())
    1108         addToGraph(CheckFunction, OpInfo(m_graph.freeze(function)), callTarget, thisArgument);
    1109     else {
    1110         ASSERT(callLinkStatus.executable());
    1111        
    1112         addToGraph(CheckExecutable, OpInfo(callLinkStatus.executable()), callTarget, thisArgument);
    1113     }
     1108    JSCell* calleeCell;
     1109    Node* callTargetForCheck;
     1110    if (callee.isClosureCall()) {
     1111        calleeCell = callee.executable();
     1112        callTargetForCheck = addToGraph(GetExecutable, callTarget);
     1113    } else {
     1114        calleeCell = callee.nonExecutableCallee();
     1115        callTargetForCheck = callTarget;
     1116    }
     1117   
     1118    ASSERT(calleeCell);
     1119    addToGraph(CheckCell, OpInfo(m_graph.freeze(calleeCell)), callTargetForCheck, thisArgument);
     1120}
     1121
     1122void ByteCodeParser::undoFunctionChecks(CallVariant callee)
     1123{
     1124    removeLastNodeFromGraph(CheckCell);
     1125    if (callee.isClosureCall())
     1126        removeLastNodeFromGraph(GetExecutable);
    11141127}
    11151128
    … …  
    11201133}
    11211134
    1122 bool ByteCodeParser::handleInlining(Node* callTargetNode, int resultOperand, const CallLinkStatus& callLinkStatus, int registerOffset, int argumentCountIncludingThis, unsigned nextOffset, InlineCallFrame::Kind kind)
     1135unsigned ByteCodeParser::inliningCost(CallVariant callee, int argumentCountIncludingThis, CodeSpecializationKind kind)
    11231136{
    1124     static const bool verbose = false;
    1125    
    1126     CodeSpecializationKind specializationKind = InlineCallFrame::specializationKindFor(kind);
    1127    
    11281137    if (verbose)
    1129         dataLog("Considering inlining ", callLinkStatus, " into ", currentCodeOrigin(), "\n");
    1130    
    1131     // First, the really simple checks: do we have an actual JS function?
    1132     if (!callLinkStatus.executable()) {
     1138        dataLog("Considering inlining ", callee, " into ", currentCodeOrigin(), "\n");
     1139   
     1140    FunctionExecutable* executable = callee.functionExecutable();
     1141    if (!executable) {
    11331142        if (verbose)
    1134             dataLog("    Failing because there is no executable.\n");
    1135         return false;
    1136     }
    1137     if (callLinkStatus.executable()->isHostFunction()) {
    1138         if (verbose)
    1139             dataLog("    Failing because it's a host function.\n");
    1140         return false;
    1141     }
    1142    
    1143     FunctionExecutable* executable = jsCast<FunctionExecutable*>(callLinkStatus.executable());
     1143            dataLog("    Failing because there is no function executable.");
     1144        return UINT_MAX;
     1145    }
    11441146   
    11451147    // Does the number of arguments we're passing match the arity of the target? We currently
    … …  
    11491151        if (verbose)
    11501152            dataLog("    Failing because of arity mismatch.\n");
    1151         return false;
     1153        return UINT_MAX;
    11521154    }
    11531155   
    … …  
    11581160    // global function, where watchpointing gives us static information. Overall, it's a rare case
    11591161    // because we expect that any hot callees would have already been compiled.
    1160     CodeBlock* codeBlock = executable->baselineCodeBlockFor(specializationKind);
     1162    CodeBlock* codeBlock = executable->baselineCodeBlockFor(kind);
    11611163    if (!codeBlock) {
    11621164        if (verbose)
    11631165            dataLog("    Failing because no code block available.\n");
    1164         return false;
     1166        return UINT_MAX;
    11651167    }
    11661168    CapabilityLevel capabilityLevel = inlineFunctionForCapabilityLevel(
    1167         codeBlock, specializationKind, callLinkStatus.isClosureCall());
     1169        codeBlock, kind, callee.isClosureCall());
    11681170    if (!canInline(capabilityLevel)) {
    11691171        if (verbose)
    11701172            dataLog("    Failing because the function is not inlineable.\n");
    1171         return false;
     1173        return UINT_MAX;
    11721174    }
    11731175   
    … …  
    11791181        if (verbose)
    11801182            dataLog("    Failing because the caller is too large.\n");
    1181         return false;
     1183        return UINT_MAX;
    11821184    }
    11831185   
    … …  
    11981200            if (verbose)
    11991201                dataLog("    Failing because depth exceeded.\n");
    1200             return false;
     1202            return UINT_MAX;
    12011203        }
    12021204       
    … …  
    12061208                if (verbose)
    12071209                    dataLog("    Failing because recursion detected.\n");
    1208                 return false;
     1210                return UINT_MAX;
    12091211            }
    12101212        }
    … …  
    12121214   
    12131215    if (verbose)
    1214         dataLog("    Committing to inlining.\n");
    1215    
    1216     // Now we know without a doubt that we are committed to inlining. So begin the process
    1217     // by checking the callee (if necessary) and making sure that arguments and the callee
    1218     // are flushed.
    1219     emitFunctionChecks(callLinkStatus, callTargetNode, registerOffset, specializationKind);
    1220    
     1216        dataLog("    Inlining should be possible.\n");
     1217   
     1218    // It might be possible to inline.
     1219    return codeBlock->instructionCount();
     1220}
     1221
     1222void ByteCodeParser::inlineCall(Node* callTargetNode, int resultOperand, CallVariant callee, int registerOffset, int argumentCountIncludingThis, unsigned nextOffset, InlineCallFrame::Kind kind, CallerLinkability callerLinkability)
     1223{
     1224    CodeSpecializationKind specializationKind = InlineCallFrame::specializationKindFor(kind);
     1225   
     1226    ASSERT(inliningCost(callee, argumentCountIncludingThis, specializationKind) != UINT_MAX);
     1227   
     1228    CodeBlock* codeBlock = callee.functionExecutable()->baselineCodeBlockFor(specializationKind);
     1229
    12211230    // FIXME: Don't flush constants!
    12221231   
    … …  
    12341243   
    12351244    InlineStackEntry inlineStackEntry(
    1236         this, codeBlock, codeBlock, m_graph.lastBlock(), callLinkStatus.function(), resultReg,
     1245        this, codeBlock, codeBlock, m_graph.lastBlock(), callee.function(), resultReg,
    12371246        (VirtualRegister)inlineCallFrameStart, argumentCountIncludingThis, kind);
    12381247   
    … …  
    12481257    RELEASE_ASSERT(
    12491258        m_inlineStackTop->m_inlineCallFrame->isClosureCall
    1250         == callLinkStatus.isClosureCall());
    1251     if (callLinkStatus.isClosureCall()) {
     1259        == callee.isClosureCall());
     1260    if (callee.isClosureCall()) {
    12521261        VariableAccessData* calleeVariable =
    12531262            set(VirtualRegister(JSStack::Callee), callTargetNode, ImmediateNakedSet)->variableAccessData();
    … …  
    12641273   
    12651274    parseCodeBlock();
    1266     prepareToParseBlock(); // Reset our state now that we're back to the outer code.
     1275    clearCaches(); // Reset our state now that we're back to the outer code.
    12671276   
    12681277    m_currentIndex = oldIndex;
    … …  
    12771286            ASSERT(inlineStackEntry.m_callsiteBlockHead->isLinked);
    12781287       
    1279         // It's possible that the callsite block head is not owned by the caller.
    1280         if (!inlineStackEntry.m_caller->m_unlinkedBlocks.isEmpty()) {
    1281             // It's definitely owned by the caller, because the caller created new blocks.
    1282             // Assert that this all adds up.
    1283             ASSERT(inlineStackEntry.m_caller->m_unlinkedBlocks.last().m_block == inlineStackEntry.m_callsiteBlockHead);
    1284             ASSERT(inlineStackEntry.m_caller->m_unlinkedBlocks.last().m_needsNormalLinking);
    1285             inlineStackEntry.m_caller->m_unlinkedBlocks.last().m_needsNormalLinking = false;
    1286         } else {
    1287             // It's definitely not owned by the caller. Tell the caller that he does not
    1288             // need to link his callsite block head, because we did it for him.
    1289             ASSERT(inlineStackEntry.m_caller->m_callsiteBlockHeadNeedsLinking);
    1290             ASSERT(inlineStackEntry.m_caller->m_callsiteBlockHead == inlineStackEntry.m_callsiteBlockHead);
    1291             inlineStackEntry.m_caller->m_callsiteBlockHeadNeedsLinking = false;
    1292         }
     1288        if (callerLinkability == CallerDoesNormalLinking)
     1289            cancelLinkingForBlock(inlineStackEntry.m_caller, inlineStackEntry.m_callsiteBlockHead);
    12931290       
    12941291        linkBlocks(inlineStackEntry.m_unlinkedBlocks, inlineStackEntry.m_blockLinkingTargets);
    … …  
    13091306            // in the linker's binary search.
    13101307            lastBlock->bytecodeBegin = m_currentIndex;
    1311             m_inlineStackTop->m_caller->m_unlinkedBlocks.append(UnlinkedBlock(m_graph.lastBlock()));
     1308            if (callerLinkability == CallerDoesNormalLinking) {
     1309                if (verbose)
     1310                    dataLog("Adding unlinked block ", RawPointer(m_graph.lastBlock()), " (one return)\n");
     1311                m_inlineStackTop->m_caller->m_unlinkedBlocks.append(UnlinkedBlock(m_graph.lastBlock()));
     1312            }
    13121313        }
    13131314       
    13141315        m_currentBlock = m_graph.lastBlock();
    1315         return true;
     1316        return;
    13161317    }
    13171318   
    13181319    // If we get to this point then all blocks must end in some sort of terminals.
    13191320    ASSERT(lastBlock->last()->isTerminal());
    1320    
    13211321
    13221322    // Need to create a new basic block for the continuation at the caller.
    … …  
    13341334        node->targetBlock() = block.get();
    13351335        inlineStackEntry.m_unlinkedBlocks[i].m_needsEarlyReturnLinking = false;
    1336 #if !ASSERT_DISABLED
    1337         blockToLink->isLinked = true;
    1338 #endif
     1336        if (verbose)
     1337            dataLog("Marking ", RawPointer(blockToLink), " as linked (jumps to return)\n");
     1338        blockToLink->didLink();
    13391339    }
    13401340   
    13411341    m_currentBlock = block.get();
    13421342    ASSERT(m_inlineStackTop->m_caller->m_blockLinkingTargets.isEmpty() || m_inlineStackTop->m_caller->m_blockLinkingTargets.last()->bytecodeBegin < nextOffset);
    1343     m_inlineStackTop->m_caller->m_unlinkedBlocks.append(UnlinkedBlock(block.get()));
    1344     m_inlineStackTop->m_caller->m_blockLinkingTargets.append(block.get());
     1343    if (verbose)
     1344        dataLog("Adding unlinked block ", RawPointer(block.get()), " (many returns)\n");
     1345    if (callerLinkability == CallerDoesNormalLinking) {
     1346        m_inlineStackTop->m_caller->m_unlinkedBlocks.append(UnlinkedBlock(block.get()));
     1347        m_inlineStackTop->m_caller->m_blockLinkingTargets.append(block.get());
     1348    }
    13451349    m_graph.appendBlock(block);
    13461350    prepareToParseBlock();
    1347    
    1348     // At this point we return and continue to generate code for the caller, but
    1349     // in the new basic block.
     1351}
     1352
     1353void ByteCodeParser::cancelLinkingForBlock(InlineStackEntry* inlineStackEntry, BasicBlock* block)
     1354{
     1355    // It's possible that the callsite block head is not owned by the caller.
     1356    if (!inlineStackEntry->m_unlinkedBlocks.isEmpty()) {
     1357        // It's definitely owned by the caller, because the caller created new blocks.
     1358        // Assert that this all adds up.
     1359        ASSERT_UNUSED(block, inlineStackEntry->m_unlinkedBlocks.last().m_block == block);
     1360        ASSERT(inlineStackEntry->m_unlinkedBlocks.last().m_needsNormalLinking);
     1361        inlineStackEntry->m_unlinkedBlocks.last().m_needsNormalLinking = false;
     1362    } else {
     1363        // It's definitely not owned by the caller. Tell the caller that he does not
     1364        // need to link his callsite block head, because we did it for him.
     1365        ASSERT(inlineStackEntry->m_callsiteBlockHeadNeedsLinking);
     1366        ASSERT_UNUSED(block, inlineStackEntry->m_callsiteBlockHead == block);
     1367        inlineStackEntry->m_callsiteBlockHeadNeedsLinking = false;
     1368    }
     1369}
     1370
     1371bool ByteCodeParser::attemptToInlineCall(Node* callTargetNode, int resultOperand, CallVariant callee, int registerOffset, int argumentCountIncludingThis, unsigned nextOffset, InlineCallFrame::Kind kind, CallerLinkability callerLinkability, SpeculatedType prediction, unsigned& inliningBalance)
     1372{
     1373    CodeSpecializationKind specializationKind = InlineCallFrame::specializationKindFor(kind);
     1374   
     1375    if (!inliningBalance)
     1376        return false;
     1377   
     1378    if (InternalFunction* function = callee.internalFunction()) {
     1379        if (handleConstantInternalFunction(resultOperand, function, registerOffset, argumentCountIncludingThis, specializationKind)) {
     1380            addToGraph(Phantom, callTargetNode);
     1381            emitArgumentPhantoms(registerOffset, argumentCountIncludingThis, specializationKind);
     1382            inliningBalance--;
     1383            return true;
     1384        }
     1385        return false;
     1386    }
     1387   
     1388    Intrinsic intrinsic = callee.intrinsicFor(specializationKind);
     1389    if (intrinsic != NoIntrinsic) {
     1390        if (handleIntrinsic(resultOperand, intrinsic, registerOffset, argumentCountIncludingThis, prediction)) {
     1391            addToGraph(Phantom, callTargetNode);
     1392            emitArgumentPhantoms(registerOffset, argumentCountIncludingThis, specializationKind);
     1393            inliningBalance--;
     1394            return true;
     1395        }
     1396        return false;
     1397    }
     1398   
     1399    unsigned myInliningCost = inliningCost(callee, argumentCountIncludingThis, specializationKind);
     1400    if (myInliningCost > inliningBalance)
     1401        return false;
     1402   
     1403    inlineCall(callTargetNode, resultOperand, callee, registerOffset, argumentCountIncludingThis, nextOffset, kind, callerLinkability);
     1404    inliningBalance -= myInliningCost;
     1405    return true;
     1406}
     1407
     1408bool ByteCodeParser::handleInlining(Node* callTargetNode, int resultOperand, const CallLinkStatus& callLinkStatus, int registerOffset, int argumentCountIncludingThis, unsigned nextOffset, NodeType callOp, InlineCallFrame::Kind kind, SpeculatedType prediction)
     1409{
     1410    if (verbose) {
     1411        dataLog("Handling inlining...\n");
     1412        dataLog("Stack: ", currentCodeOrigin(), "\n");
     1413    }
     1414    CodeSpecializationKind specializationKind = InlineCallFrame::specializationKindFor(kind);
     1415   
     1416    if (!callLinkStatus.size()) {
     1417        if (verbose)
     1418            dataLog("Bailing inlining.\n");
     1419        return false;
     1420    }
     1421   
     1422    unsigned inliningBalance = Options::maximumFunctionForCallInlineCandidateInstructionCount();
     1423    if (specializationKind == CodeForConstruct)
     1424        inliningBalance = std::min(inliningBalance, Options::maximumFunctionForConstructInlineCandidateInstructionCount());
     1425    if (callLinkStatus.isClosureCall())
     1426        inliningBalance = std::min(inliningBalance, Options::maximumFunctionForClosureCallInlineCandidateInstructionCount());
     1427   
     1428    // First check if we can avoid creating control flow. Our inliner does some CFG
     1429    // simplification on the fly and this helps reduce compile times, but we can only leverage
     1430    // this in cases where we don't need control flow diamonds to check the callee.
     1431    if (!callLinkStatus.couldTakeSlowPath() && callLinkStatus.size() == 1) {
     1432        emitFunctionChecks(
     1433            callLinkStatus[0].callee(), callTargetNode, registerOffset, specializationKind);
     1434        bool result = attemptToInlineCall(
     1435            callTargetNode, resultOperand, callLinkStatus[0].callee(), registerOffset,
     1436            argumentCountIncludingThis, nextOffset, kind, CallerDoesNormalLinking, prediction,
     1437            inliningBalance);
     1438        if (!result && !callLinkStatus.isProved())
     1439            undoFunctionChecks(callLinkStatus[0].callee());
     1440        if (verbose) {
     1441            dataLog("Done inlining (simple).\n");
     1442            dataLog("Stack: ", currentCodeOrigin(), "\n");
     1443        }
     1444        return result;
     1445    }
     1446   
     1447    // We need to create some kind of switch over callee. For now we only do this if we believe that
     1448    // we're in the top tier. We have two reasons for this: first, it provides us an opportunity to
     1449    // do more detailed polyvariant/polymorphic profiling; and second, it reduces compile times in
     1450    // the DFG. And by polyvariant profiling we mean polyvariant profiling of *this* call. Note that
     1451    // we could improve that aspect of this by doing polymorphic inlining but having the profiling
     1452    // also. Currently we opt against this, but it could be interesting. That would require having a
     1453    // separate node for call edge profiling.
     1454    // FIXME: Introduce the notion of a separate call edge profiling node.
     1455    // https://bugs.webkit.org/show_bug.cgi?id=136033
     1456    if (!isFTL(m_graph.m_plan.mode) || !Options::enablePolymorphicCallInlining()) {
     1457        if (verbose) {
     1458            dataLog("Bailing inlining (hard).\n");
     1459            dataLog("Stack: ", currentCodeOrigin(), "\n");
     1460        }
     1461        return false;
     1462    }
     1463   
     1464    unsigned oldOffset = m_currentIndex;
     1465   
     1466    bool allAreClosureCalls = true;
     1467    bool allAreDirectCalls = true;
     1468    for (unsigned i = callLinkStatus.size(); i--;) {
     1469        if (callLinkStatus[i].callee().isClosureCall())
     1470            allAreDirectCalls = false;
     1471        else
     1472            allAreClosureCalls = false;
     1473    }
     1474   
     1475    Node* thingToSwitchOn;
     1476    if (allAreDirectCalls)
     1477        thingToSwitchOn = callTargetNode;
     1478    else if (allAreClosureCalls)
     1479        thingToSwitchOn = addToGraph(GetExecutable, callTargetNode);
     1480    else {
     1481        // FIXME: We should be able to handle this case, but it's tricky and we don't know of cases
     1482        // where it would be beneficial. Also, CallLinkStatus would make all callees appear like
     1483        // closure calls if any calls were closure calls - except for calls to internal functions.
     1484        // So this will only arise if some callees are internal functions and others are closures.
     1485        // https://bugs.webkit.org/show_bug.cgi?id=136020
     1486        if (verbose) {
     1487            dataLog("Bailing inlining (mix).\n");
     1488            dataLog("Stack: ", currentCodeOrigin(), "\n");
     1489        }
     1490        return false;
     1491    }
     1492   
     1493    if (verbose) {
     1494        dataLog("Doing hard inlining...\n");
     1495        dataLog("Stack: ", currentCodeOrigin(), "\n");
     1496    }
     1497   
     1498    // This makes me wish that we were in SSA all the time. We need to pick a variable into which to
     1499    // store the callee so that it will be accessible to all of the blocks we're about to create. We
     1500    // get away with doing an immediate-set here because we wouldn't have performed any side effects
     1501    // yet.
     1502    if (verbose)
     1503        dataLog("Register offset: ", registerOffset);
     1504    VirtualRegister calleeReg(registerOffset + JSStack::Callee);
     1505    calleeReg = m_inlineStackTop->remapOperand(calleeReg);
     1506    if (verbose)
     1507        dataLog("Callee is going to be ", calleeReg, "\n");
     1508    setDirect(calleeReg, callTargetNode, ImmediateSetWithFlush);
     1509   
     1510    SwitchData& data = *m_graph.m_switchData.add();
     1511    data.kind = SwitchCell;
     1512    addToGraph(Switch, OpInfo(&data), thingToSwitchOn);
     1513   
     1514    BasicBlock* originBlock = m_currentBlock;
     1515    if (verbose)
     1516        dataLog("Marking ", RawPointer(originBlock), " as linked (origin of poly inline)\n");
     1517    originBlock->didLink();
     1518    cancelLinkingForBlock(m_inlineStackTop, originBlock);
     1519   
     1520    // Each inlined callee will have a landing block that it returns at. They should all have jumps
     1521    // to the continuation block, which we create last.
     1522    Vector<BasicBlock*> landingBlocks;
     1523   
     1524    // We make force this true if we give up on inlining any of the edges.
     1525    bool couldTakeSlowPath = callLinkStatus.couldTakeSlowPath();
     1526   
     1527    if (verbose)
     1528        dataLog("About to loop over functions at ", currentCodeOrigin(), ".\n");
     1529   
     1530    for (unsigned i = 0; i < callLinkStatus.size(); ++i) {
     1531        m_currentIndex = oldOffset;
     1532        RefPtr<BasicBlock> block = adoptRef(new BasicBlock(UINT_MAX, m_numArguments, m_numLocals, PNaN));
     1533        m_currentBlock = block.get();
     1534        m_graph.appendBlock(block);
     1535        prepareToParseBlock();
     1536       
     1537        Node* myCallTargetNode = getDirect(calleeReg);
     1538       
     1539        bool inliningResult = attemptToInlineCall(
     1540            myCallTargetNode, resultOperand, callLinkStatus[i].callee(), registerOffset,
     1541            argumentCountIncludingThis, nextOffset, kind, CallerLinksManually, prediction,
     1542            inliningBalance);
     1543       
     1544        if (!inliningResult) {
     1545            // That failed so we let the block die. Nothing interesting should have been added to
     1546            // the block. We also give up on inlining any of the (less frequent) callees.
     1547            ASSERT(m_currentBlock == block.get());
     1548            ASSERT(m_graph.m_blocks.last() == block);
     1549            m_graph.killBlockAndItsContents(block.get());
     1550            m_graph.m_blocks.removeLast();
     1551           
     1552            // The fact that inlining failed means we need a slow path.
     1553            couldTakeSlowPath = true;
     1554            break;
     1555        }
     1556       
     1557        JSCell* thingToCaseOn;
     1558        if (allAreDirectCalls)
     1559            thingToCaseOn = callLinkStatus[i].callee().nonExecutableCallee();
     1560        else {
     1561            ASSERT(allAreClosureCalls);
     1562            thingToCaseOn = callLinkStatus[i].callee().executable();
     1563        }
     1564        data.cases.append(SwitchCase(m_graph.freeze(thingToCaseOn), block.get()));
     1565        m_currentIndex = nextOffset;
     1566        processSetLocalQueue(); // This only comes into play for intrinsics, since normal inlined code will leave an empty queue.
     1567        addToGraph(Jump);
     1568        if (verbose)
     1569            dataLog("Marking ", RawPointer(m_currentBlock), " as linked (tail of poly inlinee)\n");
     1570        m_currentBlock->didLink();
     1571        landingBlocks.append(m_currentBlock);
     1572
     1573        if (verbose)
     1574            dataLog("Finished inlining ", callLinkStatus[i].callee(), " at ", currentCodeOrigin(), ".\n");
     1575    }
     1576   
     1577    RefPtr<BasicBlock> slowPathBlock = adoptRef(
     1578        new BasicBlock(UINT_MAX, m_numArguments, m_numLocals, PNaN));
     1579    m_currentIndex = oldOffset;
     1580    data.fallThrough = BranchTarget(slowPathBlock.get());
     1581    m_graph.appendBlock(slowPathBlock);
     1582    if (verbose)
     1583        dataLog("Marking ", RawPointer(slowPathBlock.get()), " as linked (slow path block)\n");
     1584    slowPathBlock->didLink();
     1585    prepareToParseBlock();
     1586    m_currentBlock = slowPathBlock.get();
     1587    Node* myCallTargetNode = getDirect(calleeReg);
     1588    if (couldTakeSlowPath) {
     1589        addCall(
     1590            resultOperand, callOp, OpInfo(), myCallTargetNode, argumentCountIncludingThis,
     1591            registerOffset, prediction);
     1592    } else {
     1593        addToGraph(CheckBadCell);
     1594        addToGraph(Phantom, myCallTargetNode);
     1595        emitArgumentPhantoms(registerOffset, argumentCountIncludingThis, specializationKind);
     1596       
     1597        set(VirtualRegister(resultOperand), addToGraph(BottomValue));
     1598    }
     1599
     1600    m_currentIndex = nextOffset;
     1601    processSetLocalQueue();
     1602    addToGraph(Jump);
     1603    landingBlocks.append(m_currentBlock);
     1604   
     1605    RefPtr<BasicBlock> continuationBlock = adoptRef(
     1606        new BasicBlock(UINT_MAX, m_numArguments, m_numLocals, PNaN));
     1607    m_graph.appendBlock(continuationBlock);
     1608    if (verbose)
     1609        dataLog("Adding unlinked block ", RawPointer(continuationBlock.get()), " (continuation)\n");
     1610    m_inlineStackTop->m_unlinkedBlocks.append(UnlinkedBlock(continuationBlock.get()));
     1611    prepareToParseBlock();
     1612    m_currentBlock = continuationBlock.get();
     1613   
     1614    for (unsigned i = landingBlocks.size(); i--;)
     1615        landingBlocks[i]->last()->targetBlock() = continuationBlock.get();
     1616   
     1617    m_currentIndex = oldOffset;
     1618   
     1619    if (verbose) {
     1620        dataLog("Done inlining (hard).\n");
     1621        dataLog("Stack: ", currentCodeOrigin(), "\n");
     1622    }
    13501623    return true;
    13511624}
    … …  
    16461919bool ByteCodeParser::handleConstantInternalFunction(
    16471920    int resultOperand, InternalFunction* function, int registerOffset,
    1648     int argumentCountIncludingThis, SpeculatedType prediction, CodeSpecializationKind kind)
     1921    int argumentCountIncludingThis, CodeSpecializationKind kind)
    16491922{
    16501923    // If we ever find that we have a lot of internal functions that we specialize for,
    … …  
    16541927    // we know about is small enough, that having just a linear cascade of if statements
    16551928    // is good enough.
    1656    
    1657     UNUSED_PARAM(prediction); // Remove this once we do more things.
    16581929   
    16591930    if (function->classInfo() == ArrayConstructor::info()) {
    … …  
    20212292void ByteCodeParser::prepareToParseBlock()
    20222293{
     2294    clearCaches();
     2295    ASSERT(m_setLocalQueue.isEmpty());
     2296}
     2297
     2298void ByteCodeParser::clearCaches()
     2299{
    20232300    m_constants.resize(0);
    20242301}
    … …  
    20602337
    20612338    while (true) {
    2062         for (unsigned i = 0; i < m_setLocalQueue.size(); ++i)
    2063             m_setLocalQueue[i].execute(this);
    2064         m_setLocalQueue.resize(0);
     2339        processSetLocalQueue();
    20652340       
    20662341        // Don't extend over jump destinations.
    … …  
    22062481            if (!cachedFunction
    22072482                || m_inlineStackTop->m_profiledBlock->couldTakeSlowCase(m_currentIndex)
    2208                 || m_inlineStackTop->m_exitProfile.hasExitSite(m_currentIndex, BadFunction)) {
     2483                || m_inlineStackTop->m_exitProfile.hasExitSite(m_currentIndex, BadCell)) {
    22092484                set(VirtualRegister(currentInstruction[1].u.operand), get(VirtualRegister(JSStack::Callee)));
    22102485            } else {
    … …  
    22122487                ASSERT(cachedFunction->inherits(JSFunction::info()));
    22132488                Node* actualCallee = get(VirtualRegister(JSStack::Callee));
    2214                 addToGraph(CheckFunction, OpInfo(frozen), actualCallee);
     2489                addToGraph(CheckCell, OpInfo(frozen), actualCallee);
    22152490                set(VirtualRegister(currentInstruction[1].u.operand), addToGraph(JSConstant, OpInfo(frozen)));
    22162491            }
    … …  
    28943169            ASSERT(pointerIsFunction(currentInstruction[2].u.specialPointer));
    28953170            addToGraph(
    2896                 CheckFunction,
     3171                CheckCell,
    28973172                OpInfo(m_graph.freeze(static_cast<JSCell*>(actualPointerFor(
    28983173                    m_inlineStackTop->m_codeBlock, currentInstruction[2].u.specialPointer)))),
    … …  
    33183593    }
    33193594   
    3320 #if !ASSERT_DISABLED
    3321     block->isLinked = true;
    3322 #endif
     3595    if (verbose)
     3596        dataLog("Marking ", RawPointer(block), " as linked (actually did linking)\n");
     3597    block->didLink();
    33233598}
    33243599
    … …  
    33263601{
    33273602    for (size_t i = 0; i < unlinkedBlocks.size(); ++i) {
     3603        if (verbose)
     3604            dataLog("Attempting to link ", RawPointer(unlinkedBlocks[i].m_block), "\n");
    33283605        if (unlinkedBlocks[i].m_needsNormalLinking) {
     3606            if (verbose)
     3607                dataLog("    Does need normal linking.\n");
    33293608            linkBlock(unlinkedBlocks[i].m_block, possibleTargets);
    33303609            unlinkedBlocks[i].m_needsNormalLinking = false;
    … …  
    34933772void ByteCodeParser::parseCodeBlock()
    34943773{
    3495     prepareToParseBlock();
     3774    clearCaches();
    34963775   
    34973776    CodeBlock* codeBlock = m_inlineStackTop->m_codeBlock;
    … …  
    35593838                    //    a peephole coalescing of this block in the if statement above. So, we're
    35603839                    //    generating suboptimal code and leaving more work for the CFG simplifier.
    3561                     ASSERT(m_inlineStackTop->m_unlinkedBlocks.isEmpty() || m_inlineStackTop->m_unlinkedBlocks.last().m_block->bytecodeBegin < m_currentIndex);
     3840                    if (!m_inlineStackTop->m_unlinkedBlocks.isEmpty()) {
     3841                        unsigned lastBegin =
     3842                            m_inlineStackTop->m_unlinkedBlocks.last().m_block->bytecodeBegin;
     3843                        ASSERT_UNUSED(
     3844                            lastBegin, lastBegin == UINT_MAX || lastBegin < m_currentIndex);
     3845                    }
    35623846                    m_inlineStackTop->m_unlinkedBlocks.append(UnlinkedBlock(block.get()));
    35633847                    m_inlineStackTop->m_blockLinkingTargets.append(block.get());

  • trunk/Source/JavaScriptCore/dfg/DFGCPSRethreadingPhase.cpp

    r172129 r172940  
    9191                    break;
    9292                case Phantom:
    93                     if (!node->child1())
     93                    if (!node->child1()) {
     94                        m_graph.m_allocator.free(node);
    9495                        continue;
     96                    }
    9597                    switch (node->child1()->op()) {
    9698                    case Phi:

  • trunk/Source/JavaScriptCore/dfg/DFGClobberize.h

    r172808 r172940  
    145145    case MakeRope:
    146146    case ValueToInt32:
     147    case GetExecutable:
     148    case BottomValue:
    147149        def(PureValue(node));
    148150        return;
    … …  
    240242        return;
    241243       
    242     case CheckFunction:
    243         def(PureValue(CheckFunction, AdjacencyList(AdjacencyList::Fixed, node->child1()), node->function()));
    244         return;
    245        
    246     case CheckExecutable:
    247         def(PureValue(node, node->executable()));
     244    case CheckCell:
     245        def(PureValue(CheckCell, AdjacencyList(AdjacencyList::Fixed, node->child1()), node->cellOperand()));
    248246        return;
    249247       
    … …  
    264262    case Throw:
    265263    case ForceOSRExit:
     264    case CheckBadCell:
    266265    case Return:
    267266    case Unreachable:
    … …  
    359358    case Call:
    360359    case Construct:
     360    case ProfiledCall:
     361    case ProfiledConstruct:
    361362    case NativeCall:
    362363    case NativeConstruct:

  • trunk/Source/JavaScriptCore/dfg/DFGCommon.h

    r172737 r172940  
    6262    RefNode,
    6363    DontRefNode
     64};
     65
     66enum SwitchKind {
     67    SwitchImm,
     68    SwitchChar,
     69    SwitchString,
     70    SwitchCell
    6471};
    6572

  • trunk/Source/JavaScriptCore/dfg/DFGConstantFoldingPhase.cpp

    r172737 r172940  
    143143            }
    144144               
    145             case CheckFunction: {
    146                 if (m_state.forNode(node->child1()).value() != node->function()->value())
     145            case CheckCell: {
     146                if (m_state.forNode(node->child1()).value() != node->cellOperand()->value())
    147147                    break;
    148148                node->convertToPhantom();
    … …  
    385385                break;
    386386            }
     387               
     388            case ProfiledCall:
     389            case ProfiledConstruct: {
     390                if (!m_state.forNode(m_graph.varArgChild(node, 0)).m_value)
     391                    break;
     392               
     393                // If we were able to prove that the callee is a constant then the normal call
     394                // inline cache will record this callee. This means that there is no need to do any
     395                // additional profiling.
     396                node->setOp(node->op() == ProfiledCall ? Call : Construct);
     397                eliminated = true;
     398                break;
     399            }
    387400
    388401            default:

  • trunk/Source/JavaScriptCore/dfg/DFGDoesGC.cpp

    r172808 r172940  
    9292    case PutByIdDirect:
    9393    case CheckStructure:
    94     case CheckExecutable:
     94    case GetExecutable:
    9595    case GetButterfly:
    9696    case CheckArray:
    … …  
    105105    case VariableWatchpoint:
    106106    case VarInjectionWatchpoint:
    107     case CheckFunction:
     107    case CheckCell:
    108108    case AllocationProfileWatchpoint:
    109109    case RegExpExec:
    … …  
    120120    case NativeCall:
    121121    case NativeConstruct:
     122    case ProfiledCall:
     123    case ProfiledConstruct:
    122124    case Breakpoint:
    123125    case ProfileWillCall:
    … …  
    196198    case FiatInt52:
    197199    case BooleanToNumber:
     200    case CheckBadCell:
     201    case BottomValue:
    198202        return false;
    199203

  • trunk/Source/JavaScriptCore/dfg/DFGDriver.cpp

    r165405 r172940  
    9090    }
    9191   
     92    if (CallEdgeLog::isEnabled())
     93        vm.ensureCallEdgeLog().processLog();
     94   
    9295    RefPtr<Plan> plan = adoptRef(
    9396        new Plan(codeBlock, profiledDFGCodeBlock, mode, osrEntryBytecodeIndex, mustHandleValues));

  • trunk/Source/JavaScriptCore/dfg/DFGFixupPhase.cpp

    r172808 r172940  
    737737                    fixEdge<StringUse>(node->child1());
    738738                break;
     739            case SwitchCell:
     740                if (node->child1()->shouldSpeculateCell())
     741                    fixEdge<CellUse>(node->child1());
     742                // else it's fine for this to have UntypedUse; we will handle this by just making
     743                // non-cells take the default case.
     744                break;
    739745            }
    740746            break;
    … …  
    898904        }
    899905
    900         case CheckExecutable: {
     906        case GetExecutable: {
    901907            fixEdge<FunctionUse>(node->child1());
    902908            break;
    … …  
    904910           
    905911        case CheckStructure:
    906         case CheckFunction:
     912        case CheckCell:
    907913        case CheckHasInstance:
    908914        case CreateThis:
    … …  
    11211127        case Call:
    11221128        case Construct:
     1129        case ProfiledCall:
     1130        case ProfiledConstruct:
    11231131        case NativeCall:
    11241132        case NativeConstruct:
    … …  
    11501158        case CountExecution:
    11511159        case ForceOSRExit:
     1160        case CheckBadCell:
    11521161        case CheckWatchdogTimer:
    11531162        case Unreachable:
    … …  
    11601169        case MovHint:
    11611170        case ZombieHint:
     1171        case BottomValue:
    11621172            break;
    11631173#else

  • trunk/Source/JavaScriptCore/dfg/DFGGraph.cpp

    r172737 r172940  
    223223    if (node->hasTransition())
    224224        out.print(comma, pointerDumpInContext(node->transition(), context));
    225     if (node->hasFunction()) {
    226         out.print(comma, "function(", pointerDump(node->function()), ", ");
    227         if (node->function()->value().isCell()
    228             && node->function()->value().asCell()->inherits(JSFunction::info())) {
    229             JSFunction* function = jsCast<JSFunction*>(node->function()->value().asCell());
    230             if (function->isHostFunction())
    231                 out.print("<host function>");
    232             else
    233                 out.print(FunctionExecutableDump(function->jsExecutable()));
    234         } else
    235             out.print("<not JSFunction>");
    236         out.print(")");
    237     }
    238     if (node->hasExecutable()) {
    239         if (node->executable()->inherits(FunctionExecutable::info()))
    240             out.print(comma, "executable(", FunctionExecutableDump(jsCast<FunctionExecutable*>(node->executable())), ")");
    241         else
    242             out.print(comma, "executable(not function: ", RawPointer(node->executable()), ")");
     225    if (node->hasCellOperand()) {
     226        if (!node->cellOperand()->value() || !node->cellOperand()->value().isCell())
     227            out.print(comma, "invalid cell operand: ", node->cellOperand()->value());
     228        else {
     229            out.print(comma, pointerDump(node->cellOperand()->value().asCell()));
     230            if (node->cellOperand()->value().isCell()) {
     231                CallVariant variant(node->cellOperand()->value().asCell());
     232                if (ExecutableBase* executable = variant.executable()) {
     233                    if (executable->isHostFunction())
     234                        out.print(comma, "<host function>");
     235                    else if (FunctionExecutable* functionExecutable = jsDynamicCast<FunctionExecutable*>(executable))
     236                        out.print(comma, FunctionExecutableDump(functionExecutable));
     237                    else
     238                        out.print(comma, "<non-function executable>");
     239                }
     240            }
     241        }
    243242    }
    244243    if (node->hasFunctionDeclIndex()) {
    … …  
    986985           
    987986            switch (node->op()) {
    988             case CheckExecutable:
    989                 visitor.appendUnbarrieredReadOnlyPointer(node->executable());
    990                 break;
    991                
    992987            case CheckStructure:
    993988                for (unsigned i = node->structureSet().size(); i--;)

  • trunk/Source/JavaScriptCore/dfg/DFGJITCompiler.cpp

    r172867 r172940  
    189189        for (unsigned j = data.cases.size(); j--;) {
    190190            SwitchCase& myCase = data.cases[j];
    191             table.ctiOffsets[myCase.value.switchLookupValue() - table.min] =
     191            table.ctiOffsets[myCase.value.switchLookupValue(data.kind) - table.min] =
    192192                linkBuffer.locationOf(m_blockHeads[myCase.target.block->index]);
    193193        }

  • trunk/Source/JavaScriptCore/dfg/DFGLazyJSValue.cpp

    r171613 r172940  
    114114}
    115115
     116uintptr_t LazyJSValue::switchLookupValue(SwitchKind kind) const
     117{
     118    // NB. Not every kind of JSValue will be able to give you a switch lookup
     119    // value, and this method will assert, or do bad things, if you use it
     120    // for a kind of value that can't.
     121    switch (m_kind) {
     122    case KnownValue:
     123        switch (kind) {
     124        case SwitchImm:
     125            return value()->value().asInt32();
     126        case SwitchCell:
     127            return bitwise_cast<uintptr_t>(value()->value().asCell());
     128        default:
     129            RELEASE_ASSERT_NOT_REACHED();
     130            return 0;
     131        }
     132    case SingleCharacterString:
     133        switch (kind) {
     134        case SwitchChar:
     135            return character();
     136        default:
     137            RELEASE_ASSERT_NOT_REACHED();
     138            return 0;
     139        }
     140    default:
     141        RELEASE_ASSERT_NOT_REACHED();
     142        return 0;
     143    }
     144}
     145
    116146void LazyJSValue::dumpInContext(PrintStream& out, DumpContext* context) const
    117147{

  • trunk/Source/JavaScriptCore/dfg/DFGLazyJSValue.h

    r171613 r172940  
    2929#if ENABLE(DFG_JIT)
    3030
     31#include "DFGCommon.h"
    3132#include "DFGFrozenValue.h"
    3233#include <wtf/text/StringImpl.h>
    … …  
    9697    TriState strictEqual(const LazyJSValue& other) const;
    9798   
    98     unsigned switchLookupValue() const
    99     {
    100         // NB. Not every kind of JSValue will be able to give you a switch lookup
    101         // value, and this method will assert, or do bad things, if you use it
    102         // for a kind of value that can't.
    103         switch (m_kind) {
    104         case KnownValue:
    105             return value()->value().asInt32();
    106         case SingleCharacterString:
    107             return character();
    108         default:
    109             RELEASE_ASSERT_NOT_REACHED();
    110             return 0;
    111         }
    112     }
     99    uintptr_t switchLookupValue(SwitchKind) const;
    113100   
    114101    void dump(PrintStream&) const;

  • trunk/Source/JavaScriptCore/dfg/DFGNode.cpp

    r171660 r172940  
    114114        out.print("SwitchString");
    115115        return;
     116    case SwitchCell:
     117        out.print("SwitchCell");
     118        return;
    116119    }
    117120    RELEASE_ASSERT_NOT_REACHED();

  • trunk/Source/JavaScriptCore/dfg/DFGNode.h

    r172176 r172940  
    158158};
    159159
    160 enum SwitchKind {
    161     SwitchImm,
    162     SwitchChar,
    163     SwitchString
    164 };
    165 
    166160struct SwitchData {
    167161    // Initializes most fields to obviously invalid values. Anyone
    … …  
    186180// a constant index, argument, or identifier) from a Node*.
    187181struct OpInfo {
     182    OpInfo() : m_value(0) { }
    188183    explicit OpInfo(int32_t value) : m_value(static_cast<uintptr_t>(value)) { }
    189184    explicit OpInfo(uint32_t value) : m_value(static_cast<uintptr_t>(value)) { }
    … …  
    10101005        case Call:
    10111006        case Construct:
     1007        case ProfiledCall:
     1008        case ProfiledConstruct:
    10121009        case NativeCall:
    10131010        case NativeConstruct:
    … …  
    10451042    }
    10461043   
    1047     bool canBeKnownFunction()
    1048     {
    1049         switch (op()) {
     1044    bool hasCellOperand()
     1045    {
     1046        switch (op()) {
     1047        case AllocationProfileWatchpoint:
     1048        case CheckCell:
    10501049        case NativeConstruct:
    10511050        case NativeCall:
    … …  
    10561055    }
    10571056
    1058     bool hasKnownFunction()
    1059     {
    1060         switch (op()) {
    1061         case NativeConstruct:
    1062         case NativeCall:
    1063             return (bool)m_opInfo;
    1064         default:
    1065             return false;
    1066         }
    1067     }
    1068    
    1069     JSFunction* knownFunction()
    1070     {
    1071         ASSERT(canBeKnownFunction());
    1072         return bitwise_cast<JSFunction*>(m_opInfo);
    1073     }
    1074 
    1075     void giveKnownFunction(JSFunction* callData)
    1076     {
    1077         ASSERT(canBeKnownFunction());
    1078         m_opInfo = bitwise_cast<uintptr_t>(callData);
    1079     }
    1080 
    1081     bool hasFunction()
    1082     {
    1083         switch (op()) {
    1084         case CheckFunction:
    1085         case AllocationProfileWatchpoint:
    1086             return true;
    1087         default:
    1088             return false;
    1089         }
    1090     }
    1091 
    1092     FrozenValue* function()
    1093     {
    1094         ASSERT(hasFunction());
     1057    FrozenValue* cellOperand()
     1058    {
     1059        ASSERT(hasCellOperand());
    10951060        return reinterpret_cast<FrozenValue*>(m_opInfo);
    10961061    }
    10971062   
    1098     bool hasExecutable()
    1099     {
    1100         return op() == CheckExecutable;
    1101     }
    1102    
    1103     ExecutableBase* executable()
    1104     {
    1105         return jsCast<ExecutableBase*>(reinterpret_cast<JSCell*>(m_opInfo));
     1063    void setCellOperand(FrozenValue* value)
     1064    {
     1065        ASSERT(hasCellOperand());
     1066        m_opInfo = bitwise_cast<uintptr_t>(value);
    11061067    }
    11071068   

  • trunk/Source/JavaScriptCore/dfg/DFGNodeType.h

    r172808 r172940  
    154154    macro(PutByIdDirect, NodeMustGenerate | NodeClobbersWorld) \
    155155    macro(CheckStructure, NodeMustGenerate) \
    156     macro(CheckExecutable, NodeMustGenerate) \
     156    macro(GetExecutable, NodeResultJS) \
    157157    macro(PutStructure, NodeMustGenerate) \
    158158    macro(AllocatePropertyStorage, NodeMustGenerate | NodeResultStorage) \
    … …  
    186186    macro(VarInjectionWatchpoint, NodeMustGenerate) \
    187187    macro(FunctionReentryWatchpoint, NodeMustGenerate) \
    188     macro(CheckFunction, NodeMustGenerate) \
     188    macro(CheckCell, NodeMustGenerate) \
     189    macro(CheckBadCell, NodeMustGenerate) \
    189190    macro(AllocationProfileWatchpoint, NodeMustGenerate) \
    190191    macro(CheckInBounds, NodeMustGenerate) \
    … …  
    215216    macro(Call, NodeResultJS | NodeMustGenerate | NodeHasVarArgs | NodeClobbersWorld) \
    216217    macro(Construct, NodeResultJS | NodeMustGenerate | NodeHasVarArgs | NodeClobbersWorld) \
     218    macro(ProfiledCall, NodeResultJS | NodeMustGenerate | NodeHasVarArgs | NodeClobbersWorld) \
     219    macro(ProfiledConstruct, NodeResultJS | NodeMustGenerate | NodeHasVarArgs | NodeClobbersWorld) \
    217220    macro(NativeCall, NodeResultJS | NodeMustGenerate | NodeHasVarArgs | NodeClobbersWorld) \
    218221    macro(NativeConstruct, NodeResultJS | NodeMustGenerate | NodeHasVarArgs | NodeClobbersWorld) \
    … …  
    287290    macro(ForceOSRExit, NodeMustGenerate) \
    288291    \
     292    /* Vends a bottom JS value. It is invalid to ever execute this. Useful for cases */\
     293    /* where we know that we would have exited but we'd like to still track the control */\
     294    /* flow. */\
     295    macro(BottomValue, NodeResultJS) \
     296    \
    289297    /* Checks the watchdog timer. If the timer has fired, we OSR exit to the */ \
    290298    /* baseline JIT to redo the watchdog timer check, and service the timer. */ \

  • trunk/Source/JavaScriptCore/dfg/DFGPhantomCanonicalizationPhase.cpp

    r172176 r172940  
    9393                    }
    9494                   
    95                     if (node->children.isEmpty())
     95                    if (node->children.isEmpty()) {
     96                        m_graph.m_allocator.free(node);
    9697                        continue;
     98                    }
    9799                   
    98100                    node->convertToCheck();

  • trunk/Source/JavaScriptCore/dfg/DFGPhantomRemovalPhase.cpp

    r172176 r172940  
    126126                   
    127127                    if (node->children.isEmpty()) {
     128                        m_graph.m_allocator.free(node);
    128129                        changed = true;
    129130                        continue;
    … …  
    143144                    }
    144145                    if (node->children.isEmpty()) {
     146                        m_graph.m_allocator.free(node);
    145147                        changed = true;
    146148                        continue;
    … …  
    150152                   
    151153                case HardPhantom: {
    152                     if (node->children.isEmpty())
     154                    if (node->children.isEmpty()) {
     155                        m_graph.m_allocator.free(node);
    153156                        continue;
     157                    }
    154158                    break;
    155159                }

  • trunk/Source/JavaScriptCore/dfg/DFGPredictionPropagationPhase.cpp

    r172808 r172940  
    189189        case Call:
    190190        case Construct:
     191        case ProfiledCall:
     192        case ProfiledConstruct:
    191193        case NativeCall:
    192194        case NativeConstruct:
    … …  
    197199        }
    198200           
    199         case GetGetterSetterByOffset: {
     201        case GetGetterSetterByOffset:
     202        case GetExecutable: {
    200203            changed |= setPrediction(SpecCellOther);
    201204            break;
    … …  
    643646        case SetArgument:
    644647        case CheckStructure:
    645         case CheckExecutable:
    646         case CheckFunction:
     648        case CheckCell:
     649        case CheckBadCell:
    647650        case PutStructure:
    648651        case TearOffActivation:
    … …  
    666669            break;
    667670           
     671        // This gets ignored because it only pretends to produce a value.
     672        case BottomValue:
     673            break;
     674           
    668675        // This gets ignored because it already has a prediction.
    669676        case ExtractOSREntryLocal:

  • trunk/Source/JavaScriptCore/dfg/DFGSafeToExecute.h

    r172808 r172940  
    160160    case PutByIdDirect:
    161161    case CheckStructure:
    162     case CheckExecutable:
     162    case GetExecutable:
    163163    case GetButterfly:
    164164    case CheckArray:
    … …  
    175175    case VariableWatchpoint:
    176176    case VarInjectionWatchpoint:
    177     case CheckFunction:
     177    case CheckCell:
     178    case CheckBadCell:
    178179    case AllocationProfileWatchpoint:
    179180    case RegExpExec:
    … …  
    188189    case Call:
    189190    case Construct:
     191    case ProfiledCall:
     192    case ProfiledConstruct:
    190193    case NewObject:
    191194    case NewArray:
    … …  
    274277        return false; // TODO: add a check for already checked.  https://bugs.webkit.org/show_bug.cgi?id=133769
    275278
     279    case BottomValue:
     280        // If in doubt, assume that this isn't safe to execute, just because we have no way of
     281        // compiling this node.
     282        return false;
     283
    276284    case GetByVal:
    277285    case GetIndexedPropertyStorage:

  • trunk/Source/JavaScriptCore/dfg/DFGSpeculativeJIT.cpp

    r172176 r172940  
    53555355        emitSwitchString(node, data);
    53565356        return;
     5357    }
     5358    case SwitchCell: {
     5359        DFG_CRASH(m_jit.graph(), node, "Bad switch kind");
     5360        return;
    53575361    } }
    53585362    RELEASE_ASSERT_NOT_REACHED();

  • trunk/Source/JavaScriptCore/dfg/DFGSpeculativeJIT32_64.cpp

    r172808 r172940  
    641641void SpeculativeJIT::emitCall(Node* node)
    642642{
    643     bool isCall = node->op() == Call;
     643    bool isCall = node->op() == Call || node->op() == ProfiledCall;
    644644    if (!isCall)
    645         ASSERT(node->op() == Construct);
     645        ASSERT(node->op() == Construct || node->op() == ProfiledConstruct);
    646646
    647647    // For constructors, the this argument is not passed but we have to make space
    … …  
    689689
    690690    m_jit.emitStoreCodeOrigin(node->origin.semantic);
     691   
     692    CallLinkInfo* info = m_jit.codeBlock()->addCallLinkInfo();
     693
     694    if (node->op() == ProfiledCall || node->op() == ProfiledConstruct) {
     695        m_jit.vm()->callEdgeLog->emitLogCode(
     696            m_jit, info->callEdgeProfile, callee.jsValueRegs());
     697    }
    691698   
    692699    slowPath.append(branchNotCell(callee.jsValueRegs()));
    … …  
    714721        m_jit.move(calleeTagGPR, GPRInfo::regT1);
    715722    }
    716     CallLinkInfo* info = m_jit.codeBlock()->addCallLinkInfo();
    717723    m_jit.move(MacroAssembler::TrustedImmPtr(info), GPRInfo::regT2);
    718724    JITCompiler::Call slowCall = m_jit.nearCall();
    … …  
    36763682        break;
    36773683       
    3678     case CheckFunction: {
     3684    case CheckCell: {
     3685        SpeculateCellOperand cell(this, node->child1());
     3686        speculationCheck(BadCell, JSValueSource::unboxedCell(cell.gpr()), node->child1(), m_jit.branchWeakPtr(JITCompiler::NotEqual, cell.gpr(), node->cellOperand()->value().asCell()));
     3687        noResult(node);
     3688        break;
     3689    }
     3690
     3691    case GetExecutable: {
    36793692        SpeculateCellOperand function(this, node->child1());
    3680         speculationCheck(BadFunction, JSValueSource::unboxedCell(function.gpr()), node->child1(), m_jit.branchWeakPtr(JITCompiler::NotEqual, function.gpr(), node->function()->value().asCell()));
    3681         noResult(node);
    3682         break;
    3683     }
    3684 
    3685     case CheckExecutable: {
    3686         SpeculateCellOperand function(this, node->child1());
    3687         speculateCellType(node->child1(), function.gpr(), SpecFunction, JSFunctionType);
    3688         speculationCheck(BadExecutable, JSValueSource::unboxedCell(function.gpr()), node->child1(), m_jit.branchWeakPtr(JITCompiler::NotEqual, JITCompiler::Address(function.gpr(), JSFunction::offsetOfExecutable()), node->executable()));
    3689         noResult(node);
     3693        GPRTemporary result(this, Reuse, function);
     3694        GPRReg functionGPR = function.gpr();
     3695        GPRReg resultGPR = result.gpr();
     3696        speculateCellType(node->child1(), functionGPR, SpecFunction, JSFunctionType);
     3697        m_jit.loadPtr(JITCompiler::Address(functionGPR, JSFunction::offsetOfExecutable()), resultGPR);
     3698        cellResult(resultGPR, node);
    36903699        break;
    36913700    }
    … …  
    41574166    case Call:
    41584167    case Construct:
     4168    case ProfiledCall:
     4169    case ProfiledConstruct:
    41594170        emitCall(node);
    41604171        break;
    … …  
    48934904    case NativeCall:
    48944905    case NativeConstruct:
     4906    case CheckBadCell:
     4907    case BottomValue:
    48954908        RELEASE_ASSERT_NOT_REACHED();
    48964909        break;

  • trunk/Source/JavaScriptCore/dfg/DFGSpeculativeJIT64.cpp

    r172808 r172940  
    627627void SpeculativeJIT::emitCall(Node* node)
    628628{
    629 
    630     bool isCall = node->op() == Call;
     629    bool isCall = node->op() == Call || node->op() == ProfiledCall;
    631630    if (!isCall)
    632         DFG_ASSERT(m_jit.graph(), node, node->op() == Construct);
     631        DFG_ASSERT(m_jit.graph(), node, node->op() == Construct || node->op() == ProfiledConstruct);
    633632   
    634633    // For constructors, the this argument is not passed but we have to make space
    … …  
    671670    m_jit.emitStoreCodeOrigin(node->origin.semantic);
    672671   
     672    CallLinkInfo* callLinkInfo = m_jit.codeBlock()->addCallLinkInfo();
     673   
     674    if (node->op() == ProfiledCall || node->op() == ProfiledConstruct) {
     675        m_jit.vm()->callEdgeLog->emitLogCode(
     676            m_jit, callLinkInfo->callEdgeProfile, JSValueRegs(calleeGPR));
     677    }
     678
    673679    slowPath = m_jit.branchPtrWithPatch(MacroAssembler::NotEqual, calleeGPR, targetToCheck, MacroAssembler::TrustedImmPtr(0));
    674680
    … …  
    683689   
    684690    m_jit.move(calleeGPR, GPRInfo::regT0); // Callee needs to be in regT0
    685     CallLinkInfo* callLinkInfo = m_jit.codeBlock()->addCallLinkInfo();
    686691    m_jit.move(MacroAssembler::TrustedImmPtr(callLinkInfo), GPRInfo::regT2); // Link info needs to be in regT2
    687692    JITCompiler::Call slowCall = m_jit.nearCall();
    … …  
    37693774        break;
    37703775       
    3771     case CheckFunction: {
     3776    case CheckCell: {
     3777        SpeculateCellOperand cell(this, node->child1());
     3778        speculationCheck(BadCell, JSValueSource::unboxedCell(cell.gpr()), node->child1(), m_jit.branchWeakPtr(JITCompiler::NotEqual, cell.gpr(), node->cellOperand()->value().asCell()));
     3779        noResult(node);
     3780        break;
     3781    }
     3782       
     3783    case GetExecutable: {
    37723784        SpeculateCellOperand function(this, node->child1());
    3773         speculationCheck(BadFunction, JSValueSource::unboxedCell(function.gpr()), node->child1(), m_jit.branchWeakPtr(JITCompiler::NotEqual, function.gpr(), node->function()->value().asCell()));
    3774         noResult(node);
    3775         break;
    3776     }
    3777        
    3778     case CheckExecutable: {
    3779         SpeculateCellOperand function(this, node->child1());
    3780         speculateCellType(node->child1(), function.gpr(), SpecFunction, JSFunctionType);
    3781         speculationCheck(BadExecutable, JSValueSource::unboxedCell(function.gpr()), node->child1(), m_jit.branchWeakPtr(JITCompiler::NotEqual, JITCompiler::Address(function.gpr(), JSFunction::offsetOfExecutable()), node->executable()));
    3782         noResult(node);
     3785        GPRTemporary result(this, Reuse, function);
     3786        GPRReg functionGPR = function.gpr();
     3787        GPRReg resultGPR = result.gpr();
     3788        speculateCellType(node->child1(), functionGPR, SpecFunction, JSFunctionType);
     3789        m_jit.loadPtr(JITCompiler::Address(functionGPR, JSFunction::offsetOfExecutable()), resultGPR);
     3790        cellResult(resultGPR, node);
    37833791        break;
    37843792    }
    … …  
    42204228    case Call:
    42214229    case Construct:
     4230    case ProfiledCall:
     4231    case ProfiledConstruct:
    42224232        emitCall(node);
    42234233        break;
    4224 
     4234       
    42254235    case CreateActivation: {
    42264236        DFG_ASSERT(m_jit.graph(), node, !node->origin.semantic.inlineCallFrame);
    … …  
    49714981    case MultiPutByOffset:
    49724982    case FiatInt52:
    4973         DFG_CRASH(m_jit.graph(), node, "Unexpected FTL node");
     4983    case CheckBadCell:
     4984    case BottomValue:
     4985        DFG_CRASH(m_jit.graph(), node, "Unexpected node");
    49744986        break;
    49754987    }

  • trunk/Source/JavaScriptCore/dfg/DFGStructureRegistrationPhase.cpp

    r172737 r172940  
    6363           
    6464                switch (node->op()) {
    65                 case CheckExecutable:
    66                     registerStructure(node->executable()->structure());
    67                     break;
    68                
    6965                case CheckStructure:
    7066                    registerStructures(node->structureSet());

  • trunk/Source/JavaScriptCore/dfg/DFGTierUpCheckInjectionPhase.cpp

    r164229 r172940  
    5151            return false;
    5252       
    53         if (m_graph.m_profiledBlock->m_didFailFTLCompilation)
     53        if (m_graph.m_profiledBlock->m_didFailFTLCompilation) {
     54            removeFTLProfiling();
    5455            return false;
     56        }
    5557       
    5658#if ENABLE(FTL_JIT)
    5759        FTL::CapabilityLevel level = FTL::canCompile(m_graph);
    58         if (level == FTL::CannotCompile)
     60        if (level == FTL::CannotCompile) {
     61            removeFTLProfiling();
    5962            return false;
     63        }
    6064       
    6165        if (!Options::enableOSREntryToFTL())
    … …  
    119123#endif // ENABLE(FTL_JIT)
    120124    }
     125
     126private:
     127    void removeFTLProfiling()
     128    {
     129        for (BlockIndex blockIndex = m_graph.numBlocks(); blockIndex--;) {
     130            BasicBlock* block = m_graph.block(blockIndex);
     131            if (!block)
     132                continue;
     133           
     134            for (unsigned nodeIndex = 0; nodeIndex < block->size(); ++nodeIndex) {
     135                Node* node = block->at(nodeIndex);
     136                switch (node->op()) {
     137                case ProfiledCall:
     138                    node->setOp(Call);
     139                    break;
     140                   
     141                case ProfiledConstruct:
     142                    node->setOp(Construct);
     143                    break;
     144                   
     145                default:
     146                    break;
     147                }
     148            }
     149        }
     150    }
    121151};
    122152

  • trunk/Source/JavaScriptCore/dfg/DFGValidate.cpp

    r171660 r172940  
    201201                VALIDATE((node), !mayExit(m_graph, node) || node->origin.forExit.isSet());
    202202                VALIDATE((node), !node->hasStructure() || !!node->structure());
    203                 VALIDATE((node), !node->hasFunction() || node->function()->value().isFunction());
     203                VALIDATE((node), !node->hasCellOperand() || node->cellOperand()->value().isCell());
     204                VALIDATE((node), !node->hasCellOperand() || !!node->cellOperand()->value());
    204205                 
    205206                if (!(node->flags() & NodeHasVarArgs)) {

  • trunk/Source/JavaScriptCore/dfg/DFGWatchpointCollectionPhase.cpp

    r171613 r172940  
    11/*
    2  * Copyright (C) 2013 Apple Inc. All rights reserved.
     2 * Copyright (C) 2013, 2014 Apple Inc. All rights reserved.
    33 *
    44 * Redistribution and use in source and binary forms, with or without
    … …  
    115115           
    116116        case AllocationProfileWatchpoint:
    117             addLazily(jsCast<JSFunction*>(m_node->function()->value())->allocationProfileWatchpointSet());
     117            addLazily(jsCast<JSFunction*>(m_node->cellOperand()->value())->allocationProfileWatchpointSet());
    118118            break;
    119119           

  • trunk/Source/JavaScriptCore/ftl/FTLCapabilities.cpp

    r172176 r172940  
    105105    case InvalidationPoint:
    106106    case StringCharAt:
    107     case CheckFunction:
     107    case CheckCell:
     108    case CheckBadCell:
    108109    case StringCharCodeAt:
    109110    case AllocatePropertyStorage:
    … …  
    127128    case Check:
    128129    case CountExecution:
    129     case CheckExecutable:
     130    case GetExecutable:
    130131    case GetScope:
    131132    case AllocationProfileWatchpoint:
    … …  
    167168    case GetEnumeratorPname:
    168169    case ToIndexString:
     170    case BottomValue:
    169171        // These are OK.
     172        break;
     173    case ProfiledCall:
     174    case ProfiledConstruct:
     175        // These are OK not because the FTL can support them, but because if the DFG sees one of
     176        // these then the FTL will see a normal Call/Construct.
    170177        break;
    171178    case Identity:
    … …  
    327334        case SwitchImm:
    328335        case SwitchChar:
     336        case SwitchCell:
    329337            break;
    330338        default:

  • trunk/Source/JavaScriptCore/ftl/FTLLowerDFGToLLVM.cpp

    r172756 r172940  
    6868    CodeBlock* codeBlock, BlockIndex blockIndex, unsigned nodeIndex)
    6969{
    70    
    7170    dataLog("Crashing in thought-to-be-unreachable FTL-generated code for ", pointerDump(codeBlock), " at basic block #", blockIndex);
    7271    if (nodeIndex != UINT_MAX)
    … …  
    154153            BasicBlock* block = depthFirst[blockIndex];
    155154            for (unsigned nodeIndex = block->size(); nodeIndex--; ) {
    156                 Node* m_node = block->at(nodeIndex);
    157                 if (m_node->hasKnownFunction()) {
     155                Node* node = block->at(nodeIndex);
     156                switch (node->op()) {
     157                case NativeCall:
     158                case NativeConstruct: {
    158159                    int numArgs = m_node->numChildren();
    159160                    if (numArgs > maxNumberOfArguments)
    160161                        maxNumberOfArguments = numArgs;
     162                    break;
     163                }
     164                default:
     165                    break;
    161166                }
    162167            }
    … …  
    469474            compileCheckStructure();
    470475            break;
    471         case CheckFunction:
    472             compileCheckFunction();
    473             break;
    474         case CheckExecutable:
    475             compileCheckExecutable();
     476        case CheckCell:
     477            compileCheckCell();
     478            break;
     479        case CheckBadCell:
     480            compileCheckBadCell();
     481            break;
     482        case GetExecutable:
     483            compileGetExecutable();
    476484            break;
    477485        case ArrayifyToStructure:
    … …  
    17441752    }
    17451753   
    1746     void compileCheckFunction()
     1754    void compileCheckCell()
    17471755    {
    17481756        LValue cell = lowCell(m_node->child1());
    17491757       
    17501758        speculate(
    1751             BadFunction, jsValueValue(cell), m_node->child1().node(),
    1752             m_out.notEqual(cell, weakPointer(m_node->function()->value().asCell())));
    1753     }
    1754    
    1755     void compileCheckExecutable()
     1759            BadCell, jsValueValue(cell), m_node->child1().node(),
     1760            m_out.notEqual(cell, weakPointer(m_node->cellOperand()->value().asCell())));
     1761    }
     1762   
     1763    void compileCheckBadCell()
     1764    {
     1765        terminate(BadCell);
     1766    }
     1767   
     1768    void compileGetExecutable()
    17561769    {
    17571770        LValue cell = lowCell(m_node->child1());
    1758        
    17591771        speculateFunction(m_node->child1(), cell);
    1760        
    1761         speculate(
    1762             BadExecutable, jsValueValue(cell), m_node->child1().node(),
    1763             m_out.notEqual(
    1764                 m_out.loadPtr(cell, m_heaps.JSFunction_executable),
    1765                 weakPointer(m_node->executable())));
     1772        setJSValue(m_out.loadPtr(cell, m_heaps.JSFunction_executable));
    17661773    }
    17671774   
    … …  
    36743681        int numArgs = numPassedArgs + dummyThisArgument;
    36753682
    3676         ASSERT(m_node->hasKnownFunction());
    3677 
    3678         JSFunction* knownFunction = m_node->knownFunction();
     3683        JSFunction* knownFunction = jsCast<JSFunction*>(m_node->cellOperand()->value().asCell());
    36793684        NativeFunction function = knownFunction->nativeFunction();
    36803685
    … …  
    39193924        }
    39203925       
    3921         case SwitchString:
     3926        case SwitchString: {
    39223927            DFG_CRASH(m_graph, m_node, "Unimplemented");
    3923             break;
    3924         }
     3928            return;
     3929        }
     3930           
     3931        case SwitchCell: {
     3932            LValue cell;
     3933            switch (m_node->child1().useKind()) {
     3934            case CellUse: {
     3935                cell = lowCell(m_node->child1());
     3936                break;
     3937            }
     3938               
     3939            case UntypedUse: {
     3940                LValue value = lowJSValue(m_node->child1());
     3941                LBasicBlock cellCase = FTL_NEW_BLOCK(m_out, ("Switch/SwitchCell cell case"));
     3942                m_out.branch(
     3943                    isCell(value), unsure(cellCase), unsure(lowBlock(data->fallThrough.block)));
     3944                m_out.appendTo(cellCase);
     3945                cell = value;
     3946                break;
     3947            }
     3948               
     3949            default:
     3950                DFG_CRASH(m_graph, m_node, "Bad use kind");
     3951                return;
     3952            }
     3953           
     3954            buildSwitch(m_node->switchData(), m_out.intPtr, cell);
     3955            return;
     3956        } }
    39253957       
    39263958        DFG_CRASH(m_graph, m_node, "Bad switch kind");
    … …  
    51875219        for (unsigned i = 0; i < data->cases.size(); ++i) {
    51885220            cases.append(SwitchCase(
    5189                 constInt(type, data->cases[i].value.switchLookupValue()),
     5221                constInt(type, data->cases[i].value.switchLookupValue(data->kind)),
    51905222                lowBlock(data->cases[i].target.block), Weight(data->cases[i].target.count)));
    51915223        }

  • trunk/Source/JavaScriptCore/heap/Heap.cpp

    r172820 r172940  
    985985    }
    986986   
     987    if (vm()->callEdgeLog) {
     988        DeferGCForAWhile awhile(*this);
     989        vm()->callEdgeLog->processLog();
     990    }
     991   
    987992    RELEASE_ASSERT(!m_deferralDepth);
    988993    ASSERT(vm()->currentThreadIsHoldingAPILock());

  • trunk/Source/JavaScriptCore/jit/AssemblyHelpers.h

    r171213 r172940  
    8989    }
    9090   
     91    void storeValue(JSValueRegs regs, void* address)
     92    {
     93#if USE(JSVALUE64)
     94        store64(regs.gpr(), address);
     95#else
     96        store32(regs.payloadGPR(), bitwise_cast<void*>(bitwise_cast<uintptr_t>(address) + PayloadOffset));
     97        store32(regs.tagGPR(), bitwise_cast<void*>(bitwise_cast<uintptr_t>(address) + TagOffset));
     98#endif
     99    }
     100   
     101    void loadValue(Address address, JSValueRegs regs)
     102    {
     103#if USE(JSVALUE64)
     104        load64(address, regs.gpr());
     105#else
     106        if (address.base == regs.payloadGPR()) {
     107            load32(address.withOffset(TagOffset), regs.tagGPR());
     108            load32(address.withOffset(PayloadOffset), regs.payloadGPR());
     109        } else {
     110            load32(address.withOffset(PayloadOffset), regs.payloadGPR());
     111            load32(address.withOffset(TagOffset), regs.tagGPR());
     112        }
     113#endif
     114    }
     115   
    91116    void moveTrustedValue(JSValue value, JSValueRegs regs)
    92117    {

  • trunk/Source/JavaScriptCore/jit/CCallHelpers.h

    r172867 r172940  
    16671667    }
    16681668#endif
     1669   
     1670    void setupArguments(JSValueRegs arg1)
     1671    {
     1672#if USE(JSVALUE64)
     1673        setupArguments(arg1.gpr());
     1674#else
     1675        setupArguments(arg1.payloadGPR(), arg1.tagGPR());
     1676#endif
     1677    }
    16691678
    16701679    void setupResults(GPRReg destA, GPRReg destB)

  • trunk/Source/JavaScriptCore/jit/GPRInfo.h

    r172734 r172940  
    6161    GPRReg payloadGPR() const { return m_gpr; }
    6262   
     63    bool uses(GPRReg gpr) const { return m_gpr == gpr; }
     64   
    6365private:
    6466    GPRReg m_gpr;
    … …  
    170172    }
    171173
     174    bool uses(GPRReg gpr) const { return m_tagGPR == gpr || m_payloadGPR == gpr; }
     175   
    172176private:
    173177    int8_t m_tagGPR;

  • trunk/Source/JavaScriptCore/jit/JITCall.cpp

    r172176 r172940  
    213213
    214214    store64(regT0, Address(stackPointerRegister, JSStack::Callee * static_cast<int>(sizeof(Register)) - sizeof(CallerFrameAndPC)));
     215   
     216    CallLinkInfo* info = m_codeBlock->addCallLinkInfo();
     217
     218    if (CallEdgeLog::isEnabled() && shouldEmitProfiling()
     219        && Options::baselineDoesCallEdgeProfiling())
     220        m_vm->ensureCallEdgeLog().emitLogCode(*this, info->callEdgeProfile, JSValueRegs(regT0));
    215221
    216222    if (opcodeID == op_call_eval) {
    … …  
    224230
    225231    ASSERT(m_callCompilationInfo.size() == callLinkInfoIndex);
    226     CallLinkInfo* info = m_codeBlock->addCallLinkInfo();
    227232    info->callType = CallLinkInfo::callTypeFor(opcodeID);
    228233    info->codeOrigin = CodeOrigin(m_bytecodeOffset);

  • trunk/Source/JavaScriptCore/jit/JITCall32_64.cpp

    r172176 r172940  
    301301    store32(regT1, Address(stackPointerRegister, JSStack::Callee * static_cast<int>(sizeof(Register)) + TagOffset - sizeof(CallerFrameAndPC)));
    302302
     303    CallLinkInfo* info = m_codeBlock->addCallLinkInfo();
     304
     305    if (CallEdgeLog::isEnabled() && shouldEmitProfiling()
     306        && Options::baselineDoesCallEdgeProfiling()) {
     307        m_vm->ensureCallEdgeLog().emitLogCode(
     308            *this, info->callEdgeProfile, JSValueRegs(regT1, regT0));
     309    }
     310
    303311    if (opcodeID == op_call_eval) {
    304312        compileCallEval(instruction);
    … …  
    314322
    315323    ASSERT(m_callCompilationInfo.size() == callLinkInfoIndex);
    316     CallLinkInfo* info = m_codeBlock->addCallLinkInfo();
    317324    info->callType = CallLinkInfo::callTypeFor(opcodeID);
    318325    info->codeOrigin = CodeOrigin(m_bytecodeOffset);

  • trunk/Source/JavaScriptCore/runtime/Options.h

    r172820 r172940  
    168168    v(bool, enablePolyvariantDevirtualization, true) \
    169169    v(bool, enablePolymorphicAccessInlining, true) \
     170    v(bool, enablePolymorphicCallInlining, true) \
     171    v(bool, callStatusShouldUseCallEdgeProfile, true) \
     172    v(bool, callEdgeProfileReallyProcessesLog, true) \
     173    v(bool, baselineDoesCallEdgeProfiling, false) \
     174    v(bool, dfgDoesCallEdgeProfiling, true) \
     175    v(bool, enableCallEdgeProfiling, true) \
     176    v(unsigned, frequentCallThreshold, 2) \
    170177    v(bool, optimizeNativeCalls, false) \
    171178    \

  • trunk/Source/JavaScriptCore/runtime/VM.cpp

    r172820 r172940  
    374374}
    375375
     376CallEdgeLog& VM::ensureCallEdgeLog()
     377{
     378    if (!callEdgeLog)
     379        callEdgeLog = std::make_unique<CallEdgeLog>();
     380    return *callEdgeLog;
     381}
     382
    376383#if ENABLE(JIT)
    377384static ThunkGenerator thunkGeneratorForIntrinsic(Intrinsic intrinsic)

  • trunk/Source/JavaScriptCore/runtime/VM.h

    r172867 r172940  
    7373    class ArityCheckFailReturnThunks;
    7474    class BuiltinExecutables;
     75    class CallEdgeLog;
    7576    class CodeBlock;
    7677    class CodeCache;
    … …  
    234235        OwnPtr<DFG::LongLivedState> dfgState;
    235236#endif // ENABLE(DFG_JIT)
     237       
     238        std::unique_ptr<CallEdgeLog> callEdgeLog;
     239        CallEdgeLog& ensureCallEdgeLog();
    236240
    237241        VMType vmType;
Note: See TracChangeset for help on using the changeset viewer.