Changeset 179478 in webkit for trunk/Source/JavaScriptCore/dfg/DFGByteCodeParser.cpp

Timestamp:

Feb 2, 2015, 10:38:08 AM (11 years ago)

Author:

[email protected]

Message:

Polymorphic call inlining should be based on polymorphic call inline caching rather than logging
​https://bugs.webkit.org/show_bug.cgi?id=140660

Reviewed by Geoffrey Garen.

When we first implemented polymorphic call inlining, we did the profiling based on a call
edge log. The idea was to store each call edge (a tuple of call site and callee) into a
global log that was processed lazily. Processing the log would give precise counts of call
edges, and could be used to drive well-informed inlining decisions - polymorphic or not.
This was a speed-up on throughput tests but a slow-down for latency tests. It was a net win
nonetheless.

Experience with this code shows three things. First, the call edge profiler is buggy and
complex. It would take work to fix the bugs. Second, the call edge profiler incurs lots of
overhead for latency code that we care deeply about. Third, it's not at all clear that
having call edge counts for every possible callee is any better than just having call edge
counts for the limited number of callees that an inline cache would catch.

So, this patch removes the call edge profiler and replaces it with a polymorphic call inline
cache. If we miss the basic call inline cache, we inflate the cache to be a jump to an
out-of-line stub that cases on the previously known callees. If that misses again, then we
rewrite that stub to include the new callee. We do this up to some number of callees. If we
hit the limit then we switch to using a plain virtual call.

Substantial speed-up on V8Spider; undoes the slow-down that the original call edge profiler
caused. Might be a SunSpider speed-up (below 1%), depending on hardware.

Rolling this back in after fixing ​https://bugs.webkit.org/show_bug.cgi?id=141107.

• CMakeLists.txt:
• JavaScriptCore.vcxproj/JavaScriptCore.vcxproj:
• JavaScriptCore.xcodeproj/project.pbxproj:
• bytecode/CallEdge.h:

(JSC::CallEdge::count):
(JSC::CallEdge::CallEdge):

• bytecode/CallEdgeProfile.cpp: Removed.
• bytecode/CallEdgeProfile.h: Removed.
• bytecode/CallEdgeProfileInlines.h: Removed.
• bytecode/CallLinkInfo.cpp:

(JSC::CallLinkInfo::unlink):
(JSC::CallLinkInfo::visitWeak):

• bytecode/CallLinkInfo.h:
• bytecode/CallLinkStatus.cpp:

(JSC::CallLinkStatus::CallLinkStatus):
(JSC::CallLinkStatus::computeFor):
(JSC::CallLinkStatus::computeFromCallLinkInfo):
(JSC::CallLinkStatus::isClosureCall):
(JSC::CallLinkStatus::makeClosureCall):
(JSC::CallLinkStatus::dump):
(JSC::CallLinkStatus::computeFromCallEdgeProfile): Deleted.

• bytecode/CallLinkStatus.h:

(JSC::CallLinkStatus::CallLinkStatus):
(JSC::CallLinkStatus::isSet):
(JSC::CallLinkStatus::variants):
(JSC::CallLinkStatus::size):
(JSC::CallLinkStatus::at):
(JSC::CallLinkStatus::operator[]):
(JSC::CallLinkStatus::canOptimize):
(JSC::CallLinkStatus::edges): Deleted.
(JSC::CallLinkStatus::canTrustCounts): Deleted.

• bytecode/CallVariant.cpp:

(JSC::variantListWithVariant):
(JSC::despecifiedVariantList):

• bytecode/CallVariant.h:
• bytecode/CodeBlock.cpp:

(JSC::CodeBlock::~CodeBlock):
(JSC::CodeBlock::linkIncomingPolymorphicCall):
(JSC::CodeBlock::unlinkIncomingCalls):
(JSC::CodeBlock::noticeIncomingCall):

• bytecode/CodeBlock.h:

(JSC::CodeBlock::isIncomingCallAlreadyLinked): Deleted.

• dfg/DFGAbstractInterpreterInlines.h:

(JSC::DFG::AbstractInterpreter<AbstractStateType>::executeEffects):

• dfg/DFGByteCodeParser.cpp:

(JSC::DFG::ByteCodeParser::addCallWithoutSettingResult):
(JSC::DFG::ByteCodeParser::handleCall):
(JSC::DFG::ByteCodeParser::handleInlining):

• dfg/DFGClobberize.h:

(JSC::DFG::clobberize):

• dfg/DFGConstantFoldingPhase.cpp:

(JSC::DFG::ConstantFoldingPhase::foldConstants):

• dfg/DFGDoesGC.cpp:

(JSC::DFG::doesGC):

• dfg/DFGDriver.cpp:

(JSC::DFG::compileImpl):

• dfg/DFGFixupPhase.cpp:

(JSC::DFG::FixupPhase::fixupNode):

• dfg/DFGNode.h:

(JSC::DFG::Node::hasHeapPrediction):

• dfg/DFGNodeType.h:
• dfg/DFGOperations.cpp:
• dfg/DFGPredictionPropagationPhase.cpp:

(JSC::DFG::PredictionPropagationPhase::propagate):

• dfg/DFGSafeToExecute.h:

(JSC::DFG::safeToExecute):

• dfg/DFGSpeculativeJIT32_64.cpp:

(JSC::DFG::SpeculativeJIT::emitCall):
(JSC::DFG::SpeculativeJIT::compile):

• dfg/DFGSpeculativeJIT64.cpp:

(JSC::DFG::SpeculativeJIT::emitCall):
(JSC::DFG::SpeculativeJIT::compile):

• dfg/DFGTierUpCheckInjectionPhase.cpp:

(JSC::DFG::TierUpCheckInjectionPhase::run):
(JSC::DFG::TierUpCheckInjectionPhase::removeFTLProfiling): Deleted.

• ftl/FTLCapabilities.cpp:

(JSC::FTL::canCompile):

• heap/Heap.cpp:

(JSC::Heap::collect):

• jit/BinarySwitch.h:
• jit/ClosureCallStubRoutine.cpp: Removed.
• jit/ClosureCallStubRoutine.h: Removed.
• jit/JITCall.cpp:

(JSC::JIT::compileOpCall):

• jit/JITCall32_64.cpp:

(JSC::JIT::compileOpCall):

• jit/JITOperations.cpp:
• jit/JITOperations.h:

(JSC::operationLinkPolymorphicCallFor):
(JSC::operationLinkClosureCallFor): Deleted.

• jit/JITStubRoutine.h:
• jit/JITWriteBarrier.h:
• jit/PolymorphicCallStubRoutine.cpp: Added.

(JSC::PolymorphicCallNode::~PolymorphicCallNode):
(JSC::PolymorphicCallNode::unlink):
(JSC::PolymorphicCallCase::dump):
(JSC::PolymorphicCallStubRoutine::PolymorphicCallStubRoutine):
(JSC::PolymorphicCallStubRoutine::~PolymorphicCallStubRoutine):
(JSC::PolymorphicCallStubRoutine::variants):
(JSC::PolymorphicCallStubRoutine::edges):
(JSC::PolymorphicCallStubRoutine::visitWeak):
(JSC::PolymorphicCallStubRoutine::markRequiredObjectsInternal):

• jit/PolymorphicCallStubRoutine.h: Added.

(JSC::PolymorphicCallNode::PolymorphicCallNode):
(JSC::PolymorphicCallCase::PolymorphicCallCase):
(JSC::PolymorphicCallCase::variant):
(JSC::PolymorphicCallCase::codeBlock):

• jit/Repatch.cpp:

(JSC::linkSlowFor):
(JSC::linkFor):
(JSC::revertCall):
(JSC::unlinkFor):
(JSC::linkVirtualFor):
(JSC::linkPolymorphicCall):
(JSC::linkClosureCall): Deleted.

• jit/Repatch.h:
• jit/ThunkGenerators.cpp:

(JSC::linkPolymorphicCallForThunkGenerator):
(JSC::linkPolymorphicCallThunkGenerator):
(JSC::linkPolymorphicCallThatPreservesRegsThunkGenerator):
(JSC::linkClosureCallForThunkGenerator): Deleted.
(JSC::linkClosureCallThunkGenerator): Deleted.
(JSC::linkClosureCallThatPreservesRegsThunkGenerator): Deleted.

• jit/ThunkGenerators.h:

(JSC::linkPolymorphicCallThunkGeneratorFor):
(JSC::linkClosureCallThunkGeneratorFor): Deleted.

• llint/LLIntSlowPaths.cpp:

(JSC::LLInt::jitCompileAndSetHeuristics):

• runtime/Options.h:
• runtime/VM.cpp:

(JSC::VM::prepareToDiscardCode):
(JSC::VM::ensureCallEdgeLog): Deleted.

• runtime/VM.h:

File:

• trunk/Source/JavaScriptCore/dfg/DFGByteCodeParser.cpp (modified) (12 diffs)

trunk/Source/JavaScriptCore/dfg/DFGByteCodeParser.cpp

@@ -1 +1 @@
  /*
- * Copyright (C) 2011, 2012, 2013, 2014 Apple Inc. All rights reserved.
+ * Copyright (C) 2011-2015 Apple Inc. All rights reserved.
  *
  * Redistribution and use in source and binary forms, with or without
@@ -676 +676 @@
             m_parameterSlots = parameterSlots;
 
-        int dummyThisArgument = op == Call || op == NativeCall || op == ProfiledCall ? 0 : 1;
+        int dummyThisArgument = op == Call || op == NativeCall ? 0 : 1;
         for (int i = 0 + dummyThisArgument; i < argCount; ++i)
             addVarArgChild(get(virtualRegisterForArgument(i, registerOffset)));
@@ -1045 +1045 @@
         callLinkStatus = CallLinkStatus(callTarget->asJSValue()).setIsProved(true);
     
-    if ((!callLinkStatus.canOptimize() || callLinkStatus.size() != 1)
-        && !isFTL(m_graph.m_plan.mode) && Options::useFTLJIT()
-        && InlineCallFrame::isNormalCall(kind)
-        && CallEdgeLog::isEnabled()
-        && Options::dfgDoesCallEdgeProfiling()) {
-        ASSERT(op == Call || op == Construct);
-        if (op == Call)
-            op = ProfiledCall;
-        else
-            op = ProfiledConstruct;
-    }
+    if (Options::verboseDFGByteCodeParsing())
+        dataLog("    Handling call at ", currentCodeOrigin(), ": ", callLinkStatus, "\n");
     
     if (!callLinkStatus.canOptimize()) {
@@ -1077 +1068 @@
 #if ENABLE(FTL_NATIVE_CALL_INLINING)
     if (isFTL(m_graph.m_plan.mode) && Options::optimizeNativeCalls() && callLinkStatus.size() == 1 && !callLinkStatus.couldTakeSlowPath()) {
-        CallVariant callee = callLinkStatus[0].callee();
+        CallVariant callee = callLinkStatus[0];
         JSFunction* function = callee.function();
         CodeSpecializationKind specializationKind = InlineCallFrame::specializationKindFor(kind);
@@ -1084 +1075 @@
             callOpInfo = OpInfo(m_graph.freeze(function));
 
-            if (op == Call || op == ProfiledCall)
+            if (op == Call)
                 op = NativeCall;
             else {
-                ASSERT(op == Construct || op == ProfiledConstruct);
+                ASSERT(op == Construct);
                 op = NativeConstruct;
             }
@@ -1427 +1418 @@
     if (!callLinkStatus.couldTakeSlowPath() && callLinkStatus.size() == 1) {
         emitFunctionChecks(
-            callLinkStatus[0].callee(), callTargetNode, registerOffset, specializationKind);
+            callLinkStatus[0], callTargetNode, registerOffset, specializationKind);
         bool result = attemptToInlineCall(
-            callTargetNode, resultOperand, callLinkStatus[0].callee(), registerOffset,
+            callTargetNode, resultOperand, callLinkStatus[0], registerOffset,
             argumentCountIncludingThis, nextOffset, kind, CallerDoesNormalLinking, prediction,
             inliningBalance);
         if (!result && !callLinkStatus.isProved())
-            undoFunctionChecks(callLinkStatus[0].callee());
+            undoFunctionChecks(callLinkStatus[0]);
         if (verbose) {
             dataLog("Done inlining (simple).\n");
@@ -1463 +1454 @@
     bool allAreDirectCalls = true;
     for (unsigned i = callLinkStatus.size(); i--;) {
-        if (callLinkStatus[i].callee().isClosureCall())
+        if (callLinkStatus[i].isClosureCall())
             allAreDirectCalls = false;
         else
@@ -1476 +1467 @@
     else {
         // FIXME: We should be able to handle this case, but it's tricky and we don't know of cases
-        // where it would be beneficial. Also, CallLinkStatus would make all callees appear like
-        // closure calls if any calls were closure calls - except for calls to internal functions.
-        // So this will only arise if some callees are internal functions and others are closures.
+        // where it would be beneficial. It might be best to handle these cases as if all calls were
+        // closure calls.
         // https://bugs.webkit.org/show_bug.cgi?id=136020
         if (verbose) {
@@ -1518 +1508 @@
     Vector<BasicBlock*> landingBlocks;
     
-    // We make force this true if we give up on inlining any of the edges.
+    // We may force this true if we give up on inlining any of the edges.
     bool couldTakeSlowPath = callLinkStatus.couldTakeSlowPath();
     
@@ -1534 +1524 @@
         
         bool inliningResult = attemptToInlineCall(
-            myCallTargetNode, resultOperand, callLinkStatus[i].callee(), registerOffset,
+            myCallTargetNode, resultOperand, callLinkStatus[i], registerOffset,
             argumentCountIncludingThis, nextOffset, kind, CallerLinksManually, prediction,
             inliningBalance);
@@ -1553 +1543 @@
         JSCell* thingToCaseOn;
         if (allAreDirectCalls)
-            thingToCaseOn = callLinkStatus[i].callee().nonExecutableCallee();
+            thingToCaseOn = callLinkStatus[i].nonExecutableCallee();
         else {
             ASSERT(allAreClosureCalls);
-            thingToCaseOn = callLinkStatus[i].callee().executable();
+            thingToCaseOn = callLinkStatus[i].executable();
         }
         data.cases.append(SwitchCase(m_graph.freeze(thingToCaseOn), block.get()));
@@ -1568 +1558 @@
 
         if (verbose)
-            dataLog("Finished inlining ", callLinkStatus[i].callee(), " at ", currentCodeOrigin(), ".\n");
+            dataLog("Finished inlining ", callLinkStatus[i], " at ", currentCodeOrigin(), ".\n");
     }