Context Navigation

← Previous Change
Next Change →

DFGOSRExitCompiler64.cpp

Timestamp:

Sep 15, 2013, 11:53:23 AM (12 years ago)

Author:

Message:

Deoptimize deoptimization: make DFGOSRExitCompiler64.cpp more hackable
https://bugs.webkit.org/show_bug.cgi?id=121374

Reviewed by Geoffrey Garen.

This reduces the size of DFGOSRExitCompiler64.cpp by almost 50%, and makes it
super easy to add new recovery kinds. For recoveries that involve reboxing, it
allows you to keep most of the code common between the on-stack and in-reg
cases: they all get funneled through the "load from scratch buffer, convert,
and then store to stack" logic.

This opens up a bunch of possibilities. It'll make adding Int48 much easier,
and it probably will come in handy as we do various DFG stack layout changes in
support of the FTL.

bytecode/ValueRecovery.h:

(JSC::ValueRecovery::dumpInContext):
(JSC::ValueRecovery::dump):

dfg/DFGOSRExitCompiler.cpp:

(JSC::DFG::shortOperandsDump):

dfg/DFGOSRExitCompiler64.cpp:

(JSC::DFG::OSRExitCompiler::compileExit):

File:

: 1 edited

trunk/Source/JavaScriptCore/dfg/DFGOSRExitCompiler64.cpp (modified) (5 diffs)

Legend:

: Unmodified
: Added
: Removed

trunk/Source/JavaScriptCore/dfg/DFGOSRExitCompiler64.cpp

-              r155711
+              r155820
     //    starts mutating state before verifying the speculation it has already made.
-    GPRReg alreadyBoxed = InvalidGPRReg;
     if (recovery) {
         switch (recovery->type()) {
 …
             m_jit.sub32(recovery->src(), recovery->dest());
             m_jit.or64(GPRInfo::tagTypeNumberRegister, recovery->dest());
-            alreadyBoxed = recovery->dest();
             break;
 …
+        }
+    }
+    // 4) Figure out how many scratch slots we'll need. We need one for every GPR/FPR
+    //    whose destination is now occupied by a DFG virtual register, and we need
+    //    one for every displaced virtual register if there are more than
+    //    GPRInfo::numberOfRegisters of them. Also see if there are any constants,
+    //    any undefined slots, any FPR slots, and any unboxed ints.
+    Vector<bool> poisonedVirtualRegisters(operands.numberOfLocals());
+    for (unsigned i = 0; i < poisonedVirtualRegisters.size(); ++i)
+        poisonedVirtualRegisters[i] = false;
+    unsigned numberOfPoisonedVirtualRegisters = 0;
+    unsigned numberOfDisplacedVirtualRegisters = 0;
+    // Booleans for fast checks. We expect that most OSR exits do not have to rebox
+    // Int32s, have no FPRs, and have no constants. If there are constants, we
+    // expect most of them to be jsUndefined(); if that's true then we handle that
+    // specially to minimize code size and execution time.
+    bool haveUnboxedInt32s = false;
+    bool haveUnboxedDoubles = false;
+    bool haveFPRs = false;
+    bool haveConstants = false;
+    bool haveUndefined = false;
+    bool haveUInt32s = false;
+    bool haveArguments = false;
+    // What follows is an intentionally simple OSR exit implementation that generates
+    // fairly poor code but is very easy to hack. In particular, it dumps all state that
+    // needs conversion into a scratch buffer so that in step 6, where we actually do the
+    // conversions, we know that all temp registers are free to use and the variable is
+    // definitely in a well-known spot in the scratch buffer regardless of whether it had
+    // originally been in a register or spilled. This allows us to decouple "where was
+    // the variable" from "how was it represented". Consider that the
+    // AlreadyInJSStackAsUnboxedInt32 recovery: it tells us that the value is in a
+    // particular place and that that place holds an unboxed int32. We have three different
+    // places that a value could be (stack, displaced, register) and a bunch of different
+    // ways of representing a value. The number of recoveries is three * a bunch. The code
+    // below means that we have to have three + a bunch cases rather than three * a bunch.
+    // Once we have loaded the value from wherever it was, the reboxing is the same
+    // regardless of its location. Likewise, before we do the reboxing, the way we get to
+    // the value (i.e. where we load it from) is the same regardless of its type. Because
+    // the code below always dumps everything into a scratch buffer first, the two
+    // questions become orthogonal, which simplifies adding new types and adding new
+    // locations.
+    //
+    // This raises the question: does using such a suboptimal implementation of OSR exit,
+    // where we always emit code to dump all state into a scratch buffer only to then
+    // dump it right back into the stack, hurt us in any way? The asnwer is that OSR exits
+    // are rare. Our tiering strategy ensures this. This is because if an OSR exit is
+    // taken more than ~100 times, we jettison the DFG code block along with all of its
+    // exits. It is impossible for an OSR exit - i.e. the code we compile below - to
+    // execute frequently enough for the codegen to matter that much. It probably matters
+    // enough that we don't want to turn this into some super-slow function call, but so
+    // long as we're generating straight-line code, that code can be pretty bad. Also
+    // because we tend to exit only along one OSR exit from any DFG code block - that's an
+    // empirical result that we're extremely confident about - the code size of this
+    // doesn't matter much. Hence any attempt to optimize the codegen here is just purely
+    // harmful to the system: it probably won't reduce either net memory usage or net
+    // execution time. It will only prevent us from cleanly decoupling "where was the
+    // variable" from "how was it represented", which will make it more difficult to add
+    // features in the future and it will make it harder to reason about bugs.
+    // 4) Save all state from GPRs into the scratch buffer.
+    ScratchBuffer* scratchBuffer = m_jit.vm()->scratchBufferForSize(sizeof(EncodedJSValue) * operands.size());
+    EncodedJSValue* scratch = scratchBuffer ? static_cast<EncodedJSValue*>(scratchBuffer->dataBuffer()) : 0;
     for (size_t index = 0; index < operands.size(); ++index) {
         const ValueRecovery& recovery = operands[index];
+        switch (recovery.technique()) {
+        case Int32DisplacedInJSStack:
+        case DoubleDisplacedInJSStack:
+        case DisplacedInJSStack: {
+            numberOfDisplacedVirtualRegisters++;
+            ASSERT(operandIsLocal(recovery.virtualRegister()));
+            // See if we might like to store to this virtual register before doing
+            // virtual register shuffling. If so, we say that the virtual register
+            // is poisoned: it cannot be stored to until after displaced virtual
+            // registers are handled. We track poisoned virtual register carefully
+            // to ensure this happens efficiently. Note that we expect this case
+            // to be rare, so the handling of it is optimized for the cases in
+            // which it does not happen.
+            int local = operandToLocal(recovery.virtualRegister());
+            if (local < (int)operands.numberOfLocals()) {
+                switch (operands.local(local).technique()) {
+                case InGPR:
+                case UnboxedInt32InGPR:
+                case UInt32InGPR:
+                case InFPR:
+                    if (!poisonedVirtualRegisters[local]) {
+                        poisonedVirtualRegisters[local] = true;
+                        numberOfPoisonedVirtualRegisters++;
+                    }
+                    break;
+                default:
+                    break;
+                }
+            }
+            break;
+            }
+        case UnboxedInt32InGPR:
+        case AlreadyInJSStackAsUnboxedInt32:
+            haveUnboxedInt32s = true;
+            break;
+        case AlreadyInJSStackAsUnboxedDouble:
+            haveUnboxedDoubles = true;
+            break;
+        case UInt32InGPR:
+            haveUInt32s = true;
+            break;
+        case InFPR:
+            haveFPRs = true;
+            break;
+        case Constant:
+            haveConstants = true;
+            if (recovery.constant().isUndefined())
+                haveUndefined = true;
+            break;
+        case ArgumentsThatWereNotCreated:
+            haveArguments = true;
+            break;
+        default:
+            break;
+        }
+    }
+#if DFG_ENABLE(DEBUG_VERBOSE)
+    dataLogF("  ");
+    if (numberOfPoisonedVirtualRegisters)
+        dataLogF("Poisoned=%u ", numberOfPoisonedVirtualRegisters);
+    if (numberOfDisplacedVirtualRegisters)
+        dataLogF("Displaced=%u ", numberOfDisplacedVirtualRegisters);
+    if (haveUnboxedInt32s)
+        dataLogF("UnboxedInt32 ");
+    if (haveUnboxedDoubles)
+        dataLogF("UnboxedDoubles ");
+    if (haveUInt32s)
+        dataLogF("UInt32 ");
+    if (haveFPRs)
+        dataLogF("FPR ");
+    if (haveConstants)
+        dataLogF("Constants ");
+    if (haveUndefined)
+        dataLogF("Undefined ");
+    dataLogF(" ");
+#endif
+    ScratchBuffer* scratchBuffer = m_jit.vm()->scratchBufferForSize(sizeof(EncodedJSValue) * std::max(haveUInt32s ? 2u : 0u, numberOfPoisonedVirtualRegisters + (numberOfDisplacedVirtualRegisters <= GPRInfo::numberOfRegisters ? 0 : numberOfDisplacedVirtualRegisters)));
+    EncodedJSValue* scratchDataBuffer = scratchBuffer ? static_cast<EncodedJSValue*>(scratchBuffer->dataBuffer()) : 0;
+    // From here on, the code assumes that it is profitable to maximize the distance
+    // between when something is computed and when it is stored.
+    // 5) Perform all reboxing of integers.
+    if (haveUnboxedInt32s || haveUInt32s) {
+        for (size_t index = 0; index < operands.size(); ++index) {
+            const ValueRecovery& recovery = operands[index];
+            switch (recovery.technique()) {
+            case UnboxedInt32InGPR:
+                if (recovery.gpr() != alreadyBoxed)
+                    m_jit.or64(GPRInfo::tagTypeNumberRegister, recovery.gpr());
+                break;
+            case AlreadyInJSStackAsUnboxedInt32:
+                m_jit.store32(AssemblyHelpers::TrustedImm32(static_cast<uint32_t>(TagTypeNumber >> 32)), AssemblyHelpers::tagFor(static_cast<VirtualRegister>(operands.operandForIndex(index))));
+                break;
+            case UInt32InGPR: {
+                // This occurs when the speculative JIT left an unsigned 32-bit integer
+                // in a GPR. If it's positive, we can just box the int. Otherwise we
+                // need to turn it into a boxed double.
+                // We don't try to be clever with register allocation here; we assume
+                // that the program is using FPRs and we don't try to figure out which
+                // ones it is using. Instead just temporarily save fpRegT0 and then
+                // restore it. This makes sense because this path is not cheap to begin
+                // with, and should happen very rarely.
+                GPRReg addressGPR = GPRInfo::regT0;
+                if (addressGPR == recovery.gpr())
+                    addressGPR = GPRInfo::regT1;
+                m_jit.store64(addressGPR, scratchDataBuffer);
+                m_jit.move(AssemblyHelpers::TrustedImmPtr(scratchDataBuffer + 1), addressGPR);
+                m_jit.storeDouble(FPRInfo::fpRegT0, addressGPR);
+                AssemblyHelpers::Jump positive = m_jit.branch32(AssemblyHelpers::GreaterThanOrEqual, recovery.gpr(), AssemblyHelpers::TrustedImm32(0));
+                m_jit.convertInt32ToDouble(recovery.gpr(), FPRInfo::fpRegT0);
+                m_jit.addDouble(AssemblyHelpers::AbsoluteAddress(&AssemblyHelpers::twoToThe32), FPRInfo::fpRegT0);
+                m_jit.boxDouble(FPRInfo::fpRegT0, recovery.gpr());
+                AssemblyHelpers::Jump done = m_jit.jump();
+                positive.link(&m_jit);
+                m_jit.or64(GPRInfo::tagTypeNumberRegister, recovery.gpr());
+                done.link(&m_jit);
+                m_jit.loadDouble(addressGPR, FPRInfo::fpRegT0);
+                m_jit.load64(scratchDataBuffer, addressGPR);
+                break;
+            }
+            default:
+                break;
+            }
+        }
+    }
+    // 6) Dump all non-poisoned GPRs. For poisoned GPRs, save them into the scratch storage.
+    //    Note that GPRs do not have a fast change (like haveFPRs) because we expect that
+    //    most OSR failure points will have at least one GPR that needs to be dumped.
+    initializePoisoned(operands.numberOfLocals());
+    unsigned currentPoisonIndex = 0;
+    for (size_t index = 0; index < operands.size(); ++index) {
+        const ValueRecovery& recovery = operands[index];
+        int operand = operands.operandForIndex(index);
         switch (recovery.technique()) {
         case InGPR:
         case UnboxedInt32InGPR:
         case UInt32InGPR:
+            if (operands.isVariable(index) && poisonedVirtualRegisters[operands.variableForIndex(index)]) {
+                m_jit.store64(recovery.gpr(), scratchDataBuffer + currentPoisonIndex);
+                m_poisonScratchIndices[operands.variableForIndex(index)] = currentPoisonIndex;
+                currentPoisonIndex++;
+            } else
+                m_jit.store64(recovery.gpr(), AssemblyHelpers::addressFor((VirtualRegister)operand));
+            break;
+            m_jit.store64(recovery.gpr(), scratch + index);
+            break;
         default:
             break;
 …
+    }
+    // At this point all GPRs are available for scratch use.
+    if (haveFPRs) {
+        // 7) Box all doubles (relies on there being more GPRs than FPRs)
+        for (size_t index = 0; index < operands.size(); ++index) {
+            const ValueRecovery& recovery = operands[index];
+            if (recovery.technique() != InFPR)
+                continue;
+            FPRReg fpr = recovery.fpr();
+            GPRReg gpr = GPRInfo::toRegister(FPRInfo::toIndex(fpr));
+            m_jit.boxDouble(fpr, gpr);
+        }
+        // 8) Dump all doubles into the stack, or to the scratch storage if
+        //    the destination virtual register is poisoned.
+        for (size_t index = 0; index < operands.size(); ++index) {
+            const ValueRecovery& recovery = operands[index];
+            if (recovery.technique() != InFPR)
+                continue;
+            GPRReg gpr = GPRInfo::toRegister(FPRInfo::toIndex(recovery.fpr()));
+            if (operands.isVariable(index) && poisonedVirtualRegisters[operands.variableForIndex(index)]) {
+                m_jit.store64(gpr, scratchDataBuffer + currentPoisonIndex);
+                m_poisonScratchIndices[operands.variableForIndex(index)] = currentPoisonIndex;
+                currentPoisonIndex++;
+            } else
+                m_jit.store64(gpr, AssemblyHelpers::addressFor((VirtualRegister)operands.operandForIndex(index)));
+        }
+    }
+    // At this point all GPRs and FPRs are available for scratch use.
+    // 9) Box all unboxed doubles in the stack.
+    if (haveUnboxedDoubles) {
+        for (size_t index = 0; index < operands.size(); ++index) {
+            const ValueRecovery& recovery = operands[index];
+            if (recovery.technique() != AlreadyInJSStackAsUnboxedDouble)
+                continue;
+            m_jit.loadDouble(AssemblyHelpers::addressFor((VirtualRegister)operands.operandForIndex(index)), FPRInfo::fpRegT0);
+    // And voila, all FPRs are free to reuse.
+    // 5) Save all state from FPRs into the scratch buffer.
+    for (size_t index = 0; index < operands.size(); ++index) {
+        const ValueRecovery& recovery = operands[index];
+        switch (recovery.technique()) {
+        case InFPR:
+            m_jit.move(AssemblyHelpers::TrustedImmPtr(scratch + index), GPRInfo::regT0);
+            m_jit.storeDouble(recovery.fpr(), GPRInfo::regT0);
+            break;
+        default:
+            break;
+        }
+    }
+    // Now, all FPRs are also free.
+    // 5) Save all state from the stack into the scratch buffer. For simplicity we
+    //    do this even for state that's already in the right place on the stack.
+    //    It makes things simpler later.
+    for (size_t index = 0; index < operands.size(); ++index) {
+        const ValueRecovery& recovery = operands[index];
+        int operand = operands.operandForIndex(index);
+        switch (recovery.technique()) {
+        case DisplacedInJSStack:
+        case Int32DisplacedInJSStack:
+        case DoubleDisplacedInJSStack:
+            m_jit.load64(AssemblyHelpers::addressFor(recovery.virtualRegister()), GPRInfo::regT0);
+            m_jit.store64(GPRInfo::regT0, scratch + index);
+            break;
+        case AlreadyInJSStackAsUnboxedInt32:
+        case AlreadyInJSStackAsUnboxedDouble:
+            m_jit.load64(AssemblyHelpers::addressFor(operand), GPRInfo::regT0);
+            m_jit.store64(GPRInfo::regT0, scratch + index);
+            break;
+        default:
+            break;
+        }
+    }
+    // 6) Do all data format conversions and store the results into the stack.
+    bool haveArguments = false;
+    for (size_t index = 0; index < operands.size(); ++index) {
+        const ValueRecovery& recovery = operands[index];
+        int operand = operands.operandForIndex(index);
+        switch (recovery.technique()) {
+        case InGPR:
+        case DisplacedInJSStack:
+            m_jit.load64(scratch + index, GPRInfo::regT0);
+            m_jit.store64(GPRInfo::regT0, AssemblyHelpers::addressFor(operand));
+            break;
+        case AlreadyInJSStackAsUnboxedInt32:
+        case UnboxedInt32InGPR:
+        case Int32DisplacedInJSStack:
+            m_jit.load64(scratch + index, GPRInfo::regT0);
+            m_jit.zeroExtend32ToPtr(GPRInfo::regT0, GPRInfo::regT0);
+            m_jit.or64(GPRInfo::tagTypeNumberRegister, GPRInfo::regT0);
+            m_jit.store64(GPRInfo::regT0, AssemblyHelpers::addressFor(operand));
+            break;
+        case AlreadyInJSStackAsUnboxedDouble:
+        case InFPR:
+        case DoubleDisplacedInJSStack:
+            m_jit.move(AssemblyHelpers::TrustedImmPtr(scratch + index), GPRInfo::regT0);
+            m_jit.loadDouble(GPRInfo::regT0, FPRInfo::fpRegT0);
             m_jit.boxDouble(FPRInfo::fpRegT0, GPRInfo::regT0);
+            m_jit.store64(GPRInfo::regT0, AssemblyHelpers::addressFor((VirtualRegister)operands.operandForIndex(index)));
+        }
+    }
+    ASSERT(currentPoisonIndex == numberOfPoisonedVirtualRegisters);
+    // 10) Reshuffle displaced virtual registers. Optimize for the case that
+    //    the number of displaced virtual registers is not more than the number
+    //    of available physical registers.
+    if (numberOfDisplacedVirtualRegisters) {
+        if (numberOfDisplacedVirtualRegisters <= GPRInfo::numberOfRegisters) {
+            // So far this appears to be the case that triggers all the time, but
+            // that is far from guaranteed.
+            unsigned displacementIndex = 0;
+            for (size_t index = 0; index < operands.size(); ++index) {
+                const ValueRecovery& recovery = operands[index];
+                switch (recovery.technique()) {
+                case DisplacedInJSStack:
+                    m_jit.load64(AssemblyHelpers::addressFor(recovery.virtualRegister()), GPRInfo::toRegister(displacementIndex++));
+                    break;
+                case Int32DisplacedInJSStack: {
+                    GPRReg gpr = GPRInfo::toRegister(displacementIndex++);
+                    m_jit.load32(AssemblyHelpers::addressFor(recovery.virtualRegister()), gpr);
+                    m_jit.or64(GPRInfo::tagTypeNumberRegister, gpr);
+                    break;
+                }
+                case DoubleDisplacedInJSStack: {
+                    GPRReg gpr = GPRInfo::toRegister(displacementIndex++);
+                    m_jit.load64(AssemblyHelpers::addressFor(recovery.virtualRegister()), gpr);
+                    m_jit.sub64(GPRInfo::tagTypeNumberRegister, gpr);
+                    break;
+                }
+                default:
+                    break;
+                }
+            }
+            displacementIndex = 0;
+            for (size_t index = 0; index < operands.size(); ++index) {
+                const ValueRecovery& recovery = operands[index];
+                switch (recovery.technique()) {
+                case DisplacedInJSStack:
+                case Int32DisplacedInJSStack:
+                case DoubleDisplacedInJSStack:
+                    m_jit.store64(GPRInfo::toRegister(displacementIndex++), AssemblyHelpers::addressFor((VirtualRegister)operands.operandForIndex(index)));
+                    break;
+                default:
+                    break;
+                }
+            }
+        } else {
+            // FIXME: This should use the shuffling algorithm that we use
+            // for speculative->non-speculative jumps, if we ever discover that
+            // some hot code with lots of live values that get displaced and
+            // spilled really enjoys frequently failing speculation.
+            // For now this code is engineered to be correct but probably not
+            // super. In particular, it correctly handles cases where for example
+            // the displacements are a permutation of the destination values, like
+            //
+            // 1 -> 2
+            // 2 -> 1
+            //
+            // It accomplishes this by simply lifting all of the virtual registers
+            // from their old (DFG JIT) locations and dropping them in a scratch
+            // location in memory, and then transferring from that scratch location
+            // to their new (old JIT) locations.
+            unsigned scratchIndex = numberOfPoisonedVirtualRegisters;
+            for (size_t index = 0; index < operands.size(); ++index) {
+                const ValueRecovery& recovery = operands[index];
+                switch (recovery.technique()) {
+                case DisplacedInJSStack:
+                    m_jit.load64(AssemblyHelpers::addressFor(recovery.virtualRegister()), GPRInfo::regT0);
+                    m_jit.store64(GPRInfo::regT0, scratchDataBuffer + scratchIndex++);
+                    break;
+                case Int32DisplacedInJSStack: {
+                    m_jit.load32(AssemblyHelpers::addressFor(recovery.virtualRegister()), GPRInfo::regT0);
+                    m_jit.or64(GPRInfo::tagTypeNumberRegister, GPRInfo::regT0);
+                    m_jit.store64(GPRInfo::regT0, scratchDataBuffer + scratchIndex++);
+                    break;
+                }
+                case DoubleDisplacedInJSStack: {
+                    m_jit.load64(AssemblyHelpers::addressFor(recovery.virtualRegister()), GPRInfo::regT0);
+                    m_jit.sub64(GPRInfo::tagTypeNumberRegister, GPRInfo::regT0);
+                    m_jit.store64(GPRInfo::regT0, scratchDataBuffer + scratchIndex++);
+                    break;
+                }
+                default:
+                    break;
+                }
+            }
+            scratchIndex = numberOfPoisonedVirtualRegisters;
+            for (size_t index = 0; index < operands.size(); ++index) {
+                const ValueRecovery& recovery = operands[index];
+                switch (recovery.technique()) {
+                case DisplacedInJSStack:
+                case Int32DisplacedInJSStack:
+                case DoubleDisplacedInJSStack:
+                    m_jit.load64(scratchDataBuffer + scratchIndex++, GPRInfo::regT0);
+                    m_jit.store64(GPRInfo::regT0, AssemblyHelpers::addressFor((VirtualRegister)operands.operandForIndex(index)));
+                    break;
+                default:
+                    break;
+                }
+            }
+            ASSERT(scratchIndex == numberOfPoisonedVirtualRegisters + numberOfDisplacedVirtualRegisters);
+        }
+    }
+    // 11) Dump all poisoned virtual registers.
+    if (numberOfPoisonedVirtualRegisters) {
+        for (int virtualRegister = 0; virtualRegister < (int)operands.numberOfLocals(); ++virtualRegister) {
+            if (!poisonedVirtualRegisters[virtualRegister])
+                continue;
+            const ValueRecovery& recovery = operands.local(virtualRegister);
+            switch (recovery.technique()) {
+            case InGPR:
+            case UnboxedInt32InGPR:
+            case UInt32InGPR:
+            case InFPR:
+                m_jit.load64(scratchDataBuffer + poisonIndex(virtualRegister), GPRInfo::regT0);
+                m_jit.store64(GPRInfo::regT0, AssemblyHelpers::addressFor((VirtualRegister)localToOperand(virtualRegister)));
+                break;
+            default:
+                break;
+            }
+        }
+    }
+    // 12) Dump all constants. Optimize for Undefined, since that's a constant we see
+    //     often.
+    if (haveConstants) {
+        if (haveUndefined)
+            m_jit.move(AssemblyHelpers::TrustedImm64(JSValue::encode(jsUndefined())), GPRInfo::regT0);
+        for (size_t index = 0; index < operands.size(); ++index) {
+            const ValueRecovery& recovery = operands[index];
+            if (recovery.technique() != Constant)
+                continue;
+            if (recovery.constant().isUndefined())
+                m_jit.store64(GPRInfo::regT0, AssemblyHelpers::addressFor((VirtualRegister)operands.operandForIndex(index)));
+            else
+                m_jit.store64(AssemblyHelpers::TrustedImm64(JSValue::encode(recovery.constant())), AssemblyHelpers::addressFor((VirtualRegister)operands.operandForIndex(index)));
+        }
+    }
+    // 13) Adjust the old JIT's execute counter. Since we are exiting OSR, we know
+    //     that all new calls into this code will go to the new JIT, so the execute
+    //     counter only affects call frames that performed OSR exit and call frames
+    //     that were still executing the old JIT at the time of another call frame's
+    //     OSR exit. We want to ensure that the following is true:
+    //
+    //     (a) Code the performs an OSR exit gets a chance to reenter optimized
+    //         code eventually, since optimized code is faster. But we don't
+    //         want to do such reentery too aggressively (see (c) below).
+    //
+    //     (b) If there is code on the call stack that is still running the old
+    //         JIT's code and has never OSR'd, then it should get a chance to
+    //         perform OSR entry despite the fact that we've exited.
+    //
+    //     (c) Code the performs an OSR exit should not immediately retry OSR
+    //         entry, since both forms of OSR are expensive. OSR entry is
+    //         particularly expensive.
+    //
+    //     (d) Frequent OSR failures, even those that do not result in the code
+    //         running in a hot loop, result in recompilation getting triggered.
+    //
+    //     To ensure (c), we'd like to set the execute counter to
+    //     counterValueForOptimizeAfterWarmUp(). This seems like it would endanger
+    //     (a) and (b), since then every OSR exit would delay the opportunity for
+    //     every call frame to perform OSR entry. Essentially, if OSR exit happens
+    //     frequently and the function has few loops, then the counter will never
+    //     become non-negative and OSR entry will never be triggered. OSR entry
+    //     will only happen if a loop gets hot in the old JIT, which does a pretty
+    //     good job of ensuring (a) and (b). But that doesn't take care of (d),
+    //     since each speculation failure would reset the execute counter.
+    //     So we check here if the number of speculation failures is significantly
+    //     larger than the number of successes (we want 90% success rate), and if
+    //     there have been a large enough number of failures. If so, we set the
+    //     counter to 0; otherwise we set the counter to
+    //     counterValueForOptimizeAfterWarmUp().
+            m_jit.store64(GPRInfo::regT0, AssemblyHelpers::addressFor(operand));
+            break;
+        case UInt32InGPR: {
+            m_jit.load64(scratch + index, GPRInfo::regT0);
+            m_jit.zeroExtend32ToPtr(GPRInfo::regT0, GPRInfo::regT0);
+            AssemblyHelpers::Jump positive = m_jit.branch32(
+                AssemblyHelpers::GreaterThanOrEqual,
+                GPRInfo::regT0, AssemblyHelpers::TrustedImm32(0));
+            m_jit.convertInt32ToDouble(GPRInfo::regT0, FPRInfo::fpRegT0);
+            m_jit.addDouble(
+                AssemblyHelpers::AbsoluteAddress(&AssemblyHelpers::twoToThe32),
+                FPRInfo::fpRegT0);
+            m_jit.boxDouble(FPRInfo::fpRegT0, GPRInfo::regT0);
+            AssemblyHelpers::Jump done = m_jit.jump();
+            positive.link(&m_jit);
+            m_jit.or64(GPRInfo::tagTypeNumberRegister, GPRInfo::regT0);
+            done.link(&m_jit);
+            m_jit.store64(GPRInfo::regT0, AssemblyHelpers::addressFor(operand));
+            break;
+        }
+        case Constant:
+            m_jit.store64(
+                AssemblyHelpers::TrustedImm64(JSValue::encode(recovery.constant())),
+                AssemblyHelpers::addressFor(operand));
+            break;
+        case ArgumentsThatWereNotCreated:
+            haveArguments = true;
+            break;
+        default:
+            break;
+        }
+    }
+    // 7) Adjust the old JIT's execute counter. Since we are exiting OSR, we know
+    //    that all new calls into this code will go to the new JIT, so the execute
+    //    counter only affects call frames that performed OSR exit and call frames
+    //    that were still executing the old JIT at the time of another call frame's
+    //    OSR exit. We want to ensure that the following is true:
+    //
+    //    (a) Code the performs an OSR exit gets a chance to reenter optimized
+    //        code eventually, since optimized code is faster. But we don't
+    //        want to do such reentery too aggressively (see (c) below).
+    //
+    //    (b) If there is code on the call stack that is still running the old
+    //        JIT's code and has never OSR'd, then it should get a chance to
+    //        perform OSR entry despite the fact that we've exited.
+    //
+    //    (c) Code the performs an OSR exit should not immediately retry OSR
+    //        entry, since both forms of OSR are expensive. OSR entry is
+    //        particularly expensive.
+    //
+    //    (d) Frequent OSR failures, even those that do not result in the code
+    //        running in a hot loop, result in recompilation getting triggered.
+    //
+    //    To ensure (c), we'd like to set the execute counter to
+    //    counterValueForOptimizeAfterWarmUp(). This seems like it would endanger
+    //    (a) and (b), since then every OSR exit would delay the opportunity for
+    //    every call frame to perform OSR entry. Essentially, if OSR exit happens
+    //    frequently and the function has few loops, then the counter will never
+    //    become non-negative and OSR entry will never be triggered. OSR entry
+    //    will only happen if a loop gets hot in the old JIT, which does a pretty
+    //    good job of ensuring (a) and (b). But that doesn't take care of (d),
+    //    since each speculation failure would reset the execute counter.
+    //    So we check here if the number of speculation failures is significantly
+    //    larger than the number of successes (we want 90% success rate), and if
+    //    there have been a large enough number of failures. If so, we set the
+    //    counter to 0; otherwise we set the counter to
+    //    counterValueForOptimizeAfterWarmUp().
     handleExitCounts(m_jit, exit);
     // 14) Reify inlined call frames.
+    // 8) Reify inlined call frames.
     reifyInlinedCallFrames(m_jit, exit);
     // 15) Create arguments if necessary and place them into the appropriate aliased
     //     registers.
+    // 9) Create arguments if necessary and place them into the appropriate aliased
+    //    registers.
     if (haveArguments) {
 …
+    }
     // 16) Load the result of the last bytecode operation into regT0.
+    // 10) Load the result of the last bytecode operation into regT0.
     if (exit.m_lastSetOperand != std::numeric_limits<int>::max())
         m_jit.load64(AssemblyHelpers::addressFor(exit.m_lastSetOperand), GPRInfo::cachedResultRegister);
     // 17) And finish.
+    // 11) And finish.
     adjustAndJumpToTarget(m_jit, exit);

Note: See TracChangeset for help on using the changeset viewer.

Context Navigation

Changeset 155820 in webkit for trunk/Source/JavaScriptCore/dfg/DFGOSRExitCompiler64.cpp

Legend:

trunk/Source/JavaScriptCore/dfg/DFGOSRExitCompiler64.cpp

Download in other formats: