Ignore:
Timestamp:
Sep 17, 2013, 6:31:04 PM (12 years ago)
Author:
[email protected]
Message:

DFG should support Int52 for local variables
​https://bugs.webkit.org/show_bug.cgi?id=121064

Source/JavaScriptCore:

Reviewed by Oliver Hunt.

This adds Int52 support for local variables to the DFG and FTL. It's a speed-up on
programs that have local int32 overflows but where a larger int representation can
prevent us from having to convert all the way up to double.

It's a small speed-up for now. But we're just supporting Int52 for a handful of
operations (add, sub, mul, neg, compare, bitops, typed array access) and this lays
the groundwork for adding Int52 to JSValue, which will probably be a bigger
speed-up.

The basic approach is:

  • We have a notion of Int52 in our typesystem. Int52 doesn't belong to BytecodeTop or HeapTop - i.e. it doesn't arise from JSValues.


  • DFG treats Int52 as being part of its FullTop and will treat it as being a subtype of double unless instructed otherwise.


  • Prediction propagator creates Int52s whenever we have a node going doubly but due to large values rather than fractional values, and that node is known to be able to produce Int52 natively in the DFG backend.


  • Fixup phase converts edges to MachineIntUses in nodes that are known to be able to deal with Int52, and where we have a subtype of Int32|Int52 as the predicted input.


  • The DFG backend and FTL LLVM IR lowering have two notions of Int52s - ones that are left-shifted by 16 (great for overflow checks) and ones that are sign-extended. Both backends know how to convert between Int52s and the other representations.
  • assembler/MacroAssemblerX86_64.h:

(JSC::MacroAssemblerX86_64::rshift64):
(JSC::MacroAssemblerX86_64::mul64):
(JSC::MacroAssemblerX86_64::branchMul64):
(JSC::MacroAssemblerX86_64::branchNeg64):
(JSC::MacroAssemblerX86_64::convertInt64ToDouble):

  • assembler/X86Assembler.h:

(JSC::X86Assembler::imulq_rr):
(JSC::X86Assembler::cvtsi2sdq_rr):

  • bytecode/DataFormat.h:

(JSC::dataFormatToString):

  • bytecode/OperandsInlines.h:

(JSC::::dumpInContext):

  • bytecode/SpeculatedType.cpp:

(JSC::dumpSpeculation):
(JSC::speculationToAbbreviatedString):
(JSC::speculationFromValue):

  • bytecode/SpeculatedType.h:

(JSC::isInt32SpeculationForArithmetic):
(JSC::isMachineIntSpeculationForArithmetic):
(JSC::isBytecodeRealNumberSpeculation):
(JSC::isFullRealNumberSpeculation):
(JSC::isBytecodeNumberSpeculation):
(JSC::isFullNumberSpeculation):
(JSC::isBytecodeNumberSpeculationExpectingDefined):
(JSC::isFullNumberSpeculationExpectingDefined):

  • bytecode/ValueRecovery.h:

(JSC::ValueRecovery::alreadyInJSStackAsUnboxedInt52):
(JSC::ValueRecovery::inGPR):
(JSC::ValueRecovery::displacedInJSStack):
(JSC::ValueRecovery::isAlreadyInJSStack):
(JSC::ValueRecovery::gpr):
(JSC::ValueRecovery::virtualRegister):
(JSC::ValueRecovery::dumpInContext):

  • dfg/DFGAbstractInterpreter.h:

(JSC::DFG::AbstractInterpreter::needsTypeCheck):
(JSC::DFG::AbstractInterpreter::filterByType):

  • dfg/DFGAbstractInterpreterInlines.h:

(JSC::DFG::::executeEffects):

  • dfg/DFGAbstractValue.cpp:

(JSC::DFG::AbstractValue::set):
(JSC::DFG::AbstractValue::checkConsistency):

  • dfg/DFGAbstractValue.h:

(JSC::DFG::AbstractValue::couldBeType):
(JSC::DFG::AbstractValue::isType):
(JSC::DFG::AbstractValue::checkConsistency):
(JSC::DFG::AbstractValue::validateType):

  • dfg/DFGArrayMode.cpp:

(JSC::DFG::ArrayMode::refine):

  • dfg/DFGAssemblyHelpers.h:

(JSC::DFG::AssemblyHelpers::boxInt52):

  • dfg/DFGCSEPhase.cpp:

(JSC::DFG::CSEPhase::pureCSE):
(JSC::DFG::CSEPhase::getByValLoadElimination):
(JSC::DFG::CSEPhase::performNodeCSE):

  • dfg/DFGClobberize.h:

(JSC::DFG::clobberize):

  • dfg/DFGCommon.h:

(JSC::DFG::enableInt52):

  • dfg/DFGFixupPhase.cpp:

(JSC::DFG::FixupPhase::run):
(JSC::DFG::FixupPhase::fixupNode):
(JSC::DFG::FixupPhase::fixupSetLocalsInBlock):
(JSC::DFG::FixupPhase::fixupUntypedSetLocalsInBlock):
(JSC::DFG::FixupPhase::observeUseKindOnNode):
(JSC::DFG::FixupPhase::fixEdge):
(JSC::DFG::FixupPhase::injectInt32ToDoubleNode):
(JSC::DFG::FixupPhase::attemptToMakeIntegerAdd):

  • dfg/DFGFlushFormat.cpp:

(WTF::printInternal):

  • dfg/DFGFlushFormat.h:

(JSC::DFG::resultFor):
(JSC::DFG::useKindFor):

  • dfg/DFGGenerationInfo.h:

(JSC::DFG::GenerationInfo::initInt52):
(JSC::DFG::GenerationInfo::initStrictInt52):
(JSC::DFG::GenerationInfo::isFormat):
(JSC::DFG::GenerationInfo::isInt52):
(JSC::DFG::GenerationInfo::isStrictInt52):
(JSC::DFG::GenerationInfo::fillInt52):
(JSC::DFG::GenerationInfo::fillStrictInt52):

  • dfg/DFGGraph.cpp:

(JSC::DFG::Graph::dump):

  • dfg/DFGGraph.h:

(JSC::DFG::Graph::addShouldSpeculateMachineInt):
(JSC::DFG::Graph::mulShouldSpeculateMachineInt):
(JSC::DFG::Graph::negateShouldSpeculateMachineInt):

  • dfg/DFGInPlaceAbstractState.cpp:

(JSC::DFG::InPlaceAbstractState::mergeStateAtTail):

  • dfg/DFGJITCode.cpp:

(JSC::DFG::JITCode::reconstruct):

  • dfg/DFGMinifiedNode.h:

(JSC::DFG::belongsInMinifiedGraph):
(JSC::DFG::MinifiedNode::hasChild):

  • dfg/DFGNode.h:

(JSC::DFG::Node::shouldSpeculateNumber):
(JSC::DFG::Node::shouldSpeculateNumberExpectingDefined):

  • dfg/DFGNodeFlags.h:
  • dfg/DFGNodeType.h:

(JSC::DFG::forwardRewiringSelectionScore):

  • dfg/DFGOSRExitCompiler.cpp:
  • dfg/DFGOSRExitCompiler64.cpp:

(JSC::DFG::OSRExitCompiler::compileExit):

  • dfg/DFGPredictionPropagationPhase.cpp:

(JSC::DFG::PredictionPropagationPhase::speculatedDoubleTypeForPrediction):
(JSC::DFG::PredictionPropagationPhase::propagate):
(JSC::DFG::PredictionPropagationPhase::doDoubleVoting):

  • dfg/DFGSafeToExecute.h:

(JSC::DFG::SafeToExecuteEdge::operator()):
(JSC::DFG::safeToExecute):

  • dfg/DFGSilentRegisterSavePlan.h:
  • dfg/DFGSpeculativeJIT.cpp:

(JSC::DFG::SpeculativeJIT::silentSavePlanForGPR):
(JSC::DFG::SpeculativeJIT::silentFill):
(JSC::DFG::SpeculativeJIT::compilePeepHoleBranch):
(JSC::DFG::SpeculativeJIT::compileInlineStart):
(JSC::DFG::SpeculativeJIT::compileDoublePutByVal):
(JSC::DFG::SpeculativeJIT::compileValueToInt32):
(JSC::DFG::SpeculativeJIT::compileInt32ToDouble):
(JSC::DFG::SpeculativeJIT::compileGetByValOnIntTypedArray):
(JSC::DFG::SpeculativeJIT::compilePutByValForIntTypedArray):
(JSC::DFG::SpeculativeJIT::compileAdd):
(JSC::DFG::SpeculativeJIT::compileArithSub):
(JSC::DFG::SpeculativeJIT::compileArithNegate):
(JSC::DFG::SpeculativeJIT::compileArithMul):
(JSC::DFG::SpeculativeJIT::compare):
(JSC::DFG::SpeculativeJIT::compileStrictEq):
(JSC::DFG::SpeculativeJIT::speculateMachineInt):
(JSC::DFG::SpeculativeJIT::speculateNumber):
(JSC::DFG::SpeculativeJIT::speculateRealNumber):
(JSC::DFG::SpeculativeJIT::speculate):

  • dfg/DFGSpeculativeJIT.h:

(JSC::DFG::SpeculativeJIT::canReuse):
(JSC::DFG::SpeculativeJIT::isFilled):
(JSC::DFG::SpeculativeJIT::isFilledDouble):
(JSC::DFG::SpeculativeJIT::use):
(JSC::DFG::SpeculativeJIT::isKnownInteger):
(JSC::DFG::SpeculativeJIT::isKnownCell):
(JSC::DFG::SpeculativeJIT::isKnownNotNumber):
(JSC::DFG::SpeculativeJIT::int52Result):
(JSC::DFG::SpeculativeJIT::strictInt52Result):
(JSC::DFG::SpeculativeJIT::initConstantInfo):
(JSC::DFG::SpeculativeJIT::isInteger):
(JSC::DFG::SpeculativeJIT::betterUseStrictInt52):
(JSC::DFG::SpeculativeJIT::generationInfo):
(JSC::DFG::SpeculateInt52Operand::SpeculateInt52Operand):
(JSC::DFG::SpeculateInt52Operand::~SpeculateInt52Operand):
(JSC::DFG::SpeculateInt52Operand::edge):
(JSC::DFG::SpeculateInt52Operand::node):
(JSC::DFG::SpeculateInt52Operand::gpr):
(JSC::DFG::SpeculateInt52Operand::use):
(JSC::DFG::SpeculateStrictInt52Operand::SpeculateStrictInt52Operand):
(JSC::DFG::SpeculateStrictInt52Operand::~SpeculateStrictInt52Operand):
(JSC::DFG::SpeculateStrictInt52Operand::edge):
(JSC::DFG::SpeculateStrictInt52Operand::node):
(JSC::DFG::SpeculateStrictInt52Operand::gpr):
(JSC::DFG::SpeculateStrictInt52Operand::use):
(JSC::DFG::SpeculateWhicheverInt52Operand::SpeculateWhicheverInt52Operand):
(JSC::DFG::SpeculateWhicheverInt52Operand::~SpeculateWhicheverInt52Operand):
(JSC::DFG::SpeculateWhicheverInt52Operand::edge):
(JSC::DFG::SpeculateWhicheverInt52Operand::node):
(JSC::DFG::SpeculateWhicheverInt52Operand::gpr):
(JSC::DFG::SpeculateWhicheverInt52Operand::use):
(JSC::DFG::SpeculateWhicheverInt52Operand::format):

  • dfg/DFGSpeculativeJIT32_64.cpp:

(JSC::DFG::SpeculativeJIT::fillSpeculateDouble):
(JSC::DFG::SpeculativeJIT::compile):

  • dfg/DFGSpeculativeJIT64.cpp:

(JSC::DFG::SpeculativeJIT::boxInt52):
(JSC::DFG::SpeculativeJIT::fillJSValue):
(JSC::DFG::SpeculativeJIT::fillSpeculateInt32Internal):
(JSC::DFG::SpeculativeJIT::fillSpeculateInt52):
(JSC::DFG::SpeculativeJIT::fillSpeculateDouble):
(JSC::DFG::SpeculativeJIT::fillSpeculateCell):
(JSC::DFG::SpeculativeJIT::fillSpeculateBoolean):
(JSC::DFG::SpeculativeJIT::compileInt52Compare):
(JSC::DFG::SpeculativeJIT::compilePeepHoleInt52Branch):
(JSC::DFG::SpeculativeJIT::compile):

  • dfg/DFGUseKind.cpp:

(WTF::printInternal):

  • dfg/DFGUseKind.h:

(JSC::DFG::typeFilterFor):
(JSC::DFG::isNumerical):

  • dfg/DFGValueSource.cpp:

(JSC::DFG::ValueSource::dump):

  • dfg/DFGValueSource.h:

(JSC::DFG::dataFormatToValueSourceKind):
(JSC::DFG::valueSourceKindToDataFormat):
(JSC::DFG::ValueSource::forFlushFormat):
(JSC::DFG::ValueSource::valueRecovery):

  • dfg/DFGVariableAccessData.h:

(JSC::DFG::VariableAccessData::shouldUseDoubleFormatAccordingToVote):
(JSC::DFG::VariableAccessData::flushFormat):

  • ftl/FTLCArgumentGetter.cpp:

(JSC::FTL::CArgumentGetter::loadNextAndBox):

  • ftl/FTLCArgumentGetter.h:
  • ftl/FTLCapabilities.cpp:

(JSC::FTL::canCompile):

  • ftl/FTLExitValue.cpp:

(JSC::FTL::ExitValue::dumpInContext):

  • ftl/FTLExitValue.h:

(JSC::FTL::ExitValue::inJSStackAsInt52):

  • ftl/FTLIntrinsicRepository.h:
  • ftl/FTLLowerDFGToLLVM.cpp:

(JSC::FTL::LowerDFGToLLVM::createPhiVariables):
(JSC::FTL::LowerDFGToLLVM::compileNode):
(JSC::FTL::LowerDFGToLLVM::compileUpsilon):
(JSC::FTL::LowerDFGToLLVM::compilePhi):
(JSC::FTL::LowerDFGToLLVM::compileSetLocal):
(JSC::FTL::LowerDFGToLLVM::compileAdd):
(JSC::FTL::LowerDFGToLLVM::compileArithSub):
(JSC::FTL::LowerDFGToLLVM::compileArithMul):
(JSC::FTL::LowerDFGToLLVM::compileArithNegate):
(JSC::FTL::LowerDFGToLLVM::compilePutByVal):
(JSC::FTL::LowerDFGToLLVM::compileCompareEq):
(JSC::FTL::LowerDFGToLLVM::compileCompareStrictEq):
(JSC::FTL::LowerDFGToLLVM::compileCompareLess):
(JSC::FTL::LowerDFGToLLVM::compileCompareLessEq):
(JSC::FTL::LowerDFGToLLVM::compileCompareGreater):
(JSC::FTL::LowerDFGToLLVM::compileCompareGreaterEq):
(JSC::FTL::LowerDFGToLLVM::lowInt32):
(JSC::FTL::LowerDFGToLLVM::lowInt52):
(JSC::FTL::LowerDFGToLLVM::lowStrictInt52):
(JSC::FTL::LowerDFGToLLVM::betterUseStrictInt52):
(JSC::FTL::LowerDFGToLLVM::bestInt52Kind):
(JSC::FTL::LowerDFGToLLVM::opposite):
(JSC::FTL::LowerDFGToLLVM::Int52s::operator[]):
(JSC::FTL::LowerDFGToLLVM::lowWhicheverInt52):
(JSC::FTL::LowerDFGToLLVM::lowWhicheverInt52s):
(JSC::FTL::LowerDFGToLLVM::lowOpposingInt52s):
(JSC::FTL::LowerDFGToLLVM::lowCell):
(JSC::FTL::LowerDFGToLLVM::lowBoolean):
(JSC::FTL::LowerDFGToLLVM::lowDouble):
(JSC::FTL::LowerDFGToLLVM::lowJSValue):
(JSC::FTL::LowerDFGToLLVM::strictInt52ToInt32):
(JSC::FTL::LowerDFGToLLVM::strictInt52ToDouble):
(JSC::FTL::LowerDFGToLLVM::strictInt52ToJSValue):
(JSC::FTL::LowerDFGToLLVM::setInt52WithStrictValue):
(JSC::FTL::LowerDFGToLLVM::strictInt52ToInt52):
(JSC::FTL::LowerDFGToLLVM::int52ToStrictInt52):
(JSC::FTL::LowerDFGToLLVM::speculateRealNumber):
(JSC::FTL::LowerDFGToLLVM::initializeOSRExitStateForBlock):
(JSC::FTL::LowerDFGToLLVM::emitOSRExitCall):
(JSC::FTL::LowerDFGToLLVM::addExitArgumentForNode):
(JSC::FTL::LowerDFGToLLVM::setInt52):
(JSC::FTL::LowerDFGToLLVM::setStrictInt52):

  • ftl/FTLOSRExitCompiler.cpp:

(JSC::FTL::compileStub):

  • ftl/FTLOutput.h:

(JSC::FTL::Output::addWithOverflow64):
(JSC::FTL::Output::subWithOverflow64):
(JSC::FTL::Output::mulWithOverflow64):

  • ftl/FTLValueFormat.cpp:

(WTF::printInternal):

  • ftl/FTLValueFormat.h:
  • ftl/FTLValueSource.cpp:

(JSC::FTL::ValueSource::dump):

  • ftl/FTLValueSource.h:
  • interpreter/Register.h:

(JSC::Register::unboxedInt52):

  • runtime/Arguments.cpp:

(JSC::Arguments::tearOffForInlineCallFrame):

  • runtime/IndexingType.cpp:

(JSC::leastUpperBoundOfIndexingTypeAndType):

  • runtime/JSCJSValue.h:
  • runtime/JSCJSValueInlines.h:

(JSC::JSValue::isMachineInt):
(JSC::JSValue::asMachineInt):

Source/WTF:

Reviewed by Oliver Hunt.

  • wtf/PrintStream.h:

(WTF::ValueIgnoringContext::ValueIgnoringContext):
(WTF::ValueIgnoringContext::dump):
(WTF::ignoringContext):

Tools:

Reviewed by Oliver Hunt.

  • Scripts/run-jsc-stress-tests:

LayoutTests:

Reviewed by Oliver Hunt.

  • js/regress/large-int-captured-expected.txt: Added.
  • js/regress/large-int-captured.html: Added.
  • js/regress/large-int-expected.txt: Added.
  • js/regress/large-int-neg-expected.txt: Added.
  • js/regress/large-int-neg.html: Added.
  • js/regress/large-int.html: Added.
  • js/regress/marsaglia-larger-ints-expected.txt: Added.
  • js/regress/marsaglia-larger-ints.html: Added.
  • js/regress/script-tests/large-int-captured.js: Added.

(.bar):
(foo):

  • js/regress/script-tests/large-int-neg.js: Added.

(foo):

  • js/regress/script-tests/large-int.js: Added.

(foo):

  • js/regress/script-tests/marsaglia-larger-ints.js: Added.

(uint):
(marsaglia):

File:
1 edited

Legend:

Unmodified
Added
Removed

  • trunk/Source/JavaScriptCore/dfg/DFGSpeculativeJIT.cpp

    r156016 r156019  
    336336        else if (registerFormat == DataFormatCell || registerFormat == DataFormatStorage)
    337337            spillAction = StorePtr;
     338        else if (registerFormat == DataFormatInt52 || registerFormat == DataFormatStrictInt52)
     339            spillAction = Store64;
    338340        else {
    339341            ASSERT(registerFormat & DataFormatJS);
    … …  
    387389        ASSERT(info.gpr() == source);
    388390        fillAction = LoadPtr;
     391    } else if (registerFormat == DataFormatInt52) {
     392        if (node->hasConstant())
     393            fillAction = SetInt52Constant;
     394        else if (isJSInt32(info.spillFormat()) || info.spillFormat() == DataFormatJS)
     395            fillAction = Load32PayloadConvertToInt52;
     396        else if (info.spillFormat() == DataFormatInt52)
     397            fillAction = Load64;
     398        else if (info.spillFormat() == DataFormatStrictInt52)
     399            fillAction = Load64ShiftInt52Left;
     400        else if (info.spillFormat() == DataFormatNone)
     401            fillAction = Load64;
     402        else {
     403            // Should never happen. Anything that qualifies as an int32 will never
     404            // be turned into a cell (immediate spec fail) or a double (to-double
     405            // conversions involve a separate node).
     406            RELEASE_ASSERT_NOT_REACHED();
     407            fillAction = Load64; // Make GCC happy.
     408        }
     409    } else if (registerFormat == DataFormatStrictInt52) {
     410        if (node->hasConstant())
     411            fillAction = SetStrictInt52Constant;
     412        else if (isJSInt32(info.spillFormat()) || info.spillFormat() == DataFormatJS)
     413            fillAction = Load32PayloadSignExtend;
     414        else if (info.spillFormat() == DataFormatInt52)
     415            fillAction = Load64ShiftInt52Right;
     416        else if (info.spillFormat() == DataFormatStrictInt52)
     417            fillAction = Load64;
     418        else if (info.spillFormat() == DataFormatNone)
     419            fillAction = Load64;
     420        else {
     421            // Should never happen. Anything that qualifies as an int32 will never
     422            // be turned into a cell (immediate spec fail) or a double (to-double
     423            // conversions involve a separate node).
     424            RELEASE_ASSERT_NOT_REACHED();
     425            fillAction = Load64; // Make GCC happy.
     426        }
    389427    } else {
    390428        ASSERT(registerFormat & DataFormatJS);
    … …  
    394432            if (valueOfJSConstant(node).isCell())
    395433                fillAction = SetTrustedJSConstant;
    396             else
    397434                fillAction = SetJSConstant;
    398435        } else if (info.spillFormat() == DataFormatInt32) {
    … …  
    513550        m_jit.move(Imm32(valueOfInt32Constant(plan.node())), plan.gpr());
    514551        break;
     552#if USE(JSVALUE64)
     553    case SetInt52Constant:
     554        m_jit.move(Imm64(valueOfJSConstant(plan.node()).asMachineInt() << JSValue::int52ShiftAmount), plan.gpr());
     555        break;
     556    case SetStrictInt52Constant:
     557        m_jit.move(Imm64(valueOfJSConstant(plan.node()).asMachineInt()), plan.gpr());
     558        break;
     559#endif // USE(JSVALUE64)
    515560    case SetBooleanConstant:
    516561        m_jit.move(TrustedImm32(valueOfBooleanConstant(plan.node())), plan.gpr());
    … …  
    534579        m_jit.or64(GPRInfo::tagTypeNumberRegister, plan.gpr());
    535580        break;
     581    case Load32PayloadConvertToInt52:
     582        m_jit.load32(JITCompiler::payloadFor(plan.node()->virtualRegister()), plan.gpr());
     583        m_jit.signExtend32ToPtr(plan.gpr(), plan.gpr());
     584        m_jit.lshift64(TrustedImm32(JSValue::int52ShiftAmount), plan.gpr());
     585        break;
     586    case Load32PayloadSignExtend:
     587        m_jit.load32(JITCompiler::payloadFor(plan.node()->virtualRegister()), plan.gpr());
     588        m_jit.signExtend32ToPtr(plan.gpr(), plan.gpr());
     589        break;
    536590    case LoadDoubleBoxDouble:
    537591        m_jit.load64(JITCompiler::addressFor(plan.node()->virtualRegister()), plan.gpr());
    … …  
    574628    case Load64:
    575629        m_jit.load64(JITCompiler::addressFor(plan.node()->virtualRegister()), plan.gpr());
     630        break;
     631    case Load64ShiftInt52Right:
     632        m_jit.load64(JITCompiler::addressFor(plan.node()->virtualRegister()), plan.gpr());
     633        m_jit.rshift64(TrustedImm32(JSValue::int52ShiftAmount), plan.gpr());
     634        break;
     635    case Load64ShiftInt52Left:
     636        m_jit.load64(JITCompiler::addressFor(plan.node()->virtualRegister()), plan.gpr());
     637        m_jit.lshift64(TrustedImm32(JSValue::int52ShiftAmount), plan.gpr());
    576638        break;
    577639#endif
    … …  
    14171479        if (node->isBinaryUseKind(Int32Use))
    14181480            compilePeepHoleInt32Branch(node, branchNode, condition);
     1481#if USE(JSVALUE64)
     1482        else if (node->isBinaryUseKind(MachineIntUse))
     1483            compilePeepHoleInt52Branch(node, branchNode, condition);
     1484#endif // USE(JSVALUE64)
    14191485        else if (node->isBinaryUseKind(NumberUse))
    14201486            compilePeepHoleDoubleBranch(node, branchNode, doubleCondition);
    … …  
    15071573            case FlushedInt32:
    15081574                valueSource = ValueSource(Int32InJSStack);
     1575                break;
     1576            case FlushedInt52:
     1577                valueSource = ValueSource(Int52InJSStack);
    15091578                break;
    15101579            case FlushedCell:
    … …  
    18741943   
    18751944    DFG_TYPE_CHECK(
    1876         JSValueRegs(), child3, SpecRealNumber,
     1945        JSValueRegs(), child3, SpecFullRealNumber,
    18771946        m_jit.branchDouble(
    18781947            MacroAssembler::DoubleNotEqualOrUnordered, valueReg, valueReg));
    … …  
    21502219        return;
    21512220    }
     2221       
     2222#if USE(JSVALUE64)
     2223    case MachineIntUse: {
     2224        SpeculateStrictInt52Operand op1(this, node->child1());
     2225        GPRTemporary result(this, Reuse, op1);
     2226        GPRReg op1GPR = op1.gpr();
     2227        GPRReg resultGPR = result.gpr();
     2228        m_jit.zeroExtend32ToPtr(op1GPR, resultGPR);
     2229        int32Result(resultGPR, node, DataFormatInt32);
     2230        return;
     2231    }
     2232#endif // USE(JSVALUE64)
    21522233   
    21532234    case NumberUse:
    … …  
    21882269            if (node->child1().useKind() == NumberUse) {
    21892270                DFG_TYPE_CHECK(
    2190                     JSValueRegs(gpr), node->child1(), SpecNumber,
     2271                    JSValueRegs(gpr), node->child1(), SpecFullNumber,
    21912272                    m_jit.branchTest64(
    21922273                        MacroAssembler::Zero, gpr, GPRInfo::tagTypeNumberRegister));
    … …  
    22442325                if (node->child1().useKind() == NumberUse) {
    22452326                    DFG_TYPE_CHECK(
    2246                         JSValueRegs(tagGPR, payloadGPR), node->child1(), SpecNumber,
     2327                        JSValueRegs(tagGPR, payloadGPR), node->child1(), SpecFullNumber,
    22472328                        m_jit.branch32(
    22482329                            MacroAssembler::AboveOrEqual, tagGPR,
    … …  
    23902471        MacroAssembler::AboveOrEqual, op1GPR, GPRInfo::tagTypeNumberRegister);
    23912472   
    2392     if (needsTypeCheck(node->child1(), SpecNumber)) {
     2473    if (needsTypeCheck(node->child1(), SpecFullNumber)) {
    23932474        if (node->flags() & NodeExitsForward) {
    23942475            forwardTypeCheck(
    2395                 JSValueRegs(op1GPR), node->child1(), SpecNumber,
     2476                JSValueRegs(op1GPR), node->child1(), SpecFullNumber,
    23962477                m_jit.branchTest64(MacroAssembler::Zero, op1GPR, GPRInfo::tagTypeNumberRegister),
    23972478                ValueRecovery::inGPR(op1GPR, DataFormatJS));
    23982479        } else {
    23992480            backwardTypeCheck(
    2400                 JSValueRegs(op1GPR), node->child1(), SpecNumber,
     2481                JSValueRegs(op1GPR), node->child1(), SpecFullNumber,
    24012482                m_jit.branchTest64(MacroAssembler::Zero, op1GPR, GPRInfo::tagTypeNumberRegister));
    24022483        }
    … …  
    24212502        MacroAssembler::Equal, op1TagGPR, TrustedImm32(JSValue::Int32Tag));
    24222503   
    2423     if (needsTypeCheck(node->child1(), SpecNumber)) {
     2504    if (needsTypeCheck(node->child1(), SpecFullNumber)) {
    24242505        if (node->flags() & NodeExitsForward) {
    24252506            forwardTypeCheck(
    2426                 JSValueRegs(op1TagGPR, op1PayloadGPR), node->child1(), SpecNumber,
     2507                JSValueRegs(op1TagGPR, op1PayloadGPR), node->child1(), SpecFullNumber,
    24272508                m_jit.branch32(MacroAssembler::AboveOrEqual, op1TagGPR, TrustedImm32(JSValue::LowestTag)),
    24282509                ValueRecovery::inPair(op1TagGPR, op1PayloadGPR));
    24292510        } else {
    24302511            backwardTypeCheck(
    2431                 JSValueRegs(op1TagGPR, op1PayloadGPR), node->child1(), SpecNumber,
     2512                JSValueRegs(op1TagGPR, op1PayloadGPR), node->child1(), SpecFullNumber,
    24322513                m_jit.branch32(MacroAssembler::AboveOrEqual, op1TagGPR, TrustedImm32(JSValue::LowestTag)));
    24332514        }
    … …  
    25492630        return;
    25502631    }
     2632   
     2633#if USE(JSVALUE64)
     2634    if (node->shouldSpeculateMachineInt()) {
     2635        m_jit.zeroExtend32ToPtr(resultReg, resultReg);
     2636        strictInt52Result(resultReg, node);
     2637        return;
     2638    }
     2639#endif
    25512640   
    25522641    FPRTemporary fresult(this);
    … …  
    26022691            break;
    26032692        }
     2693           
     2694#if USE(JSVALUE64)
     2695        case MachineIntUse: {
     2696            SpeculateStrictInt52Operand valueOp(this, valueUse);
     2697            GPRTemporary scratch(this);
     2698            GPRReg scratchReg = scratch.gpr();
     2699            m_jit.move(valueOp.gpr(), scratchReg);
     2700            if (isClamped(type)) {
     2701                ASSERT(elementSize(type) == 1);
     2702                MacroAssembler::Jump inBounds = m_jit.branch64(
     2703                    MacroAssembler::BelowOrEqual, scratchReg, JITCompiler::TrustedImm64(0xff));
     2704                MacroAssembler::Jump tooBig = m_jit.branch64(
     2705                    MacroAssembler::GreaterThan, scratchReg, JITCompiler::TrustedImm64(0xff));
     2706                m_jit.move(TrustedImm32(0), scratchReg);
     2707                MacroAssembler::Jump clamped = m_jit.jump();
     2708                tooBig.link(&m_jit);
     2709                m_jit.move(JITCompiler::TrustedImm32(255), scratchReg);
     2710                clamped.link(&m_jit);
     2711                inBounds.link(&m_jit);
     2712            }
     2713            value.adopt(scratch);
     2714            valueGPR = scratchReg;
     2715            break;
     2716        }
     2717#endif // USE(JSVALUE64)
    26042718           
    26052719        case NumberUse: {
    … …  
    28712985            return;
    28722986        }
    2873                
     2987       
    28742988        if (isNumberConstant(node->child2().node())) {
    28752989            SpeculateInt32Operand op1(this, node->child1());
    … …  
    29163030        return;
    29173031    }
     3032       
     3033#if USE(JSVALUE64)
     3034    case MachineIntUse: {
     3035        // Will we need an overflow check? If we can prove that neither input can be
     3036        // Int52 then the overflow check will not be necessary.
     3037        if (!m_state.forNode(node->child1()).couldBeType(SpecInt52)
     3038            && !m_state.forNode(node->child2()).couldBeType(SpecInt52)) {
     3039            SpeculateWhicheverInt52Operand op1(this, node->child1());
     3040            SpeculateWhicheverInt52Operand op2(this, node->child2(), op1);
     3041            GPRTemporary result(this, Reuse, op1);
     3042            m_jit.move(op1.gpr(), result.gpr());
     3043            m_jit.add64(op2.gpr(), result.gpr());
     3044            int52Result(result.gpr(), node, op1.format());
     3045            return;
     3046        }
     3047       
     3048        SpeculateInt52Operand op1(this, node->child1());
     3049        SpeculateInt52Operand op2(this, node->child2());
     3050        GPRTemporary result(this, Reuse, op1, op2);
     3051        m_jit.move(op1.gpr(), result.gpr());
     3052        speculationCheck(
     3053            Int52Overflow, JSValueRegs(), 0,
     3054            m_jit.branchAdd64(MacroAssembler::Overflow, op2.gpr(), result.gpr()));
     3055        int52Result(result.gpr(), node);
     3056        return;
     3057    }
     3058#endif // USE(JSVALUE64)
    29183059   
    29193060    case NumberUse: {
    … …  
    30613202    }
    30623203       
     3204#if USE(JSVALUE64)
     3205    case MachineIntUse: {
     3206        // Will we need an overflow check? If we can prove that neither input can be
     3207        // Int52 then the overflow check will not be necessary.
     3208        if (!m_state.forNode(node->child1()).couldBeType(SpecInt52)
     3209            && !m_state.forNode(node->child2()).couldBeType(SpecInt52)) {
     3210            SpeculateWhicheverInt52Operand op1(this, node->child1());
     3211            SpeculateWhicheverInt52Operand op2(this, node->child2(), op1);
     3212            GPRTemporary result(this, Reuse, op1);
     3213            m_jit.move(op1.gpr(), result.gpr());
     3214            m_jit.sub64(op2.gpr(), result.gpr());
     3215            int52Result(result.gpr(), node, op1.format());
     3216            return;
     3217        }
     3218       
     3219        SpeculateInt52Operand op1(this, node->child1());
     3220        SpeculateInt52Operand op2(this, node->child2());
     3221        GPRTemporary result(this, Reuse, op1, op2);
     3222        m_jit.move(op1.gpr(), result.gpr());
     3223        speculationCheck(
     3224            Int52Overflow, JSValueRegs(), 0,
     3225            m_jit.branchSub64(MacroAssembler::Overflow, op2.gpr(), result.gpr()));
     3226        int52Result(result.gpr(), node);
     3227        return;
     3228    }
     3229#endif // USE(JSVALUE64)
     3230
    30633231    case NumberUse: {
    30643232        SpeculateDoubleOperand op1(this, node->child1());
    … …  
    30893257        m_jit.move(op1.gpr(), result.gpr());
    30903258
     3259        // Note: there is no notion of being not used as a number, but someone
     3260        // caring about negative zero.
     3261       
    30913262        if (bytecodeCanTruncateInteger(node->arithNodeFlags()))
    30923263            m_jit.neg32(result.gpr());
    … …  
    31013272        return;
    31023273    }
     3274
     3275#if USE(JSVALUE64)
     3276    case MachineIntUse: {
     3277        if (!m_state.forNode(node->child1()).couldBeType(SpecInt52)) {
     3278            SpeculateWhicheverInt52Operand op1(this, node->child1());
     3279            GPRTemporary result(this);
     3280            GPRReg op1GPR = op1.gpr();
     3281            GPRReg resultGPR = result.gpr();
     3282            m_jit.move(op1GPR, resultGPR);
     3283            m_jit.neg64(resultGPR);
     3284            if (!bytecodeCanIgnoreNegativeZero(node->arithNodeFlags())) {
     3285                speculationCheck(
     3286                    NegativeZero, JSValueRegs(), 0,
     3287                    m_jit.branchTest64(MacroAssembler::Zero, resultGPR));
     3288            }
     3289            int52Result(resultGPR, node, op1.format());
     3290            return;
     3291        }
     3292       
     3293        SpeculateInt52Operand op1(this, node->child1());
     3294        GPRTemporary result(this);
     3295        GPRReg op1GPR = op1.gpr();
     3296        GPRReg resultGPR = result.gpr();
     3297        m_jit.move(op1GPR, resultGPR);
     3298        speculationCheck(
     3299            Int52Overflow, JSValueRegs(), 0,
     3300            m_jit.branchNeg64(MacroAssembler::Overflow, resultGPR));
     3301        if (!bytecodeCanIgnoreNegativeZero(node->arithNodeFlags())) {
     3302            speculationCheck(
     3303                NegativeZero, JSValueRegs(), 0,
     3304                m_jit.branchTest64(MacroAssembler::Zero, resultGPR));
     3305        }
     3306        int52Result(resultGPR, node);
     3307        return;
     3308    }
     3309#endif // USE(JSVALUE64)
    31033310       
    31043311    case NumberUse: {
    … …  
    31663373        return;
    31673374    }
     3375   
     3376#if USE(JSVALUE64)   
     3377    case MachineIntUse: {
     3378        // This is super clever. We want to do an int52 multiplication and check the
     3379        // int52 overflow bit. There is no direct hardware support for this, but we do
     3380        // have the ability to do an int64 multiplication and check the int64 overflow
     3381        // bit. We leverage that. Consider that a, b are int52 numbers inside int64
     3382        // registers, with the high 12 bits being sign-extended. We can do:
     3383        //
     3384        //     (a * (b << 12))
     3385        //
     3386        // This will give us a left-shifted int52 (value is in high 52 bits, low 16
     3387        // bits are zero) plus the int52 overflow bit. I.e. whether this 64-bit
     3388        // multiplication overflows is identical to whether the 'a * b' 52-bit
     3389        // multiplication overflows.
     3390        //
     3391        // In our nomenclature, this is:
     3392        //
     3393        //     strictInt52(a) * int52(b) => int52
     3394        //
     3395        // That is "strictInt52" means unshifted and "int52" means left-shifted by 16
     3396        // bits.
     3397        //
     3398        // We don't care which of op1 or op2 serves as the left-shifted operand, so
     3399        // we just do whatever is more convenient for op1 and have op2 do the
     3400        // opposite. This ensures that we do at most one shift.
     3401
     3402        SpeculateWhicheverInt52Operand op1(this, node->child1());
     3403        SpeculateWhicheverInt52Operand op2(this, node->child2(), OppositeShift, op1);
     3404        GPRTemporary result(this);
     3405       
     3406        GPRReg op1GPR = op1.gpr();
     3407        GPRReg op2GPR = op2.gpr();
     3408        GPRReg resultGPR = result.gpr();
     3409       
     3410        m_jit.move(op1GPR, resultGPR);
     3411        speculationCheck(
     3412            Int52Overflow, JSValueRegs(), 0,
     3413            m_jit.branchMul64(MacroAssembler::Overflow, op2GPR, resultGPR));
     3414       
     3415        if (!bytecodeCanIgnoreNegativeZero(node->arithNodeFlags())) {
     3416            MacroAssembler::Jump resultNonZero = m_jit.branchTest64(
     3417                MacroAssembler::NonZero, resultGPR);
     3418            speculationCheck(
     3419                NegativeZero, JSValueRegs(), 0,
     3420                m_jit.branch64(MacroAssembler::LessThan, op1GPR, TrustedImm64(0)));
     3421            speculationCheck(
     3422                NegativeZero, JSValueRegs(), 0,
     3423                m_jit.branch64(MacroAssembler::LessThan, op2GPR, TrustedImm64(0)));
     3424            resultNonZero.link(&m_jit);
     3425        }
     3426       
     3427        int52Result(resultGPR, node);
     3428        return;
     3429    }
     3430#endif // USE(JSVALUE64)
    31683431       
    31693432    case NumberUse: {
    … …  
    35893852        return false;
    35903853    }
    3591 
     3854   
     3855#if USE(JSVALUE64)
     3856    if (node->isBinaryUseKind(MachineIntUse)) {
     3857        compileInt52Compare(node, condition);
     3858        return false;
     3859    }
     3860#endif // USE(JSVALUE64)
     3861   
    35923862    if (node->isBinaryUseKind(NumberUse)) {
    35933863        compileDoubleCompare(node, doubleCondition);
    … …  
    37394009        return false;
    37404010    }
     4011   
     4012#if USE(JSVALUE64)   
     4013    case MachineIntUse: {
     4014        unsigned branchIndexInBlock = detectPeepHoleBranch();
     4015        if (branchIndexInBlock != UINT_MAX) {
     4016            Node* branchNode = m_block->at(branchIndexInBlock);
     4017            compilePeepHoleInt52Branch(node, branchNode, MacroAssembler::Equal);
     4018            use(node->child1());
     4019            use(node->child2());
     4020            m_indexInBlock = branchIndexInBlock;
     4021            m_currentNode = branchNode;
     4022            return true;
     4023        }
     4024        compileInt52Compare(node, MacroAssembler::Equal);
     4025        return false;
     4026    }
     4027#endif // USE(JSVALUE64)
    37414028       
    37424029    case NumberUse: {
    … …  
    44944781}
    44954782
     4783void SpeculativeJIT::speculateMachineInt(Edge edge)
     4784{
     4785#if USE(JSVALUE64)
     4786    if (!needsTypeCheck(edge, SpecMachineInt))
     4787        return;
     4788   
     4789    (SpeculateWhicheverInt52Operand(this, edge)).gpr();
     4790#else // USE(JSVALUE64)
     4791    UNUSED_PARAM(edge);
     4792    UNREACHABLE_FOR_PLATFORM();
     4793#endif // USE(JSVALUE64)
     4794}
     4795
    44964796void SpeculativeJIT::speculateNumber(Edge edge)
    44974797{
    4498     if (!needsTypeCheck(edge, SpecNumber))
     4798    if (!needsTypeCheck(edge, SpecFullNumber))
    44994799        return;
    45004800   
    … …  
    45044804void SpeculativeJIT::speculateRealNumber(Edge edge)
    45054805{
    4506     if (!needsTypeCheck(edge, SpecRealNumber))
     4806    if (!needsTypeCheck(edge, SpecFullRealNumber))
    45074807        return;
    45084808   
    … …  
    45104810    FPRReg fpr = operand.fpr();
    45114811    DFG_TYPE_CHECK(
    4512         JSValueRegs(), edge, SpecRealNumber,
     4812        JSValueRegs(), edge, SpecFullRealNumber,
    45134813        m_jit.branchDouble(
    45144814            MacroAssembler::DoubleNotEqualOrUnordered, fpr, fpr));
    … …  
    47725072        break;
    47735073    case KnownNumberUse:
    4774         ASSERT(!needsTypeCheck(edge, SpecNumber));
     5074        ASSERT(!needsTypeCheck(edge, SpecFullNumber));
    47755075        break;
    47765076    case KnownCellUse:
    … …  
    47825082    case Int32Use:
    47835083        speculateInt32(edge);
     5084        break;
     5085    case MachineIntUse:
     5086        speculateMachineInt(edge);
    47845087        break;
    47855088    case RealNumberUse:
Note: See TracChangeset for help on using the changeset viewer.