Changeset 65042 in webkit for trunk/JavaScriptCore/assembler


Ignore:
Timestamp:
Aug 9, 2010, 8:19:19 PM (15 years ago)
Author:
[email protected]
Message:

2010-08-09 Oliver Hunt <​[email protected]>

Reviewed by Gavin Barraclough.

Allow an assembler/macroassembler to compact branches to more concise forms when linking
​https://bugs.webkit.org/show_bug.cgi?id=43745

This patch makes it possible for an assembler to convert jumps into a different
(presumably more efficient) form at link time. Currently implemented in the
ARMv7 JIT as that already had logic to delay linking of jumps until the end of
compilation already. The ARMv7 JIT chooses between either a 4 byte short jump
or a full 32-bit offset (and rewrites ITTT instructions as appropriate), so does
not yet produce the most compact form possible. The general design of the linker
should make it relatively simple to introduce new branch types with little effort,
as the linker has no knowledge of the exact form of any of the branches.

  • JavaScriptCore.xcodeproj/project.pbxproj:
  • assembler/ARMv7Assembler.cpp: Added. (JSC::): Record jump sizes
  • assembler/ARMv7Assembler.h: (JSC::ARMv7Assembler::LinkRecord::LinkRecord): (JSC::ARMv7Assembler::LinkRecord::from): (JSC::ARMv7Assembler::LinkRecord::setFrom): (JSC::ARMv7Assembler::LinkRecord::to): (JSC::ARMv7Assembler::LinkRecord::type): (JSC::ARMv7Assembler::LinkRecord::linkType): (JSC::ARMv7Assembler::LinkRecord::setLinkType): Encapsulate LinkRecord fields so we can compress the values somewhat

(JSC::ARMv7Assembler::JmpSrc::JmpSrc):

Need to record the jump type now

(JSC::ARMv7Assembler::b):
(JSC::ARMv7Assembler::blx):
(JSC::ARMv7Assembler::bx):

Need to pass the jump types

(JSC::ARMv7Assembler::executableOffsetFor):
(JSC::ARMv7Assembler::jumpSizeDelta):
(JSC::ARMv7Assembler::linkRecordSourceComparator):
(JSC::ARMv7Assembler::computeJumpType):
(JSC::ARMv7Assembler::convertJumpTo):
(JSC::ARMv7Assembler::recordLinkOffsets):
(JSC::ARMv7Assembler::jumpsToLink):
(JSC::ARMv7Assembler::link):
(JSC::ARMv7Assembler::unlinkedCode):

Helper functions for the linker

(JSC::ARMv7Assembler::linkJump):
(JSC::ARMv7Assembler::canBeShortJump):
(JSC::ARMv7Assembler::linkLongJump):
(JSC::ARMv7Assembler::linkShortJump):
(JSC::ARMv7Assembler::linkJumpAbsolute):

Moving code around for the various jump linking functions

  • assembler/AbstractMacroAssembler.h: (JSC::AbstractMacroAssembler::beginUninterruptedSequence): (JSC::AbstractMacroAssembler::endUninterruptedSequence): We have to track uninterrupted sequences in any assembler that compacts branches as that's not something we're allowed to do in such sequences. AbstractMacroAssembler has a nop version of these functions as it makes the code elsewhere nicer.
  • assembler/LinkBuffer.h: (JSC::LinkBuffer::LinkBuffer): (JSC::LinkBuffer::link): (JSC::LinkBuffer::patch): (JSC::LinkBuffer::locationOf): (JSC::LinkBuffer::locationOfNearCall): (JSC::LinkBuffer::returnAddressOffset): (JSC::LinkBuffer::trampolineAt): Updated these functions to adjust for any changed offsets in the linked code

(JSC::LinkBuffer::applyOffset):

A helper function to deal with the now potentially moved labels

(JSC::LinkBuffer::linkCode):

The new and mighty linker function

  • assembler/MacroAssemblerARMv7.h: (JSC::MacroAssemblerARMv7::MacroAssemblerARMv7): (JSC::MacroAssemblerARMv7::beginUninterruptedSequence): (JSC::MacroAssemblerARMv7::endUninterruptedSequence): (JSC::MacroAssemblerARMv7::jumpsToLink): (JSC::MacroAssemblerARMv7::unlinkedCode): (JSC::MacroAssemblerARMv7::computeJumpType): (JSC::MacroAssemblerARMv7::convertJumpTo): (JSC::MacroAssemblerARMv7::recordLinkOffsets): (JSC::MacroAssemblerARMv7::jumpSizeDelta): (JSC::MacroAssemblerARMv7::link): (JSC::MacroAssemblerARMv7::jump): (JSC::MacroAssemblerARMv7::branchMul32): (JSC::MacroAssemblerARMv7::breakpoint): (JSC::MacroAssemblerARMv7::nearCall): (JSC::MacroAssemblerARMv7::call): (JSC::MacroAssemblerARMv7::ret): (JSC::MacroAssemblerARMv7::tailRecursiveCall): (JSC::MacroAssemblerARMv7::executableOffsetFor): (JSC::MacroAssemblerARMv7::inUninterruptedSequence): (JSC::MacroAssemblerARMv7::makeJump): (JSC::MacroAssemblerARMv7::makeBranch):

All branches need to pass on their type now

  • jit/ExecutableAllocator.h: (JSC::ExecutablePool::returnLastBytes):

We can't know ahead of time how much space will be necessary to
hold the linked code if we're compacting branches, this new
function allows us to return the unused bytes at the end of linking

  • jit/JIT.cpp: (JSC::JIT::JIT): (JSC::JIT::privateCompile):
  • jit/JIT.h: (JSC::JIT::compile):

The JIT class now needs to take a linker offset so that recompilation
can generate the same jumps when using branch compaction.

  • jit/JITArithmetic32_64.cpp: (JSC::JIT::emitSlow_op_mod):
  • jit/JITOpcodes.cpp: (JSC::JIT::privateCompileCTIMachineTrampolines):
  • jit/JITOpcodes32_64.cpp: (JSC::JIT::privateCompileCTIMachineTrampolines): (JSC::JIT::privateCompileCTINativeCall): Update for new trampolineAt changes
  • wtf/FastMalloc.cpp: (WTF::TCMallocStats::):
  • wtf/Platform.h:
Location:
trunk/JavaScriptCore/assembler
Files:
1 added
4 edited

Legend:

Unmodified
Added
Removed

  • trunk/JavaScriptCore/assembler/ARMv7Assembler.h

    r64608 r65042  
    382382        u.d = d;
    383383
    384         int sign = (u.i >> 63);
    385         int exponent = (u.i >> 52) & 0x7ff;
     384        int sign = static_cast<int>(u.i >> 63);
     385        int exponent = static_cast<int>(u.i >> 52) & 0x7ff;
    386386        uint64_t mantissa = u.i & 0x000fffffffffffffull;
    387387
    … …  
    445445    } m_u;
    446446};
    447 
    448447
    449448class ARMv7Assembler {
    … …  
    477476        ConditionLE,
    478477        ConditionAL,
    479 
     478       
    480479        ConditionCS = ConditionHS,
    481480        ConditionCC = ConditionLO,
    482481    } Condition;
    483482
     483    enum JumpType { JumpNoCondition, JumpCondition, JumpFullSize };
     484    enum JumpLinkType { LinkInvalid, LinkShortJump, LinkConditionalShortJump, LinkLongJump, JumpTypeCount };
     485    static const int JumpSizes[JumpTypeCount];
     486    enum { JumpPaddingSize = 5 * sizeof(uint16_t) };
     487    class LinkRecord {
     488    public:
     489        LinkRecord(intptr_t from, intptr_t to, JumpType type, Condition condition)
     490            : m_from(from)
     491            , m_to(to)
     492            , m_type(type)
     493            , m_linkType(LinkInvalid)
     494            , m_condition(condition)
     495        {
     496        }
     497        intptr_t from() const { return m_from; }
     498        void setFrom(intptr_t from) { m_from = from; }
     499        intptr_t to() const { return m_to; }
     500        JumpType type() const { return m_type; }
     501        JumpLinkType linkType() const { return m_linkType; }
     502        void setLinkType(JumpLinkType linkType) { ASSERT(m_linkType == LinkInvalid); m_linkType = linkType; }
     503        Condition condition() const { return m_condition; }
     504    private:
     505        intptr_t m_from : 31;
     506        intptr_t m_to : 31;
     507        JumpType m_type : 2;
     508        JumpLinkType m_linkType : 3;
     509        Condition m_condition : 16;
     510    };
     511   
    484512    class JmpSrc {
    485513        friend class ARMv7Assembler;
    486514        friend class ARMInstructionFormatter;
     515        friend class LinkBuffer;
    487516    public:
    488517        JmpSrc()
    … …  
    492521
    493522    private:
    494         JmpSrc(int offset)
     523        JmpSrc(int offset, JumpType type)
    495524            : m_offset(offset)
    496         {
     525            , m_condition(0xffff)
     526            , m_type(type)
     527        {
     528            ASSERT(m_type != JumpCondition);
     529        }
     530
     531        JmpSrc(int offset, JumpType type, Condition condition)
     532            : m_offset(offset)
     533            , m_condition(condition)
     534            , m_type(type)
     535        {
     536            ASSERT(m_type == JumpCondition || m_type == JumpFullSize);
    497537        }
    498538
    499539        int m_offset;
     540        Condition m_condition : 16;
     541        JumpType m_type : 16;
     542       
    500543    };
    501544   
    … …  
    503546        friend class ARMv7Assembler;
    504547        friend class ARMInstructionFormatter;
     548        friend class LinkBuffer;
    505549    public:
    506550        JmpDst()
    … …  
    525569
    526570private:
    527 
    528     struct LinkRecord {
    529         LinkRecord(intptr_t from, intptr_t to)
    530             : from(from)
    531             , to(to)
    532         {
    533         }
    534 
    535         intptr_t from;
    536         intptr_t to;
    537     };
    538571
    539572    // ARMv7, Appx-A.6.3
    … …  
    740773
    741774public:
    742 
     775   
    743776    void add(RegisterID rd, RegisterID rn, ARMThumbImmediate imm)
    744777    {
    … …  
    879912        m_formatter.twoWordOp12Reg4FourFours(OP_ASR_reg_T2, rn, FourFours(0xf, rd, 0, rm));
    880913    }
    881 
     914   
    882915    // Only allowed in IT (if then) block if last instruction.
    883     JmpSrc b()
     916    JmpSrc b(JumpType type)
    884917    {
    885918        m_formatter.twoWordOp16Op16(OP_B_T4a, OP_B_T4b);
    886         return JmpSrc(m_formatter.size());
     919        return JmpSrc(m_formatter.size(), type);
    887920    }
    888921   
    889922    // Only allowed in IT (if then) block if last instruction.
    890     JmpSrc blx(RegisterID rm)
     923    JmpSrc blx(RegisterID rm, JumpType type)
    891924    {
    892925        ASSERT(rm != ARMRegisters::pc);
    893926        m_formatter.oneWordOp8RegReg143(OP_BLX, rm, (RegisterID)8);
    894         return JmpSrc(m_formatter.size());
     927        return JmpSrc(m_formatter.size(), type);
    895928    }
    896929
    897930    // Only allowed in IT (if then) block if last instruction.
    898     JmpSrc bx(RegisterID rm)
     931    JmpSrc bx(RegisterID rm, JumpType type, Condition condition)
    899932    {
    900933        m_formatter.oneWordOp8RegReg143(OP_BX, rm, (RegisterID)0);
    901         return JmpSrc(m_formatter.size());
     934        return JmpSrc(m_formatter.size(), type, condition);
     935    }
     936
     937    JmpSrc bx(RegisterID rm, JumpType type)
     938    {
     939        m_formatter.oneWordOp8RegReg143(OP_BX, rm, (RegisterID)0);
     940        return JmpSrc(m_formatter.size(), type);
    902941    }
    903942
    … …  
    16181657        return dst.m_offset - src.m_offset;
    16191658    }
     1659
     1660    int executableOffsetFor(int location)
     1661    {
     1662        if (!location)
     1663            return 0;
     1664        return static_cast<int32_t*>(m_formatter.data())[location / sizeof(int32_t) - 1];
     1665    }
     1666   
     1667    int jumpSizeDelta(JumpLinkType jumpLinkType) { return JumpPaddingSize - JumpSizes[jumpLinkType]; }
    16201668   
    16211669    // Assembler admin methods:
    … …  
    16261674    }
    16271675
    1628     void* executableCopy(ExecutablePool* allocator)
    1629     {
    1630         void* copy = m_formatter.executableCopy(allocator);
    1631         if (!copy)
    1632             return 0;
    1633 
    1634         unsigned jumpCount = m_jumpsToLink.size();
    1635         for (unsigned i = 0; i < jumpCount; ++i) {
    1636             uint16_t* location = reinterpret_cast<uint16_t*>(reinterpret_cast<intptr_t>(copy) + m_jumpsToLink[i].from);
    1637             uint16_t* target = reinterpret_cast<uint16_t*>(reinterpret_cast<intptr_t>(copy) + m_jumpsToLink[i].to);
    1638             linkJumpAbsolute(location, target);
    1639         }
    1640         m_jumpsToLink.clear();
    1641 
    1642         return copy;
    1643     }
    1644 
     1676    static bool linkRecordSourceComparator(const LinkRecord& a, const LinkRecord& b)
     1677    {
     1678        return a.from() < b.from();
     1679    }
     1680
     1681    JumpLinkType computeJumpType(LinkRecord& record, const uint8_t* from, const uint8_t* to)
     1682    {
     1683        if (record.type() >= JumpFullSize) {
     1684            record.setLinkType(LinkLongJump);
     1685            return LinkLongJump;
     1686        }
     1687        bool mayTriggerErrata = false;
     1688        const uint16_t* shortJumpLocation = reinterpret_cast<const uint16_t*>(from  - (JumpPaddingSize - JumpSizes[LinkShortJump]));
     1689        if (!canBeShortJump(shortJumpLocation, to, mayTriggerErrata)) {
     1690            record.setLinkType(LinkLongJump);
     1691            return LinkLongJump;
     1692        }
     1693        if (mayTriggerErrata) {
     1694            record.setLinkType(LinkLongJump);
     1695            return LinkLongJump;
     1696        }
     1697        if (record.type() == JumpCondition) {
     1698            record.setLinkType(LinkConditionalShortJump);
     1699            return LinkConditionalShortJump;
     1700        }
     1701        record.setLinkType(LinkShortJump);
     1702        return LinkShortJump;
     1703    }
     1704
     1705    void recordLinkOffsets(int32_t regionStart, int32_t regionEnd, int32_t offset)
     1706    {
     1707        int32_t ptr = regionStart / sizeof(int32_t);
     1708        const int32_t end = regionEnd / sizeof(int32_t);
     1709        int32_t* offsets = static_cast<int32_t*>(m_formatter.data());
     1710        while (ptr < end)
     1711            offsets[ptr++] = offset;
     1712    }
     1713   
     1714    Vector<LinkRecord>& jumpsToLink()
     1715    {
     1716        std::sort(m_jumpsToLink.begin(), m_jumpsToLink.end(), linkRecordSourceComparator);
     1717        return m_jumpsToLink;
     1718    }
     1719
     1720    void link(LinkRecord& record, uint8_t* from, uint8_t* to)
     1721    {
     1722        uint16_t* itttLocation;
     1723        if (record.linkType() == LinkConditionalShortJump) {
     1724            itttLocation = reinterpret_cast<uint16_t*>(from - JumpSizes[LinkConditionalShortJump] - 2);
     1725            itttLocation[0] = ifThenElse(record.condition()) | OP_IT;
     1726        }
     1727        ASSERT(record.linkType() != LinkInvalid);
     1728        if (record.linkType() != LinkLongJump)
     1729            linkShortJump(reinterpret_cast<uint16_t*>(from), to);
     1730        else
     1731            linkLongJump(reinterpret_cast<uint16_t*>(from), to);
     1732    }
     1733
     1734    void* unlinkedCode() { return m_formatter.data(); }
     1735   
    16451736    static unsigned getCallReturnOffset(JmpSrc call)
    16461737    {
    … …  
    16611752        ASSERT(to.m_offset != -1);
    16621753        ASSERT(from.m_offset != -1);
    1663         m_jumpsToLink.append(LinkRecord(from.m_offset, to.m_offset));
     1754        m_jumpsToLink.append(LinkRecord(from.m_offset, to.m_offset, from.m_type, from.m_condition));
    16641755    }
    16651756
    … …  
    18641955    }
    18651956
    1866     static void linkJumpAbsolute(uint16_t* instruction, void* target)
    1867     {
    1868         // FIMXE: this should be up in the MacroAssembler layer. :-(
    1869         const uint16_t JUMP_TEMPORARY_REGISTER = ARMRegisters::ip;
    1870 
     1957    static bool canBeShortJump(const uint16_t* instruction, const void* target, bool& mayTriggerErrata)
     1958    {
    18711959        ASSERT(!(reinterpret_cast<intptr_t>(instruction) & 1));
    18721960        ASSERT(!(reinterpret_cast<intptr_t>(target) & 1));
    1873 
    1874         ASSERT( (isMOV_imm_T3(instruction - 5) && isMOVT(instruction - 3) && isBX(instruction - 1))
    1875             || (isNOP_T1(instruction - 5) && isNOP_T2(instruction - 4) && isB(instruction - 2)) );
    1876 
     1961       
    18771962        intptr_t relative = reinterpret_cast<intptr_t>(target) - (reinterpret_cast<intptr_t>(instruction));
    1878 
    18791963        // From Cortex-A8 errata:
    18801964        // If the 32-bit Thumb-2 branch instruction spans two 4KiB regions and
    … …  
    18851969        // The instruction is spanning two pages if it ends at an address ending 0x002
    18861970        bool spansTwo4K = ((reinterpret_cast<intptr_t>(instruction) & 0xfff) == 0x002);
     1971        mayTriggerErrata = spansTwo4K;
    18871972        // The target is in the first page if the jump branch back by [3..0x1002] bytes
    18881973        bool targetInFirstPage = (relative >= -0x1002) && (relative < -2);
    18891974        bool wouldTriggerA8Errata = spansTwo4K && targetInFirstPage;
    1890 
    1891         if (((relative << 7) >> 7) == relative && !wouldTriggerA8Errata) {
     1975        return ((relative << 7) >> 7) == relative && !wouldTriggerA8Errata;
     1976    }
     1977
     1978    static void linkLongJump(uint16_t* instruction, void* target)
     1979    {
     1980        linkJumpAbsolute(instruction, target);
     1981    }
     1982   
     1983    static void linkShortJump(uint16_t* instruction, void* target)
     1984    {
     1985        // FIMXE: this should be up in the MacroAssembler layer. :-(       
     1986        ASSERT(!(reinterpret_cast<intptr_t>(instruction) & 1));
     1987        ASSERT(!(reinterpret_cast<intptr_t>(target) & 1));
     1988       
     1989        intptr_t relative = reinterpret_cast<intptr_t>(target) - (reinterpret_cast<intptr_t>(instruction));
     1990        bool scratch;
     1991        UNUSED_PARAM(scratch);
     1992        ASSERT(canBeShortJump(instruction, target, scratch));
     1993        // ARM encoding for the top two bits below the sign bit is 'peculiar'.
     1994        if (relative >= 0)
     1995            relative ^= 0xC00000;
     1996
     1997        // All branch offsets should be an even distance.
     1998        ASSERT(!(relative & 1));
     1999        instruction[-2] = OP_B_T4a | ((relative & 0x1000000) >> 14) | ((relative & 0x3ff000) >> 12);
     2000        instruction[-1] = OP_B_T4b | ((relative & 0x800000) >> 10) | ((relative & 0x400000) >> 11) | ((relative & 0xffe) >> 1);
     2001    }
     2002
     2003    static void linkJumpAbsolute(uint16_t* instruction, void* target)
     2004    {
     2005        // FIMXE: this should be up in the MacroAssembler layer. :-(
     2006        ASSERT(!(reinterpret_cast<intptr_t>(instruction) & 1));
     2007        ASSERT(!(reinterpret_cast<intptr_t>(target) & 1));
     2008
     2009        ASSERT((isMOV_imm_T3(instruction - 5) && isMOVT(instruction - 3) && isBX(instruction - 1))
     2010            || (isNOP_T1(instruction - 5) && isNOP_T2(instruction - 4) && isB(instruction - 2)));
     2011
     2012        intptr_t relative = reinterpret_cast<intptr_t>(target) - (reinterpret_cast<intptr_t>(instruction));
     2013        bool scratch;
     2014        if (canBeShortJump(instruction, target, scratch)) {
    18922015            // ARM encoding for the top two bits below the sign bit is 'peculiar'.
    18932016            if (relative >= 0)
    … …  
    19072030            instruction[-1] = OP_B_T4b | ((relative & 0x800000) >> 10) | ((relative & 0x400000) >> 11) | ((relative & 0xffe) >> 1);
    19082031        } else {
     2032            const uint16_t JUMP_TEMPORARY_REGISTER = ARMRegisters::ip;
    19092033            ARMThumbImmediate lo16 = ARMThumbImmediate::makeUInt16(static_cast<uint16_t>(reinterpret_cast<uint32_t>(target) + 1));
    19102034            ARMThumbImmediate hi16 = ARMThumbImmediate::makeUInt16(static_cast<uint16_t>(reinterpret_cast<uint32_t>(target) >> 16));
    … …  
    19212045        return op | (imm.m_value.i << 10) | imm.m_value.imm4;
    19222046    }
     2047
    19232048    static uint16_t twoWordOp5i6Imm4Reg4EncodedImmSecond(uint16_t rd, ARMThumbImmediate imm)
    19242049    {
    … …  
    20372162
    20382163    Vector<LinkRecord> m_jumpsToLink;
     2164    Vector<int32_t> m_offsets;
    20392165};
    20402166

  • trunk/JavaScriptCore/assembler/AbstractMacroAssembler.h

    r64943 r65042  
    419419
    420420    // Section 3: Misc admin methods
    421 
    422     static CodePtr trampolineAt(CodeRef ref, Label label)
    423     {
    424         return CodePtr(AssemblerType::getRelocatedAddress(ref.m_code.dataLocation(), label.m_label));
    425     }
    426 
    427421    size_t size()
    428422    {
    … …  
    480474        return AssemblerType::getDifferenceBetweenLabels(from.m_label, to.m_jmp);
    481475    }
     476   
     477    void beginUninterruptedSequence() { }
     478    void endUninterruptedSequence() { }
    482479
    483480protected:

  • trunk/JavaScriptCore/assembler/LinkBuffer.h

    r64608 r65042  
    5050class LinkBuffer : public Noncopyable {
    5151    typedef MacroAssemblerCodeRef CodeRef;
     52    typedef MacroAssemblerCodePtr CodePtr;
    5253    typedef MacroAssembler::Label Label;
    5354    typedef MacroAssembler::Jump Jump;
    … …  
    5657    typedef MacroAssembler::DataLabel32 DataLabel32;
    5758    typedef MacroAssembler::DataLabelPtr DataLabelPtr;
     59    typedef MacroAssembler::JmpDst JmpDst;
     60#if ENABLE(BRANCH_COMPACTION)
     61    typedef MacroAssembler::LinkRecord LinkRecord;
     62    typedef MacroAssembler::JumpLinkType JumpLinkType;
     63#endif
    5864
    5965    enum LinkBufferState {
    … …  
    6773    //       First, executablePool is copied into m_executablePool, then the initialization of
    6874    //       m_code uses m_executablePool, *not* executablePool, since this is no longer valid.
    69     LinkBuffer(MacroAssembler* masm, PassRefPtr<ExecutablePool> executablePool)
     75    // The linkOffset parameter should only be non-null when recompiling for exception info
     76    LinkBuffer(MacroAssembler* masm, PassRefPtr<ExecutablePool> executablePool, void* linkOffset)
    7077        : m_executablePool(executablePool)
    71         , m_code(masm->m_assembler.executableCopy(m_executablePool.get()))
    72         , m_size(masm->m_assembler.size())
     78        , m_size(0)
     79        , m_code(0)
     80        , m_assembler(masm)
    7381#ifndef NDEBUG
    7482        , m_state(StateInit)
    7583#endif
    7684    {
     85        linkCode(linkOffset);
    7786    }
    7887
    … …  
    98107    {
    99108        ASSERT(call.isFlagSet(Call::Linkable));
     109        call.m_jmp = applyOffset(call.m_jmp);
    100110        MacroAssembler::linkCall(code(), call, function);
    101111    }
    … …  
    103113    void link(Jump jump, CodeLocationLabel label)
    104114    {
     115        jump.m_jmp = applyOffset(jump.m_jmp);
    105116        MacroAssembler::linkJump(code(), jump, label);
    106117    }
    … …  
    109120    {
    110121        for (unsigned i = 0; i < list.m_jumps.size(); ++i)
    111             MacroAssembler::linkJump(code(), list.m_jumps[i], label);
     122            link(list.m_jumps[i], label);
    112123    }
    113124
    114125    void patch(DataLabelPtr label, void* value)
    115126    {
    116         MacroAssembler::linkPointer(code(), label.m_label, value);
     127        JmpDst target = applyOffset(label.m_label);
     128        MacroAssembler::linkPointer(code(), target, value);
    117129    }
    118130
    119131    void patch(DataLabelPtr label, CodeLocationLabel value)
    120132    {
    121         MacroAssembler::linkPointer(code(), label.m_label, value.executableAddress());
     133        JmpDst target = applyOffset(label.m_label);
     134        MacroAssembler::linkPointer(code(), target, value.executableAddress());
    122135    }
    123136
    … …  
    128141        ASSERT(call.isFlagSet(Call::Linkable));
    129142        ASSERT(!call.isFlagSet(Call::Near));
    130         return CodeLocationCall(MacroAssembler::getLinkerAddress(code(), call.m_jmp));
     143        return CodeLocationCall(MacroAssembler::getLinkerAddress(code(), applyOffset(call.m_jmp)));
    131144    }
    132145
    … …  
    135148        ASSERT(call.isFlagSet(Call::Linkable));
    136149        ASSERT(call.isFlagSet(Call::Near));
    137         return CodeLocationNearCall(MacroAssembler::getLinkerAddress(code(), call.m_jmp));
     150        return CodeLocationNearCall(MacroAssembler::getLinkerAddress(code(), applyOffset(call.m_jmp)));
    138151    }
    139152
    140153    CodeLocationLabel locationOf(Label label)
    141154    {
    142         return CodeLocationLabel(MacroAssembler::getLinkerAddress(code(), label.m_label));
     155        return CodeLocationLabel(MacroAssembler::getLinkerAddress(code(), applyOffset(label.m_label)));
    143156    }
    144157
    145158    CodeLocationDataLabelPtr locationOf(DataLabelPtr label)
    146159    {
    147         return CodeLocationDataLabelPtr(MacroAssembler::getLinkerAddress(code(), label.m_label));
     160        return CodeLocationDataLabelPtr(MacroAssembler::getLinkerAddress(code(), applyOffset(label.m_label)));
    148161    }
    149162
    150163    CodeLocationDataLabel32 locationOf(DataLabel32 label)
    151164    {
    152         return CodeLocationDataLabel32(MacroAssembler::getLinkerAddress(code(), label.m_label));
     165        return CodeLocationDataLabel32(MacroAssembler::getLinkerAddress(code(), applyOffset(label.m_label)));
    153166    }
    154167
    … …  
    157170    unsigned returnAddressOffset(Call call)
    158171    {
     172        call.m_jmp = applyOffset(call.m_jmp);
    159173        return MacroAssembler::getLinkerCallReturnOffset(call);
    160174    }
    … …  
    170184        return CodeRef(m_code, m_executablePool, m_size);
    171185    }
     186
    172187    CodeLocationLabel finalizeCodeAddendum()
    173188    {
    … …  
    177192    }
    178193
     194    CodePtr trampolineAt(Label label)
     195    {
     196        return CodePtr(MacroAssembler::AssemblerType_T::getRelocatedAddress(code(), applyOffset(label.m_label)));
     197    }
     198
    179199private:
     200    template <typename T> T applyOffset(T src)
     201    {
     202#if ENABLE(BRANCH_COMPACTION)
     203        src.m_offset -= m_assembler->executableOffsetFor(src.m_offset);
     204#endif
     205        return src;
     206    }
     207   
    180208    // Keep this private! - the underlying code should only be obtained externally via
    181209    // finalizeCode() or finalizeCodeAddendum().
    … …  
    185213    }
    186214
     215    void linkCode(void* linkOffset)
     216    {
     217        UNUSED_PARAM(linkOffset);
     218        ASSERT(!m_code);
     219#if !ENABLE(BRANCH_COMPACTION)
     220        m_code = m_assembler->m_assembler.executableCopy(m_executablePool.get());
     221        m_size = m_assembler->size();
     222#else
     223        size_t initialSize = m_assembler->size();
     224        m_code = (uint8_t*)m_executablePool->alloc(initialSize);
     225        if (!m_code)
     226            return;
     227        ExecutableAllocator::makeWritable(m_code, m_assembler->size());
     228        uint8_t* inData = (uint8_t*)m_assembler->unlinkedCode();
     229        uint8_t* outData = reinterpret_cast<uint8_t*>(m_code);
     230        const uint8_t* linkBase = linkOffset ? reinterpret_cast<uint8_t*>(linkOffset) : outData;
     231        int readPtr = 0;
     232        int writePtr = 0;
     233        Vector<LinkRecord>& jumpsToLink = m_assembler->jumpsToLink();
     234        unsigned jumpCount = jumpsToLink.size();
     235        for (unsigned i = 0; i < jumpCount; ++i) {
     236            int offset = readPtr - writePtr;
     237            ASSERT(!(offset & 1));
     238           
     239            // Copy the instructions from the last jump to the current one.
     240            size_t regionSize = jumpsToLink[i].from() - readPtr;
     241            memcpy(outData + writePtr, inData + readPtr, regionSize);
     242            m_assembler->recordLinkOffsets(readPtr, jumpsToLink[i].from(), offset);
     243            readPtr += regionSize;
     244            writePtr += regionSize;
     245           
     246            // Calculate absolute address of the jump target, in the case of backwards
     247            // branches we need to be precise, forward branches we are pessimistic
     248            const uint8_t* target;
     249            if (jumpsToLink[i].to() >= jumpsToLink[i].from())
     250                target = linkBase + jumpsToLink[i].to() - offset; // Compensate for what we have collapsed so far
     251            else
     252                target = linkBase + jumpsToLink[i].to() - m_assembler->executableOffsetFor(jumpsToLink[i].to());
     253           
     254            JumpLinkType jumpLinkType = m_assembler->computeJumpType(jumpsToLink[i], linkBase + writePtr, target);
     255
     256            // Step back in the write stream
     257            int32_t delta = m_assembler->jumpSizeDelta(jumpLinkType);
     258            if (delta) {
     259                writePtr -= delta;
     260                m_assembler->recordLinkOffsets(jumpsToLink[i].from() - delta, readPtr, readPtr - writePtr);
     261            }
     262            jumpsToLink[i].setFrom(writePtr);
     263        }
     264        // Copy everything after the last jump
     265        memcpy(outData + writePtr, inData + readPtr, m_assembler->size() - readPtr);
     266        m_assembler->recordLinkOffsets(readPtr, m_assembler->size(), readPtr - writePtr);
     267       
     268        // Actually link everything (don't link if we've be given a linkoffset as it's a
     269        // waste of time: linkOffset is used for recompiling to get exception info)
     270        if (!linkOffset) {
     271            for (unsigned i = 0; i < jumpCount; ++i) {
     272                uint8_t* location = outData + jumpsToLink[i].from();
     273                uint8_t* target = outData + jumpsToLink[i].to() - m_assembler->executableOffsetFor(jumpsToLink[i].to());
     274                m_assembler->link(jumpsToLink[i], location, target);
     275            }
     276        }
     277
     278        jumpsToLink.clear();
     279        m_size = writePtr + m_assembler->size() - readPtr;
     280        m_executablePool->returnLastBytes(initialSize - m_size);
     281#endif
     282    }
     283
    187284    void performFinalization()
    188285    {
    … …  
    197294
    198295    RefPtr<ExecutablePool> m_executablePool;
     296    size_t m_size;
    199297    void* m_code;
    200     size_t m_size;
     298    MacroAssembler* m_assembler;
    201299#ifndef NDEBUG
    202300    LinkBufferState m_state;

  • trunk/JavaScriptCore/assembler/MacroAssemblerARMv7.h

    r62419 r65042  
    4646
    4747public:
     48    typedef ARMv7Assembler::LinkRecord LinkRecord;
     49    typedef ARMv7Assembler::JumpLinkType JumpLinkType;
     50
     51    MacroAssemblerARMv7()
     52        : m_inUninterruptedSequence(false)
     53    {
     54    }
     55   
     56    void beginUninterruptedSequence() { m_inUninterruptedSequence = true; }
     57    void endUninterruptedSequence() { m_inUninterruptedSequence = false; }
     58    Vector<LinkRecord>& jumpsToLink() { return m_assembler.jumpsToLink(); }
     59    void* unlinkedCode() { return m_assembler.unlinkedCode(); }
     60    JumpLinkType computeJumpType(LinkRecord& record, const uint8_t* from, const uint8_t* to) { return m_assembler.computeJumpType(record, from, to); }
     61    void recordLinkOffsets(int32_t regionStart, int32_t regionEnd, int32_t offset) {return m_assembler.recordLinkOffsets(regionStart, regionEnd, offset); }
     62    int jumpSizeDelta(JumpLinkType jumpLinkType) { return m_assembler.jumpSizeDelta(jumpLinkType); }
     63    void link(LinkRecord& record, uint8_t* from, uint8_t* to) { return m_assembler.link(record, from, to); }
     64
    4865    struct ArmAddress {
    4966        enum AddressType {
    … …  
    970987    void jump(RegisterID target)
    971988    {
    972         m_assembler.bx(target);
     989        m_assembler.bx(target, inUninterruptedSequence() ? ARMv7Assembler::JumpFullSize : ARMv7Assembler::JumpNoCondition);
    973990    }
    974991
    … …  
    977994    {
    978995        load32(address, dataTempRegister);
    979         m_assembler.bx(dataTempRegister);
     996        m_assembler.bx(dataTempRegister, inUninterruptedSequence() ? ARMv7Assembler::JumpFullSize : ARMv7Assembler::JumpNoCondition);
    980997    }
    981998
    … …  
    10131030    Jump branchMul32(Condition cond, RegisterID src, RegisterID dest)
    10141031    {
    1015         ASSERT(cond == Overflow);
     1032        ASSERT_UNUSED(cond, cond == Overflow);
    10161033        m_assembler.smull(dest, dataTempRegister, dest, src);
    10171034        m_assembler.asr(addressTempRegister, dest, 31);
    … …  
    10211038    Jump branchMul32(Condition cond, Imm32 imm, RegisterID src, RegisterID dest)
    10221039    {
    1023         ASSERT(cond == Overflow);
     1040        ASSERT_UNUSED(cond, cond == Overflow);
    10241041        move(imm, dataTempRegister);
    10251042        m_assembler.smull(dest, dataTempRegister, src, dataTempRegister);
    … …  
    10601077    void breakpoint()
    10611078    {
    1062         m_assembler.bkpt();
     1079        m_assembler.bkpt(0);
    10631080    }
    10641081
    … …  
    10661083    {
    10671084        moveFixedWidthEncoding(Imm32(0), dataTempRegister);
    1068         return Call(m_assembler.blx(dataTempRegister), Call::LinkableNear);
     1085        return Call(m_assembler.blx(dataTempRegister, ARMv7Assembler::JumpFullSize), Call::LinkableNear);
    10691086    }
    10701087
    … …  
    10721089    {
    10731090        moveFixedWidthEncoding(Imm32(0), dataTempRegister);
    1074         return Call(m_assembler.blx(dataTempRegister), Call::Linkable);
     1091        return Call(m_assembler.blx(dataTempRegister, ARMv7Assembler::JumpFullSize), Call::Linkable);
    10751092    }
    10761093
    10771094    Call call(RegisterID target)
    10781095    {
    1079         return Call(m_assembler.blx(target), Call::None);
     1096        return Call(m_assembler.blx(target, ARMv7Assembler::JumpFullSize), Call::None);
    10801097    }
    10811098
    … …  
    10831100    {
    10841101        load32(address, dataTempRegister);
    1085         return Call(m_assembler.blx(dataTempRegister), Call::None);
     1102        return Call(m_assembler.blx(dataTempRegister, ARMv7Assembler::JumpFullSize), Call::None);
    10861103    }
    10871104
    10881105    void ret()
    10891106    {
    1090         m_assembler.bx(linkRegister);
     1107        m_assembler.bx(linkRegister, ARMv7Assembler::JumpFullSize);
    10911108    }
    10921109
    … …  
    11881205        // Like a normal call, but don't link.
    11891206        moveFixedWidthEncoding(Imm32(0), dataTempRegister);
    1190         return Call(m_assembler.bx(dataTempRegister), Call::Linkable);
     1207        return Call(m_assembler.bx(dataTempRegister, ARMv7Assembler::JumpFullSize), Call::Linkable);
    11911208    }
    11921209
    … …  
    11971214    }
    11981215
     1216   
     1217    int executableOffsetFor(int location)
     1218    {
     1219        return m_assembler.executableOffsetFor(location);
     1220    }
    11991221
    12001222protected:
     1223    bool inUninterruptedSequence()
     1224    {
     1225        return m_inUninterruptedSequence;
     1226    }
     1227
    12011228    ARMv7Assembler::JmpSrc makeJump()
    12021229    {
    12031230        moveFixedWidthEncoding(Imm32(0), dataTempRegister);
    1204         return m_assembler.bx(dataTempRegister);
     1231        return m_assembler.bx(dataTempRegister, inUninterruptedSequence() ? ARMv7Assembler::JumpFullSize : ARMv7Assembler::JumpNoCondition);
    12051232    }
    12061233
    … …  
    12091236        m_assembler.it(cond, true, true);
    12101237        moveFixedWidthEncoding(Imm32(0), dataTempRegister);
    1211         return m_assembler.bx(dataTempRegister);
     1238        return m_assembler.bx(dataTempRegister, inUninterruptedSequence() ? ARMv7Assembler::JumpFullSize : ARMv7Assembler::JumpCondition, cond);
    12121239    }
    12131240    ARMv7Assembler::JmpSrc makeBranch(Condition cond) { return makeBranch(armV7Condition(cond)); }
    … …  
    12991326        ARMv7Assembler::relinkCall(call.dataLocation(), destination.executableAddress());
    13001327    }
     1328   
     1329    bool m_inUninterruptedSequence;
    13011330};
    13021331
Note: See TracChangeset for help on using the changeset viewer.