Changeset 65042 in webkit for trunk/JavaScriptCore/assembler/LinkBuffer.h

Timestamp:

Aug 9, 2010, 8:19:19 PM (15 years ago)

Author:

[email protected]

Message:

2010-08-09 Oliver Hunt <​[email protected]>

Reviewed by Gavin Barraclough.

Allow an assembler/macroassembler to compact branches to more concise forms when linking
​https://bugs.webkit.org/show_bug.cgi?id=43745

This patch makes it possible for an assembler to convert jumps into a different
(presumably more efficient) form at link time. Currently implemented in the
ARMv7 JIT as that already had logic to delay linking of jumps until the end of
compilation already. The ARMv7 JIT chooses between either a 4 byte short jump
or a full 32-bit offset (and rewrites ITTT instructions as appropriate), so does
not yet produce the most compact form possible. The general design of the linker
should make it relatively simple to introduce new branch types with little effort,
as the linker has no knowledge of the exact form of any of the branches.

• JavaScriptCore.xcodeproj/project.pbxproj:
• assembler/ARMv7Assembler.cpp: Added. (JSC::): Record jump sizes

• assembler/ARMv7Assembler.h: (JSC::ARMv7Assembler::LinkRecord::LinkRecord): (JSC::ARMv7Assembler::LinkRecord::from): (JSC::ARMv7Assembler::LinkRecord::setFrom): (JSC::ARMv7Assembler::LinkRecord::to): (JSC::ARMv7Assembler::LinkRecord::type): (JSC::ARMv7Assembler::LinkRecord::linkType): (JSC::ARMv7Assembler::LinkRecord::setLinkType): Encapsulate LinkRecord fields so we can compress the values somewhat

(JSC::ARMv7Assembler::JmpSrc::JmpSrc):

Need to record the jump type now

(JSC::ARMv7Assembler::b):
(JSC::ARMv7Assembler::blx):
(JSC::ARMv7Assembler::bx):

Need to pass the jump types

(JSC::ARMv7Assembler::executableOffsetFor):
(JSC::ARMv7Assembler::jumpSizeDelta):
(JSC::ARMv7Assembler::linkRecordSourceComparator):
(JSC::ARMv7Assembler::computeJumpType):
(JSC::ARMv7Assembler::convertJumpTo):
(JSC::ARMv7Assembler::recordLinkOffsets):
(JSC::ARMv7Assembler::jumpsToLink):
(JSC::ARMv7Assembler::link):
(JSC::ARMv7Assembler::unlinkedCode):

Helper functions for the linker

(JSC::ARMv7Assembler::linkJump):
(JSC::ARMv7Assembler::canBeShortJump):
(JSC::ARMv7Assembler::linkLongJump):
(JSC::ARMv7Assembler::linkShortJump):
(JSC::ARMv7Assembler::linkJumpAbsolute):

Moving code around for the various jump linking functions

• assembler/AbstractMacroAssembler.h: (JSC::AbstractMacroAssembler::beginUninterruptedSequence): (JSC::AbstractMacroAssembler::endUninterruptedSequence): We have to track uninterrupted sequences in any assembler that compacts branches as that's not something we're allowed to do in such sequences. AbstractMacroAssembler has a nop version of these functions as it makes the code elsewhere nicer.

• assembler/LinkBuffer.h: (JSC::LinkBuffer::LinkBuffer): (JSC::LinkBuffer::link): (JSC::LinkBuffer::patch): (JSC::LinkBuffer::locationOf): (JSC::LinkBuffer::locationOfNearCall): (JSC::LinkBuffer::returnAddressOffset): (JSC::LinkBuffer::trampolineAt): Updated these functions to adjust for any changed offsets in the linked code

(JSC::LinkBuffer::applyOffset):

A helper function to deal with the now potentially moved labels

(JSC::LinkBuffer::linkCode):

The new and mighty linker function

• assembler/MacroAssemblerARMv7.h: (JSC::MacroAssemblerARMv7::MacroAssemblerARMv7): (JSC::MacroAssemblerARMv7::beginUninterruptedSequence): (JSC::MacroAssemblerARMv7::endUninterruptedSequence): (JSC::MacroAssemblerARMv7::jumpsToLink): (JSC::MacroAssemblerARMv7::unlinkedCode): (JSC::MacroAssemblerARMv7::computeJumpType): (JSC::MacroAssemblerARMv7::convertJumpTo): (JSC::MacroAssemblerARMv7::recordLinkOffsets): (JSC::MacroAssemblerARMv7::jumpSizeDelta): (JSC::MacroAssemblerARMv7::link): (JSC::MacroAssemblerARMv7::jump): (JSC::MacroAssemblerARMv7::branchMul32): (JSC::MacroAssemblerARMv7::breakpoint): (JSC::MacroAssemblerARMv7::nearCall): (JSC::MacroAssemblerARMv7::call): (JSC::MacroAssemblerARMv7::ret): (JSC::MacroAssemblerARMv7::tailRecursiveCall): (JSC::MacroAssemblerARMv7::executableOffsetFor): (JSC::MacroAssemblerARMv7::inUninterruptedSequence): (JSC::MacroAssemblerARMv7::makeJump): (JSC::MacroAssemblerARMv7::makeBranch):

All branches need to pass on their type now

• jit/ExecutableAllocator.h: (JSC::ExecutablePool::returnLastBytes):

We can't know ahead of time how much space will be necessary to
hold the linked code if we're compacting branches, this new
function allows us to return the unused bytes at the end of linking

• jit/JIT.cpp: (JSC::JIT::JIT): (JSC::JIT::privateCompile):
• jit/JIT.h: (JSC::JIT::compile):

The JIT class now needs to take a linker offset so that recompilation
can generate the same jumps when using branch compaction.

• jit/JITArithmetic32_64.cpp: (JSC::JIT::emitSlow_op_mod):
• jit/JITOpcodes.cpp: (JSC::JIT::privateCompileCTIMachineTrampolines):
• jit/JITOpcodes32_64.cpp: (JSC::JIT::privateCompileCTIMachineTrampolines): (JSC::JIT::privateCompileCTINativeCall): Update for new trampolineAt changes

• wtf/FastMalloc.cpp: (WTF::TCMallocStats::):
• wtf/Platform.h:

File:

• trunk/JavaScriptCore/assembler/LinkBuffer.h (modified) (13 diffs)

trunk/JavaScriptCore/assembler/LinkBuffer.h

@@ -50 +50 @@
 class LinkBuffer : public Noncopyable {
     typedef MacroAssemblerCodeRef CodeRef;
+    typedef MacroAssemblerCodePtr CodePtr;
     typedef MacroAssembler::Label Label;
     typedef MacroAssembler::Jump Jump;
@@ -56 +57 @@
     typedef MacroAssembler::DataLabel32 DataLabel32;
     typedef MacroAssembler::DataLabelPtr DataLabelPtr;
+    typedef MacroAssembler::JmpDst JmpDst;
+#if ENABLE(BRANCH_COMPACTION)
+    typedef MacroAssembler::LinkRecord LinkRecord;
+    typedef MacroAssembler::JumpLinkType JumpLinkType;
+#endif
 
     enum LinkBufferState {
@@ -67 +73 @@
     //       First, executablePool is copied into m_executablePool, then the initialization of
     //       m_code uses m_executablePool, *not* executablePool, since this is no longer valid.
-    LinkBuffer(MacroAssembler* masm, PassRefPtr<ExecutablePool> executablePool)
+    // The linkOffset parameter should only be non-null when recompiling for exception info
+    LinkBuffer(MacroAssembler* masm, PassRefPtr<ExecutablePool> executablePool, void* linkOffset)
         : m_executablePool(executablePool)
-        , m_code(masm->m_assembler.executableCopy(m_executablePool.get()))
-        , m_size(masm->m_assembler.size())
+        , m_size(0)
+        , m_code(0)
+        , m_assembler(masm)
 #ifndef NDEBUG
         , m_state(StateInit)
 #endif
     {
+        linkCode(linkOffset);
     }
 
@@ -98 +107 @@
     {
         ASSERT(call.isFlagSet(Call::Linkable));
+        call.m_jmp = applyOffset(call.m_jmp);
         MacroAssembler::linkCall(code(), call, function);
     }
@@ -103 +113 @@
     void link(Jump jump, CodeLocationLabel label)
     {
+        jump.m_jmp = applyOffset(jump.m_jmp);
         MacroAssembler::linkJump(code(), jump, label);
     }
@@ -109 +120 @@
     {
         for (unsigned i = 0; i < list.m_jumps.size(); ++i)
-            MacroAssembler::linkJump(code(), list.m_jumps[i], label);
+            link(list.m_jumps[i], label);
     }
 
     void patch(DataLabelPtr label, void* value)
     {
-        MacroAssembler::linkPointer(code(), label.m_label, value);
+        JmpDst target = applyOffset(label.m_label);
+        MacroAssembler::linkPointer(code(), target, value);
     }
 
     void patch(DataLabelPtr label, CodeLocationLabel value)
     {
-        MacroAssembler::linkPointer(code(), label.m_label, value.executableAddress());
+        JmpDst target = applyOffset(label.m_label);
+        MacroAssembler::linkPointer(code(), target, value.executableAddress());
     }
 
@@ -128 +141 @@
         ASSERT(call.isFlagSet(Call::Linkable));
         ASSERT(!call.isFlagSet(Call::Near));
-        return CodeLocationCall(MacroAssembler::getLinkerAddress(code(), call.m_jmp));
+        return CodeLocationCall(MacroAssembler::getLinkerAddress(code(), applyOffset(call.m_jmp)));
     }
 
@@ -135 +148 @@
         ASSERT(call.isFlagSet(Call::Linkable));
         ASSERT(call.isFlagSet(Call::Near));
-        return CodeLocationNearCall(MacroAssembler::getLinkerAddress(code(), call.m_jmp));
+        return CodeLocationNearCall(MacroAssembler::getLinkerAddress(code(), applyOffset(call.m_jmp)));
     }
 
     CodeLocationLabel locationOf(Label label)
     {
-        return CodeLocationLabel(MacroAssembler::getLinkerAddress(code(), label.m_label));
+        return CodeLocationLabel(MacroAssembler::getLinkerAddress(code(), applyOffset(label.m_label)));
     }
 
     CodeLocationDataLabelPtr locationOf(DataLabelPtr label)
     {
-        return CodeLocationDataLabelPtr(MacroAssembler::getLinkerAddress(code(), label.m_label));
+        return CodeLocationDataLabelPtr(MacroAssembler::getLinkerAddress(code(), applyOffset(label.m_label)));
     }
 
     CodeLocationDataLabel32 locationOf(DataLabel32 label)
     {
-        return CodeLocationDataLabel32(MacroAssembler::getLinkerAddress(code(), label.m_label));
+        return CodeLocationDataLabel32(MacroAssembler::getLinkerAddress(code(), applyOffset(label.m_label)));
     }
 
@@ -157 +170 @@
     unsigned returnAddressOffset(Call call)
     {
+        call.m_jmp = applyOffset(call.m_jmp);
         return MacroAssembler::getLinkerCallReturnOffset(call);
     }
@@ -170 +184 @@
         return CodeRef(m_code, m_executablePool, m_size);
     }
+
     CodeLocationLabel finalizeCodeAddendum()
     {
@@ -177 +192 @@
     }
 
+    CodePtr trampolineAt(Label label)
+    {
+        return CodePtr(MacroAssembler::AssemblerType_T::getRelocatedAddress(code(), applyOffset(label.m_label)));
+    }
+
 private:
+    template <typename T> T applyOffset(T src)
+    {
+#if ENABLE(BRANCH_COMPACTION)
+        src.m_offset -= m_assembler->executableOffsetFor(src.m_offset);
+#endif
+        return src;
+    }
+    
     // Keep this private! - the underlying code should only be obtained externally via 
     // finalizeCode() or finalizeCodeAddendum().
@@ -185 +213 @@
     }
 
+    void linkCode(void* linkOffset)
+    {
+        UNUSED_PARAM(linkOffset);
+        ASSERT(!m_code);
+#if !ENABLE(BRANCH_COMPACTION)
+        m_code = m_assembler->m_assembler.executableCopy(m_executablePool.get());
+        m_size = m_assembler->size();
+#else
+        size_t initialSize = m_assembler->size();
+        m_code = (uint8_t*)m_executablePool->alloc(initialSize);
+        if (!m_code)
+            return;
+        ExecutableAllocator::makeWritable(m_code, m_assembler->size());
+        uint8_t* inData = (uint8_t*)m_assembler->unlinkedCode();
+        uint8_t* outData = reinterpret_cast<uint8_t*>(m_code);
+        const uint8_t* linkBase = linkOffset ? reinterpret_cast<uint8_t*>(linkOffset) : outData;
+        int readPtr = 0;
+        int writePtr = 0;
+        Vector<LinkRecord>& jumpsToLink = m_assembler->jumpsToLink();
+        unsigned jumpCount = jumpsToLink.size();
+        for (unsigned i = 0; i < jumpCount; ++i) {
+            int offset = readPtr - writePtr;
+            ASSERT(!(offset & 1));
+            
+            // Copy the instructions from the last jump to the current one.
+            size_t regionSize = jumpsToLink[i].from() - readPtr;
+            memcpy(outData + writePtr, inData + readPtr, regionSize);
+            m_assembler->recordLinkOffsets(readPtr, jumpsToLink[i].from(), offset);
+            readPtr += regionSize;
+            writePtr += regionSize;
+            
+            // Calculate absolute address of the jump target, in the case of backwards
+            // branches we need to be precise, forward branches we are pessimistic
+            const uint8_t* target;
+            if (jumpsToLink[i].to() >= jumpsToLink[i].from())
+                target = linkBase + jumpsToLink[i].to() - offset; // Compensate for what we have collapsed so far
+            else
+                target = linkBase + jumpsToLink[i].to() - m_assembler->executableOffsetFor(jumpsToLink[i].to());
+            
+            JumpLinkType jumpLinkType = m_assembler->computeJumpType(jumpsToLink[i], linkBase + writePtr, target);
+
+            // Step back in the write stream
+            int32_t delta = m_assembler->jumpSizeDelta(jumpLinkType);
+            if (delta) {
+                writePtr -= delta;
+                m_assembler->recordLinkOffsets(jumpsToLink[i].from() - delta, readPtr, readPtr - writePtr);
+            }
+            jumpsToLink[i].setFrom(writePtr);
+        }
+        // Copy everything after the last jump
+        memcpy(outData + writePtr, inData + readPtr, m_assembler->size() - readPtr);
+        m_assembler->recordLinkOffsets(readPtr, m_assembler->size(), readPtr - writePtr);
+        
+        // Actually link everything (don't link if we've be given a linkoffset as it's a
+        // waste of time: linkOffset is used for recompiling to get exception info)
+        if (!linkOffset) {
+            for (unsigned i = 0; i < jumpCount; ++i) {
+                uint8_t* location = outData + jumpsToLink[i].from();
+                uint8_t* target = outData + jumpsToLink[i].to() - m_assembler->executableOffsetFor(jumpsToLink[i].to());
+                m_assembler->link(jumpsToLink[i], location, target);
+            }
+        }
+
+        jumpsToLink.clear();
+        m_size = writePtr + m_assembler->size() - readPtr;
+        m_executablePool->returnLastBytes(initialSize - m_size);
+#endif
+    }
+
     void performFinalization()
     {
@@ -197 +294 @@
 
     RefPtr<ExecutablePool> m_executablePool;
+    size_t m_size;
     void* m_code;
-    size_t m_size;
+    MacroAssembler* m_assembler;
 #ifndef NDEBUG
     LinkBufferState m_state;