Ignore:
Timestamp:
Aug 20, 2017, 9:26:40 PM (8 years ago)
Author:
[email protected]
Message:

Enhance MacroAssembler::probe() to allow the probe function to resize the stack frame and alter stack data in one pass.
​https://bugs.webkit.org/show_bug.cgi?id=175688
<rdar://problem/33436870>

Reviewed by JF Bastien.

With this patch, the clients of the MacroAssembler::probe() can now change
stack values without having to worry about whether there is enough room in the
current stack frame for it or not. This is done using the Probe::Context's stack
member like so:

jit.probe([] (Probe::Context& context) {

auto cpu = context.cpu;
auto stack = context.stack();
uintptr_t* currentSP = cpu.sp<uintptr_t*>();

Get a value at the current stack pointer location.
auto value = stack.get<uintptr_t>(currentSP);

Set a value above the current stack pointer (within current frame).
stack.set<uintptr_t>(currentSP + 10, value);

Set a value below the current stack pointer (out of current frame).
stack.set<uintptr_t>(currentSP - 10, value);

Set the new stack pointer.
cpu.sp() = currentSP - 20;

});

What happens behind the scene:

  1. the generated JIT probe code will now call Probe::executeProbe(), and Probe::executeProbe() will in turn call the client's probe function.

Probe::executeProbe() receives the Probe::State on the machine stack passed
to it by the probe trampoline. Probe::executeProbe() will instantiate a
Probe::Context to be passed to the client's probe function. The client will
no longer see the Probe::State directly.

  1. The Probe::Context comes with a Probe::Stack which serves as a manager of stack pages. Currently, each page is 1K in size. Probe::Context::stack() returns a reference to an instance of Probe::Stack.
  1. Invoking get() of set() on Probe::Stack with an address will lead to the following:
  1. the address will be decoded to a baseAddress that points to the 1K page that contains that address.
  1. the Probe::Stack will check if it already has a cached 1K page for that baseAddress. If so, go to step (f). Else, continue with step (c).
  1. the Probe::Stack will malloc a 1K mirror page, and memcpy the 1K stack page for that specified baseAddress to this mirror page.
  1. the mirror page will be added to the ProbeStack's m_pages HashMap, keyed on the baseAddress.
  1. the ProbeStack will also cache the last baseAddress and its corresponding mirror page in use. With memory accesses tending to be localized, this will save us from having to look up the page in the HashMap.
  1. get() will map the requested address to a physical address in the mirror page, and return the value at that location.
  1. set() will map the requested address to a physical address in the mirror page, and set the value at that location in the mirror page.

set() will also set a dirty bit corresponding to the "cache line" that
was modified in the mirror page.

  1. When the client's probe function returns, Probe::executeProbe() will check if there are stack changes that need to be applied. If stack changes are needed:
  1. Probe::executeProbe() will adjust the stack pointer to ensure enough stack space is available to flush the dirty stack pages. It will also register a flushStackDirtyPages callback function in the Probe::State. Thereafter, Probe::executeProbe() returns to the probe trampoline.
  1. the probe trampoline adjusts the stack pointer, moves the Probe::State to a safe place if needed, and then calls the flushStackDirtyPages callback if needed.
  1. the flushStackDirtyPages() callback iterates the Probe::Stack's m_pages HashMap and flush all dirty "cache lines" to the machine stack. Thereafter, flushStackDirtyPages() returns to the probe trampoline.
  1. lastly, the probe trampoline will restore all register values and return to the pc set in the Probe::State.

To make this patch work, I also had to do the following work:

  1. Refactor MacroAssembler::CPUState into Probe::CPUState. Mainly, this means moving the code over to ProbeContext.h. I also added some convenience accessor methods for spr registers.

Moved Probe::Context over to its own file ProbeContext.h/cpp.

  1. Fix all probe trampolines to pass the address of Probe::executeProbe in addition to the client's probe function and arg.

I also took this opportunity to optimize the generated JIT probe code to
minimize the amount of memory stores needed.

  1. Simplified the ARM64 probe trampoline. The ARM64 probe only supports changing either lr or pc (or neither), but not both at in the same probe invocation. The ARM64 probe trampoline used to have to check for this invariant in the assembly trampoline code. With the introduction of Probe::executeProbe(), we can now do it there and simplify the trampoline.
  1. Fix a bug in the old ARM64 probe trampoline for the case where the client changes lr. That code path never worked before, but has now been fixed.
  1. Removed trustedImm32FromPtr() helper functions in MacroAssemblerARM and MacroAssemblerARMv7.

We can now use move() with TrustedImmPtr, and it does the same thing but in a
more generic way.

  1. ARMv7's move() emitter may encode a T1 move instruction, which happens to have

the same semantics as movs (according to the Thumb spec). This means these
instructions may trash the APSR flags before we have a chance to preserve them.

This patch changes MacroAssemblerARMv7's probe() to preserve the APSR register
early on. This entails adding support for the mrs instruction in the
ARMv7Assembler.

  1. Change testmasm's testProbeModifiesStackValues() to now modify stack values

the easy way.

Also fixed testmasm tests which check flag registers to only compare the
portions that are modifiable by the client i.e. some masking is applied.

This patch has passed the testmasm tests on x86, x86_64, arm64, and armv7.

  • CMakeLists.txt:
  • JavaScriptCore.xcodeproj/project.pbxproj:
  • assembler/ARMv7Assembler.h:

(JSC::ARMv7Assembler::mrs):

  • assembler/AbstractMacroAssembler.h:
  • assembler/MacroAssembler.cpp:

(JSC::stdFunctionCallback):
(JSC::MacroAssembler::probe):

  • assembler/MacroAssembler.h:

(JSC::MacroAssembler::CPUState::gprName): Deleted.
(JSC::MacroAssembler::CPUState::sprName): Deleted.
(JSC::MacroAssembler::CPUState::fprName): Deleted.
(JSC::MacroAssembler::CPUState::gpr): Deleted.
(JSC::MacroAssembler::CPUState::spr): Deleted.
(JSC::MacroAssembler::CPUState::fpr): Deleted.
(JSC:: const): Deleted.
(JSC::MacroAssembler::CPUState::fpr const): Deleted.
(JSC::MacroAssembler::CPUState::pc): Deleted.
(JSC::MacroAssembler::CPUState::fp): Deleted.
(JSC::MacroAssembler::CPUState::sp): Deleted.
(JSC::MacroAssembler::CPUState::pc const): Deleted.
(JSC::MacroAssembler::CPUState::fp const): Deleted.
(JSC::MacroAssembler::CPUState::sp const): Deleted.
(JSC::Probe::State::gpr): Deleted.
(JSC::Probe::State::spr): Deleted.
(JSC::Probe::State::fpr): Deleted.
(JSC::Probe::State::gprName): Deleted.
(JSC::Probe::State::sprName): Deleted.
(JSC::Probe::State::fprName): Deleted.
(JSC::Probe::State::pc): Deleted.
(JSC::Probe::State::fp): Deleted.
(JSC::Probe::State::sp): Deleted.

  • assembler/MacroAssemblerARM.cpp:

(JSC::MacroAssembler::probe):

  • assembler/MacroAssemblerARM.h:

(JSC::MacroAssemblerARM::trustedImm32FromPtr): Deleted.

  • assembler/MacroAssemblerARM64.cpp:

(JSC::MacroAssembler::probe):
(JSC::arm64ProbeError): Deleted.

  • assembler/MacroAssemblerARMv7.cpp:

(JSC::MacroAssembler::probe):

  • assembler/MacroAssemblerARMv7.h:

(JSC::MacroAssemblerARMv7::armV7Condition):
(JSC::MacroAssemblerARMv7::trustedImm32FromPtr): Deleted.

  • assembler/MacroAssemblerPrinter.cpp:

(JSC::Printer::printCallback):

  • assembler/MacroAssemblerPrinter.h:
  • assembler/MacroAssemblerX86Common.cpp:

(JSC::ctiMasmProbeTrampoline):
(JSC::MacroAssembler::probe):

  • assembler/Printer.h:

(JSC::Printer::Context::Context):

  • assembler/ProbeContext.cpp: Added.

(JSC::Probe::executeProbe):
(JSC::Probe::handleProbeStackInitialization):
(JSC::Probe::probeStateForContext):

  • assembler/ProbeContext.h: Added.

(JSC::Probe::CPUState::gprName):
(JSC::Probe::CPUState::sprName):
(JSC::Probe::CPUState::fprName):
(JSC::Probe::CPUState::gpr):
(JSC::Probe::CPUState::spr):
(JSC::Probe::CPUState::fpr):
(JSC::Probe:: const):
(JSC::Probe::CPUState::fpr const):
(JSC::Probe::CPUState::pc):
(JSC::Probe::CPUState::fp):
(JSC::Probe::CPUState::sp):
(JSC::Probe::CPUState::pc const):
(JSC::Probe::CPUState::fp const):
(JSC::Probe::CPUState::sp const):
(JSC::Probe::Context::Context):
(JSC::Probe::Context::gpr):
(JSC::Probe::Context::spr):
(JSC::Probe::Context::fpr):
(JSC::Probe::Context::gprName):
(JSC::Probe::Context::sprName):
(JSC::Probe::Context::fprName):
(JSC::Probe::Context::pc):
(JSC::Probe::Context::fp):
(JSC::Probe::Context::sp):
(JSC::Probe::Context::stack):
(JSC::Probe::Context::hasWritesToFlush):
(JSC::Probe::Context::releaseStack):

  • assembler/ProbeStack.cpp: Added.

(JSC::Probe::Page::Page):
(JSC::Probe::Page::flushWrites):
(JSC::Probe::Stack::Stack):
(JSC::Probe::Stack::hasWritesToFlush):
(JSC::Probe::Stack::flushWrites):
(JSC::Probe::Stack::ensurePageFor):

  • assembler/ProbeStack.h: Added.

(JSC::Probe::Page::baseAddressFor):
(JSC::Probe::Page::chunkAddressFor):
(JSC::Probe::Page::baseAddress):
(JSC::Probe::Page::get):
(JSC::Probe::Page::set):
(JSC::Probe::Page::hasWritesToFlush const):
(JSC::Probe::Page::flushWritesIfNeeded):
(JSC::Probe::Page::dirtyBitFor):
(JSC::Probe::Page::physicalAddressFor):
(JSC::Probe::Stack::Stack):
(JSC::Probe::Stack::lowWatermark):
(JSC::Probe::Stack::get):
(JSC::Probe::Stack::set):
(JSC::Probe::Stack::newStackPointer const):
(JSC::Probe::Stack::setNewStackPointer):
(JSC::Probe::Stack::isValid):
(JSC::Probe::Stack::pageFor):

  • assembler/testmasm.cpp:

(JSC::testProbeReadsArgumentRegisters):
(JSC::testProbeWritesArgumentRegisters):
(JSC::testProbePreservesGPRS):
(JSC::testProbeModifiesStackPointer):
(JSC::testProbeModifiesStackPointerToInsideProbeStateOnStack):
(JSC::testProbeModifiesStackPointerToNBytesBelowSP):
(JSC::testProbeModifiesProgramCounter):
(JSC::testProbeModifiesStackValues):
(JSC::run):
(): Deleted.
(JSC::fillStack): Deleted.
(JSC::testProbeModifiesStackWithCallback): Deleted.

Location:
trunk/Source/JavaScriptCore/assembler
Files:
4 added
14 edited

Legend:

Unmodified
Added
Removed

  • trunk/Source/JavaScriptCore/assembler/ARMv7Assembler.h

    r220871 r220958  
    736736        OP_UDIV_T1      = 0xFBB0,
    737737#endif
     738        OP_MRS_T1       = 0xF3EF,
    738739    } OpcodeID1;
    739740
    … …  
    14341435    }
    14351436
     1437    ALWAYS_INLINE void mrs(RegisterID rd, SPRegisterID specReg)
     1438    {
     1439        ASSERT(specReg == ARMRegisters::apsr);
     1440        ASSERT(!BadReg(rd));
     1441        unsigned short specialRegisterBit = (specReg == ARMRegisters::apsr) ? 0 : (1 << 4);
     1442        OpcodeID1 mrsOp = static_cast<OpcodeID1>(OP_MRS_T1 | specialRegisterBit);
     1443        m_formatter.twoWordOp16FourFours(mrsOp, FourFours(0x8, rd, 0, 0));
     1444    }
     1445
    14361446    ALWAYS_INLINE void neg(RegisterID rd, RegisterID rm)
    14371447    {

  • trunk/Source/JavaScriptCore/assembler/AbstractMacroAssembler.h

    r220921 r220958  
    4444#if ENABLE(ASSEMBLER)
    4545
    46 #if ENABLE(MASM_PROBE)
    47 namespace Probe {
    48 struct State;
    49 typedef void (*Function)(struct State*);
    50 } // namespace Probe
    51 #endif
    52    
    5346class AllowMacroScratchRegisterUsage;
    5447class DisallowMacroScratchRegisterUsage;

  • trunk/Source/JavaScriptCore/assembler/MacroAssembler.cpp

    r220921 r220958  
    2929#if ENABLE(ASSEMBLER)
    3030
     31#include "ProbeContext.h"
    3132#include <wtf/PrintStream.h>
    3233
    … …  
    3637
    3738#if ENABLE(MASM_PROBE)
    38 static void stdFunctionCallback(Probe::State* state)
     39static void stdFunctionCallback(Probe::Context& context)
    3940{
    40     auto func = static_cast<const std::function<void(Probe::State*)>*>(state->arg);
    41     (*func)(state);
     41    auto func = static_cast<const std::function<void(Probe::Context&)>*>(context.arg);
     42    (*func)(context);
    4243}
    4344   
    44 void MacroAssembler::probe(std::function<void(Probe::State*)> func)
     45void MacroAssembler::probe(std::function<void(Probe::Context&)> func)
    4546{
    46     probe(stdFunctionCallback, new std::function<void(Probe::State*)>(func));
     47    probe(stdFunctionCallback, new std::function<void(Probe::Context&)>(func));
    4748}
    4849#endif // ENABLE(MASM_PROBE)

  • trunk/Source/JavaScriptCore/assembler/MacroAssembler.h

    r220921 r220958  
    6464namespace JSC {
    6565
     66#if ENABLE(MASM_PROBE)
     67namespace Probe {
     68
     69class Context;
     70typedef void (*Function)(Context&);
     71
     72} // namespace Probe
     73#endif // ENABLE(MASM_PROBE)
     74
    6675namespace Printer {
    6776
    … …  
    6978typedef Vector<PrintRecord> PrintRecordList;
    7079
    71 }
     80} // namespace Printer
    7281
    7382class MacroAssembler : public MacroAssemblerBase {
    … …  
    18261835
    18271836#if ENABLE(MASM_PROBE)
    1828     struct CPUState;
    1829 
    18301837    // This function emits code to preserve the CPUState (e.g. registers),
    18311838    // call a user supplied probe function, and restore the CPUState before
    … …  
    18791886    void probe(Probe::Function, void* arg);
    18801887
    1881     JS_EXPORT_PRIVATE void probe(std::function<void(Probe::State*)>);
     1888    JS_EXPORT_PRIVATE void probe(std::function<void(Probe::Context&)>);
    18821889
    18831890    // Let's you print from your JIT generated code.
    … …  
    18891896#endif // ENABLE(MASM_PROBE)
    18901897};
    1891 
    1892 #if ENABLE(MASM_PROBE)
    1893 struct MacroAssembler::CPUState {
    1894     static inline const char* gprName(RegisterID id) { return MacroAssembler::gprName(id); }
    1895     static inline const char* sprName(SPRegisterID id) { return MacroAssembler::sprName(id); }
    1896     static inline const char* fprName(FPRegisterID id) { return MacroAssembler::fprName(id); }
    1897     inline uintptr_t& gpr(RegisterID);
    1898     inline uintptr_t& spr(SPRegisterID);
    1899     inline double& fpr(FPRegisterID);
    1900 
    1901     template<typename T, typename std::enable_if<std::is_integral<T>::value>::type* = nullptr>
    1902     T gpr(RegisterID) const;
    1903     template<typename T, typename std::enable_if<std::is_pointer<T>::value>::type* = nullptr>
    1904     T gpr(RegisterID) const;
    1905     template<typename T> T fpr(FPRegisterID) const;
    1906 
    1907     void*& pc();
    1908     void*& fp();
    1909     void*& sp();
    1910     template<typename T> T pc() const;
    1911     template<typename T> T fp() const;
    1912     template<typename T> T sp() const;
    1913 
    1914     uintptr_t gprs[MacroAssembler::numberOfRegisters()];
    1915     uintptr_t sprs[MacroAssembler::numberOfSPRegisters()];
    1916     double fprs[MacroAssembler::numberOfFPRegisters()];
    1917 };
    1918 
    1919 inline uintptr_t& MacroAssembler::CPUState::gpr(RegisterID id)
    1920 {
    1921     ASSERT(id >= MacroAssembler::firstRegister() && id <= MacroAssembler::lastRegister());
    1922     return gprs[id];
    1923 }
    1924 
    1925 inline uintptr_t& MacroAssembler::CPUState::spr(SPRegisterID id)
    1926 {
    1927     ASSERT(id >= MacroAssembler::firstSPRegister() && id <= MacroAssembler::lastSPRegister());
    1928     return sprs[id];
    1929 }
    1930 
    1931 inline double& MacroAssembler::CPUState::fpr(FPRegisterID id)
    1932 {
    1933     ASSERT(id >= MacroAssembler::firstFPRegister() && id <= MacroAssembler::lastFPRegister());
    1934     return fprs[id];
    1935 }
    1936 
    1937 template<typename T, typename std::enable_if<std::is_integral<T>::value>::type*>
    1938 T MacroAssembler::CPUState::gpr(RegisterID id) const
    1939 {
    1940     CPUState* cpu = const_cast<CPUState*>(this);
    1941     return static_cast<T>(cpu->gpr(id));
    1942 }
    1943 
    1944 template<typename T, typename std::enable_if<std::is_pointer<T>::value>::type*>
    1945 T MacroAssembler::CPUState::gpr(RegisterID id) const
    1946 {
    1947     CPUState* cpu = const_cast<CPUState*>(this);
    1948     return reinterpret_cast<T>(cpu->gpr(id));
    1949 }
    1950 
    1951 template<typename T>
    1952 T MacroAssembler::CPUState::fpr(FPRegisterID id) const
    1953 {
    1954     CPUState* cpu = const_cast<CPUState*>(this);
    1955     return bitwise_cast<T>(cpu->fpr(id));
    1956 }
    1957 
    1958 inline void*& MacroAssembler::CPUState::pc()
    1959 {
    1960 #if CPU(X86) || CPU(X86_64)
    1961     return *reinterpret_cast<void**>(&spr(X86Registers::eip));
    1962 #elif CPU(ARM64)
    1963     return *reinterpret_cast<void**>(&spr(ARM64Registers::pc));
    1964 #elif CPU(ARM_THUMB2) || CPU(ARM_TRADITIONAL)
    1965     return *reinterpret_cast<void**>(&gpr(ARMRegisters::pc));
    1966 #elif CPU(MIPS)
    1967     RELEASE_ASSERT_NOT_REACHED();
    1968 #else
    1969 #error "Unsupported CPU"
    1970 #endif
    1971 }
    1972 
    1973 inline void*& MacroAssembler::CPUState::fp()
    1974 {
    1975 #if CPU(X86) || CPU(X86_64)
    1976     return *reinterpret_cast<void**>(&gpr(X86Registers::ebp));
    1977 #elif CPU(ARM64)
    1978     return *reinterpret_cast<void**>(&gpr(ARM64Registers::fp));
    1979 #elif CPU(ARM_THUMB2) || CPU(ARM_TRADITIONAL)
    1980     return *reinterpret_cast<void**>(&gpr(ARMRegisters::fp));
    1981 #elif CPU(MIPS)
    1982     return *reinterpret_cast<void**>(&gpr(MIPSRegisters::fp));
    1983 #else
    1984 #error "Unsupported CPU"
    1985 #endif
    1986 }
    1987 
    1988 inline void*& MacroAssembler::CPUState::sp()
    1989 {
    1990 #if CPU(X86) || CPU(X86_64)
    1991     return *reinterpret_cast<void**>(&gpr(X86Registers::esp));
    1992 #elif CPU(ARM64)
    1993     return *reinterpret_cast<void**>(&gpr(ARM64Registers::sp));
    1994 #elif CPU(ARM_THUMB2) || CPU(ARM_TRADITIONAL)
    1995     return *reinterpret_cast<void**>(&gpr(ARMRegisters::sp));
    1996 #elif CPU(MIPS)
    1997     return *reinterpret_cast<void**>(&gpr(MIPSRegisters::sp));
    1998 #else
    1999 #error "Unsupported CPU"
    2000 #endif
    2001 }
    2002 
    2003 template<typename T>
    2004 T MacroAssembler::CPUState::pc() const
    2005 {
    2006     CPUState* cpu = const_cast<CPUState*>(this);
    2007     return reinterpret_cast<T>(cpu->pc());
    2008 }
    2009 
    2010 template<typename T>
    2011 T MacroAssembler::CPUState::fp() const
    2012 {
    2013     CPUState* cpu = const_cast<CPUState*>(this);
    2014     return reinterpret_cast<T>(cpu->fp());
    2015 }
    2016 
    2017 template<typename T>
    2018 T MacroAssembler::CPUState::sp() const
    2019 {
    2020     CPUState* cpu = const_cast<CPUState*>(this);
    2021     return reinterpret_cast<T>(cpu->sp());
    2022 }
    2023 
    2024 namespace Probe {
    2025 
    2026 struct State {
    2027     using CPUState = MacroAssembler::CPUState;
    2028     using RegisterID = MacroAssembler::RegisterID;
    2029     using SPRegisterID = MacroAssembler::SPRegisterID;
    2030     using FPRegisterID = MacroAssembler::FPRegisterID;
    2031 
    2032     Function probeFunction;
    2033     void* arg;
    2034     Function initializeStackFunction;
    2035     void* initializeStackArg;
    2036     CPUState cpu;
    2037 
    2038     // Convenience methods:
    2039     uintptr_t& gpr(RegisterID id) { return cpu.gpr(id); }
    2040     uintptr_t& spr(SPRegisterID id) { return cpu.spr(id); }
    2041     double& fpr(FPRegisterID id) { return cpu.fpr(id); }
    2042     const char* gprName(RegisterID id) { return cpu.gprName(id); }
    2043     const char* sprName(SPRegisterID id) { return cpu.sprName(id); }
    2044     const char* fprName(FPRegisterID id) { return cpu.fprName(id); }
    2045 
    2046     void*& pc() { return cpu.pc(); }
    2047     void*& fp() { return cpu.fp(); }
    2048     void*& sp() { return cpu.sp(); }
    2049 
    2050     template<typename T> T pc() { return cpu.pc<T>(); }
    2051     template<typename T> T fp() { return cpu.fp<T>(); }
    2052     template<typename T> T sp() { return cpu.sp<T>(); }
    2053 };
    2054 
    2055 } // namespace Probe
    2056 
    2057 #endif // ENABLE(MASM_PROBE)
    20581898
    20591899} // namespace JSC

  • trunk/Source/JavaScriptCore/assembler/MacroAssemblerARM.cpp

    r220921 r220958  
    3131#include "MacroAssembler.h"
    3232
     33#include "ProbeContext.h"
    3334#include <wtf/InlineASM.h>
    3435
    … …  
    101102extern "C" void ctiMasmProbeTrampoline();
    102103
     104using namespace ARMRegisters;
     105
    103106#if COMPILER(GCC_OR_CLANG)
    104107
    … …  
    251254#undef PROBE_OFFSETOF
    252255
     256struct IncomingRecord {
     257    uintptr_t lr;
     258    uintptr_t ip;
     259    uintptr_t r6;
     260    uintptr_t r0;
     261    uintptr_t r1;
     262    uintptr_t r2;
     263};
     264
     265#define IN_LR_OFFSET (0 * PTR_SIZE)
     266#define IN_IP_OFFSET (1 * PTR_SIZE)
     267#define IN_R6_OFFSET (2 * PTR_SIZE)
     268#define IN_R0_OFFSET (3 * PTR_SIZE)
     269#define IN_R1_OFFSET (4 * PTR_SIZE)
     270#define IN_R2_OFFSET (5 * PTR_SIZE)
     271#define IN_SIZE      (6 * PTR_SIZE)
     272
     273static_assert(IN_LR_OFFSET == offsetof(IncomingRecord, lr), "IN_LR_OFFSET is incorrect");
     274static_assert(IN_IP_OFFSET == offsetof(IncomingRecord, ip), "IN_IP_OFFSET is incorrect");
     275static_assert(IN_R6_OFFSET == offsetof(IncomingRecord, r6), "IN_R6_OFFSET is incorrect");
     276static_assert(IN_R0_OFFSET == offsetof(IncomingRecord, r0), "IN_R0_OFFSET is incorrect");
     277static_assert(IN_R1_OFFSET == offsetof(IncomingRecord, r1), "IN_R1_OFFSET is incorrect");
     278static_assert(IN_R2_OFFSET == offsetof(IncomingRecord, r2), "IN_R2_OFFSET is incorrect");
     279static_assert(IN_SIZE == sizeof(IncomingRecord), "IN_SIZE is incorrect");
     280
    253281asm (
    254282    ".text" "\n"
    … …  
    258286    SYMBOL_STRING(ctiMasmProbeTrampoline) ":" "\n"
    259287
    260     // MacroAssemblerARM::probe() has already generated code to store some values.
    261     // The top of stack now looks like this:
    262     //     esp[0 * ptrSize]: probe handler function
    263     //     esp[1 * ptrSize]: probe arg
    264     //     esp[2 * ptrSize]: saved r3 / S0
    265     //     esp[3 * ptrSize]: saved ip
    266     //     esp[4 * ptrSize]: saved lr
    267     //     esp[5 * ptrSize]: saved sp
     288    // MacroAssemblerARMv7::probe() has already generated code to store some values.
     289    // The top of stack now contains the IncomingRecord.
     290    //
     291    // Incoming register values:
     292    //     r0: probe function
     293    //     r1: probe arg
     294    //     r2: Probe::executeProbe
     295    //     r6: scratch
     296    //     ip: scratch, was ctiMasmProbeTrampoline
     297    //     lr: return address
    268298
    269299    "mov       ip, sp" "\n"
    270     "mov       r3, sp" "\n"
    271     "sub       r3, r3, #" STRINGIZE_VALUE_OF(PROBE_SIZE + OUT_SIZE) "\n"
     300    "mov       r6, sp" "\n"
     301    "sub       r6, r6, #" STRINGIZE_VALUE_OF(PROBE_SIZE + OUT_SIZE) "\n"
    272302
    273303    // The ARM EABI specifies that the stack needs to be 16 byte aligned.
    274     "bic       r3, r3, #0xf" "\n"
    275     "mov       sp, r3" "\n" // Set the sp to protect the Probe::State from interrupts before we initialize it.
    276 
     304    "bic       r6, r6, #0xf" "\n"
     305    "mov       sp, r6" "\n" // Set the sp to protect the Probe::State from interrupts before we initialize it.
     306
     307    "str       r0, [sp, #" STRINGIZE_VALUE_OF(PROBE_PROBE_FUNCTION_OFFSET) "]" "\n"
     308    "str       r1, [sp, #" STRINGIZE_VALUE_OF(PROBE_ARG_OFFSET) "]" "\n"
    277309    "str       lr, [sp, #" STRINGIZE_VALUE_OF(PROBE_CPU_PC_OFFSET) "]" "\n"
    278     "add       lr, sp, #" STRINGIZE_VALUE_OF(PROBE_CPU_R0_OFFSET) "\n"
    279     "stmia     lr, { r0-r11 }" "\n"
     310
     311    "add       r0, ip, #" STRINGIZE_VALUE_OF(IN_SIZE) "\n"
     312    "str       r0, [sp, #" STRINGIZE_VALUE_OF(PROBE_CPU_SP_OFFSET) "]" "\n"
     313
     314    "add       lr, sp, #" STRINGIZE_VALUE_OF(PROBE_CPU_R3_OFFSET) "\n"
     315    "stmia     lr, { r3-r5 }" "\n"
     316    "add       lr, sp, #" STRINGIZE_VALUE_OF(PROBE_CPU_R7_OFFSET) "\n"
     317    "stmia     lr, { r7-r11 }" "\n"
    280318    "mrs       lr, APSR" "\n"
    281319    "str       lr, [sp, #" STRINGIZE_VALUE_OF(PROBE_CPU_APSR_OFFSET) "]" "\n"
    … …  
    283321    "str       lr, [sp, #" STRINGIZE_VALUE_OF(PROBE_CPU_FPSCR_OFFSET) "]" "\n"
    284322
    285     "ldr       lr, [ip, #0 * " STRINGIZE_VALUE_OF(PTR_SIZE) "]" "\n"
    286     "str       lr, [sp, #" STRINGIZE_VALUE_OF(PROBE_PROBE_FUNCTION_OFFSET) "]" "\n"
    287     "ldr       lr, [ip, #1 * " STRINGIZE_VALUE_OF(PTR_SIZE) "]" "\n"
    288     "str       lr, [sp, #" STRINGIZE_VALUE_OF(PROBE_ARG_OFFSET) "]" "\n"
    289     "ldr       lr, [ip, #2 * " STRINGIZE_VALUE_OF(PTR_SIZE) "]" "\n"
    290     "str       lr, [sp, #" STRINGIZE_VALUE_OF(PROBE_CPU_R3_OFFSET) "]" "\n"
    291     "ldr       lr, [ip, #3 * " STRINGIZE_VALUE_OF(PTR_SIZE) "]" "\n"
    292     "str       lr, [sp, #" STRINGIZE_VALUE_OF(PROBE_CPU_IP_OFFSET) "]" "\n"
    293     "ldr       lr, [ip, #4 * " STRINGIZE_VALUE_OF(PTR_SIZE) "]" "\n"
    294     "str       lr, [sp, #" STRINGIZE_VALUE_OF(PROBE_CPU_LR_OFFSET) "]" "\n"
    295     "ldr       lr, [ip, #5 * " STRINGIZE_VALUE_OF(PTR_SIZE) "]" "\n"
    296     "str       lr, [sp, #" STRINGIZE_VALUE_OF(PROBE_CPU_SP_OFFSET) "]" "\n"
    297 
    298     "ldr       lr, [sp, #" STRINGIZE_VALUE_OF(PROBE_CPU_PC_OFFSET) "]" "\n"
     323    "ldr       r4, [ip, #" STRINGIZE_VALUE_OF(IN_LR_OFFSET) "]" "\n"
     324    "ldr       r5, [ip, #" STRINGIZE_VALUE_OF(IN_IP_OFFSET) "]" "\n"
     325    "ldr       r6, [ip, #" STRINGIZE_VALUE_OF(IN_R6_OFFSET) "]" "\n"
     326    "ldr       r7, [ip, #" STRINGIZE_VALUE_OF(IN_R0_OFFSET) "]" "\n"
     327    "ldr       r8, [ip, #" STRINGIZE_VALUE_OF(IN_R1_OFFSET) "]" "\n"
     328    "ldr       r9, [ip, #" STRINGIZE_VALUE_OF(IN_R2_OFFSET) "]" "\n"
     329    "str       r4, [sp, #" STRINGIZE_VALUE_OF(PROBE_CPU_LR_OFFSET) "]" "\n"
     330    "str       r5, [sp, #" STRINGIZE_VALUE_OF(PROBE_CPU_IP_OFFSET) "]" "\n"
     331    "str       r6, [sp, #" STRINGIZE_VALUE_OF(PROBE_CPU_R6_OFFSET) "]" "\n"
     332    "str       r7, [sp, #" STRINGIZE_VALUE_OF(PROBE_CPU_R0_OFFSET) "]" "\n"
     333    "str       r8, [sp, #" STRINGIZE_VALUE_OF(PROBE_CPU_R1_OFFSET) "]" "\n"
     334    "str       r9, [sp, #" STRINGIZE_VALUE_OF(PROBE_CPU_R2_OFFSET) "]" "\n"
    299335
    300336    "add       ip, sp, #" STRINGIZE_VALUE_OF(PROBE_CPU_D0_OFFSET) "\n"
    … …  
    303339    "vstmia.64 ip!, { d16-d31 }" "\n"
    304340#endif
    305     "mov       fp, sp" "\n" // Save the Probe::State*.
    306 
    307     // Initialize Probe::State::initializeStackFunction to zero.
    308     "mov       r0, #0" "\n"
    309     "str       r0, [fp, #" STRINGIZE_VALUE_OF(PROBE_INIT_STACK_FUNCTION_OFFSET) "]" "\n"
    310 
    311     "ldr       ip, [sp, #" STRINGIZE_VALUE_OF(PROBE_PROBE_FUNCTION_OFFSET) "]" "\n"
     341
     342    // r5 is a callee saved register. We'll use it for preserving the Probe::State*.
     343    // https://stackoverflow.com/questions/261419/arm-to-c-calling-convention-registers-to-save#261496
     344    "mov       r5, sp" "\n"
     345
    312346    "mov       r0, sp" "\n" // the Probe::State* arg.
    313     "blx       ip" "\n"
     347    "blx       r2" "\n" // Call Probe::executeProbe.
    314348
    315349    // Make sure the Probe::State is entirely below the result stack pointer so
    316350    // that register values are still preserved when we call the initializeStack
    317351    // function.
    318     "ldr       r1, [fp, #" STRINGIZE_VALUE_OF(PROBE_CPU_SP_OFFSET) "]" "\n" // Result sp.
    319     "add       r2, fp, #" STRINGIZE_VALUE_OF(PROBE_SIZE + OUT_SIZE) "\n" // End of ProveContext + buffer.
     352    "ldr       r1, [r5, #" STRINGIZE_VALUE_OF(PROBE_CPU_SP_OFFSET) "]" "\n" // Result sp.
     353    "add       r2, r5, #" STRINGIZE_VALUE_OF(PROBE_SIZE + OUT_SIZE) "\n" // End of ProveContext + buffer.
    320354    "cmp       r1, r2" "\n"
    321355    "bge     " LOCAL_LABEL_STRING(ctiMasmProbeTrampolineProbeStateIsSafe) "\n"
    … …  
    329363    // Note: we have to copy from low address to higher address because we're moving the
    330364    // Probe::State to a lower address.
    331     "mov       r5, fp" "\n"
    332     "mov       r6, r1" "\n"
    333     "add       r7, fp, #" STRINGIZE_VALUE_OF(PROBE_SIZE) "\n"
     365    "add       r7, r5, #" STRINGIZE_VALUE_OF(PROBE_SIZE) "\n"
    334366
    335367    LOCAL_LABEL_STRING(ctiMasmProbeTrampolineCopyLoop) ":" "\n"
    336368    "ldr       r3, [r5], #4" "\n"
    337369    "ldr       r4, [r5], #4" "\n"
    338     "str       r3, [r6], #4" "\n"
    339     "str       r4, [r6], #4" "\n"
     370    "str       r3, [r1], #4" "\n"
     371    "str       r4, [r1], #4" "\n"
    340372    "cmp       r5, r7" "\n"
    341373    "blt     " LOCAL_LABEL_STRING(ctiMasmProbeTrampolineCopyLoop) "\n"
    342374
    343     "mov       fp, r1" "\n"
     375    "mov       r5, sp" "\n"
    344376
    345377    // Call initializeStackFunction if present.
    346378    LOCAL_LABEL_STRING(ctiMasmProbeTrampolineProbeStateIsSafe) ":" "\n"
    347     "ldr       r2, [fp, #" STRINGIZE_VALUE_OF(PROBE_INIT_STACK_FUNCTION_OFFSET) "]" "\n"
     379    "ldr       r2, [r5, #" STRINGIZE_VALUE_OF(PROBE_INIT_STACK_FUNCTION_OFFSET) "]" "\n"
    348380    "cmp       r2, #0" "\n"
    349381    "beq     " LOCAL_LABEL_STRING(ctiMasmProbeTrampolineRestoreRegisters) "\n"
    350382
    351     "mov       r0, fp" "\n" // Set the Probe::State* arg.
     383    "mov       r0, r5" "\n" // Set the Probe::State* arg.
    352384    "blx       r2" "\n" // Call the initializeStackFunction (loaded into r2 above).
    353385
    354386    LOCAL_LABEL_STRING(ctiMasmProbeTrampolineRestoreRegisters) ":" "\n"
    355387
    356     "mov       sp, fp" "\n"
     388    "mov       sp, r5" "\n" // Ensure that sp points to the Probe::State*.
    357389
    358390    // To enable probes to modify register state, we copy all registers
    … …  
    400432void MacroAssembler::probe(Probe::Function function, void* arg)
    401433{
    402     push(RegisterID::sp);
    403     push(RegisterID::lr);
    404     push(RegisterID::ip);
    405     push(RegisterID::S0);
    406     // The following uses RegisterID::S0. So, they must come after we push S0 above.
    407     push(trustedImm32FromPtr(arg));
    408     push(trustedImm32FromPtr(function));
    409 
    410     move(trustedImm32FromPtr(ctiMasmProbeTrampoline), RegisterID::S0);
    411     m_assembler.blx(RegisterID::S0);
    412 
     434    sub32(TrustedImm32(sizeof(IncomingRecord)), sp);
     435
     436    store32(lr, Address(sp, offsetof(IncomingRecord, lr)));
     437    store32(ip, Address(sp, offsetof(IncomingRecord, ip)));
     438    store32(r6, Address(sp, offsetof(IncomingRecord, r6))); // S0 == r6.
     439    store32(r0, Address(sp, offsetof(IncomingRecord, r0)));
     440    store32(r1, Address(sp, offsetof(IncomingRecord, r1)));
     441    store32(r2, Address(sp, offsetof(IncomingRecord, r2)));
     442    // The following uses RegisterID::S0. So, they must come after we preserve S0 above.
     443    move(TrustedImmPtr(reinterpret_cast<void*>(function)), r0);
     444    move(TrustedImmPtr(arg), r1);
     445    move(TrustedImmPtr(reinterpret_cast<void*>(Probe::executeProbe)), r2);
     446    move(TrustedImmPtr(reinterpret_cast<void*>(ctiMasmProbeTrampoline)), ip);
     447    m_assembler.blx(ip);
    413448}
    414449#endif // ENABLE(MASM_PROBE)

  • trunk/Source/JavaScriptCore/assembler/MacroAssemblerARM.h

    r220921 r220958  
    15871587    }
    15881588
    1589 #if ENABLE(MASM_PROBE)
    1590     inline TrustedImm32 trustedImm32FromPtr(void* ptr)
    1591     {
    1592         return TrustedImm32(TrustedImmPtr(ptr));
    1593     }
    1594 
    1595     inline TrustedImm32 trustedImm32FromPtr(Probe::Function function)
    1596     {
    1597         return TrustedImm32(TrustedImmPtr(reinterpret_cast<void*>(function)));
    1598     }
    1599 
    1600     inline TrustedImm32 trustedImm32FromPtr(void (*function)())
    1601     {
    1602         return TrustedImm32(TrustedImmPtr(reinterpret_cast<void*>(function)));
    1603     }
    1604 #endif // ENABLE(MASM_PROBE)
    1605 
    16061589private:
    16071590    friend class LinkBuffer;

  • trunk/Source/JavaScriptCore/assembler/MacroAssemblerARM64.cpp

    r220921 r220958  
    2929#include "MacroAssembler.h"
    3030
     31#include "ProbeContext.h"
    3132#include <wtf/InlineASM.h>
    3233
    … …  
    127128
    128129#define SAVED_PROBE_RETURN_PC_OFFSET        (PROBE_SIZE + (0 * PTR_SIZE))
    129 #define SAVED_PROBE_LR_OFFSET               (PROBE_SIZE + (1 * PTR_SIZE))
    130 #define SAVED_PROBE_ERROR_FUNCTION_OFFSET   (PROBE_SIZE + (2 * PTR_SIZE))
    131130#define PROBE_SIZE_PLUS_EXTRAS              (PROBE_SIZE + (3 * PTR_SIZE))
    132131
    … …  
    226225
    227226struct IncomingProbeRecord {
    228     uintptr_t probeHandlerFunction;
    229     uintptr_t probeArg;
     227    uintptr_t x24;
     228    uintptr_t x25;
    230229    uintptr_t x26;
    231230    uintptr_t x27;
    232     uintptr_t lr;
    233     uintptr_t sp;
    234     uintptr_t probeErrorFunction;
    235     uintptr_t unused; // Padding for alignment.
     231    uintptr_t x28;
     232    uintptr_t x30; // lr
    236233};
    237234
    238 #define IN_HANDLER_FUNCTION_OFFSET (0 * PTR_SIZE)
    239 #define IN_ARG_OFFSET              (1 * PTR_SIZE)
    240 #define IN_X26_OFFSET              (2 * PTR_SIZE)
    241 #define IN_X27_OFFSET              (3 * PTR_SIZE)
    242 #define IN_LR_OFFSET              (4 * PTR_SIZE)
    243 #define IN_SP_OFFSET              (5 * PTR_SIZE)
    244 #define IN_ERROR_FUNCTION_OFFSET   (6 * PTR_SIZE)
    245 
    246 static_assert(IN_HANDLER_FUNCTION_OFFSET == offsetof(IncomingProbeRecord, probeHandlerFunction), "IN_HANDLER_FUNCTION_OFFSET is incorrect");
    247 static_assert(IN_ARG_OFFSET == offsetof(IncomingProbeRecord, probeArg), "IN_ARG_OFFSET is incorrect");
     235#define IN_X24_OFFSET (0 * PTR_SIZE)
     236#define IN_X25_OFFSET (1 * PTR_SIZE)
     237#define IN_X26_OFFSET (2 * PTR_SIZE)
     238#define IN_X27_OFFSET (3 * PTR_SIZE)
     239#define IN_X28_OFFSET (4 * PTR_SIZE)
     240#define IN_X30_OFFSET (5 * PTR_SIZE)
     241#define IN_SIZE       (6 * PTR_SIZE)
     242
     243static_assert(IN_X24_OFFSET == offsetof(IncomingProbeRecord, x24), "IN_X24_OFFSET is incorrect");
     244static_assert(IN_X25_OFFSET == offsetof(IncomingProbeRecord, x25), "IN_X25_OFFSET is incorrect");
    248245static_assert(IN_X26_OFFSET == offsetof(IncomingProbeRecord, x26), "IN_X26_OFFSET is incorrect");
    249246static_assert(IN_X27_OFFSET == offsetof(IncomingProbeRecord, x27), "IN_X27_OFFSET is incorrect");
    250 static_assert(IN_LR_OFFSET == offsetof(IncomingProbeRecord, lr), "IN_LR_OFFSET is incorrect");
    251 static_assert(IN_SP_OFFSET == offsetof(IncomingProbeRecord, sp), "IN_SP_OFFSET is incorrect");
    252 static_assert(IN_ERROR_FUNCTION_OFFSET == offsetof(IncomingProbeRecord, probeErrorFunction), "IN_ERROR_FUNCTION_OFFSET is incorrect");
     247static_assert(IN_X28_OFFSET == offsetof(IncomingProbeRecord, x28), "IN_X22_OFFSET is incorrect");
     248static_assert(IN_X30_OFFSET == offsetof(IncomingProbeRecord, x30), "IN_X23_OFFSET is incorrect");
     249static_assert(IN_SIZE == sizeof(IncomingProbeRecord), "IN_SIZE is incorrect");
    253250static_assert(!(sizeof(IncomingProbeRecord) & 0xf), "IncomingProbeStack must be 16-byte aligned");
    254251
    … …  
    279276static_assert(!(sizeof(OutgoingProbeRecord) & 0xf), "OutgoingProbeStack must be 16-byte aligned");
    280277
    281 #define STATE_PC_NOT_CHANGED 0
    282 #define STATE_PC_CHANGED 1
    283 static_assert(STATE_PC_NOT_CHANGED != STATE_PC_CHANGED, "STATE_PC_NOT_CHANGED and STATE_PC_CHANGED should not be equal");
     278struct LRRestorationRecord {
     279    uintptr_t lr;
     280    uintptr_t unusedDummyToEnsureSizeIs16ByteAligned;
     281};
     282
     283#define LR_RESTORATION_LR_OFFSET (0 * PTR_SIZE)
     284#define LR_RESTORATION_SIZE      (2 * PTR_SIZE)
     285
     286static_assert(LR_RESTORATION_LR_OFFSET == offsetof(LRRestorationRecord, lr), "LR_RESTORATION_LR_OFFSET is incorrect");
     287static_assert(LR_RESTORATION_SIZE == sizeof(LRRestorationRecord), "LR_RESTORATION_SIZE is incorrect");
     288static_assert(!(sizeof(LRRestorationRecord) & 0xf), "LRRestorationRecord must be 16-byte aligned");
    284289
    285290// We use x29 and x30 instead of fp and lr because GCC's inline assembler does not recognize fp and lr.
    … …  
    294299    // MacroAssemblerARM64::probe() has already generated code to store some values in an
    295300    // IncomingProbeRecord. sp points to the IncomingProbeRecord.
     301    //
     302    // Incoming register values:
     303    //     x24: probe function
     304    //     x25: probe arg
     305    //     x26: scratch, was ctiMasmProbeTrampoline
     306    //     x27: scratch
     307    //     x28: Probe::executeProbe
     308    //     x30: return address
    296309
    297310    "mov       x26, sp" "\n"
    … …  
    301314    "bic       x27, x27, #0xf" "\n" // The ARM EABI specifies that the stack needs to be 16 byte aligned.
    302315    "mov       sp, x27" "\n" // Set the sp to protect the Probe::State from interrupts before we initialize it.
     316
     317    "stp       x24, x25, [sp, #" STRINGIZE_VALUE_OF(PROBE_PROBE_FUNCTION_OFFSET) "]" "\n" // Store the probe handler function and arg (preloaded into x24 and x25
    303318
    304319    "stp       x0, x1, [sp, #" STRINGIZE_VALUE_OF(PROBE_CPU_X0_OFFSET) "]" "\n"
    … …  
    310325    "stp       x8, x9, [sp, #" STRINGIZE_VALUE_OF(PROBE_CPU_X8_OFFSET) "]" "\n"
    311326
    312     "ldp       x2, x3, [x26, #" STRINGIZE_VALUE_OF(IN_HANDLER_FUNCTION_OFFSET) "]" "\n" // Preload probe handler function and probe arg.
    313     "ldp       x4, x5, [x26, #" STRINGIZE_VALUE_OF(IN_X26_OFFSET) "]" "\n" // Preload saved r26 and r27.
    314     "ldp       x6, x7, [x26, #" STRINGIZE_VALUE_OF(IN_LR_OFFSET) "]" "\n" // Preload saved lr and sp.
    315     "ldr       x8, [x26, #" STRINGIZE_VALUE_OF(IN_ERROR_FUNCTION_OFFSET) "]" "\n" // Preload probe error function.
     327    "ldp       x2, x3, [x26, #" STRINGIZE_VALUE_OF(IN_X24_OFFSET) "]" "\n" // Preload saved x24 and x25.
     328    "ldp       x4, x5, [x26, #" STRINGIZE_VALUE_OF(IN_X26_OFFSET) "]" "\n" // Preload saved x26 and x27.
     329    "ldp       x6, x7, [x26, #" STRINGIZE_VALUE_OF(IN_X28_OFFSET) "]" "\n" // Preload saved x28 and lr.
     330    "add       x26, x26, #" STRINGIZE_VALUE_OF(IN_SIZE) "\n" // Compute the sp before the probe.
    316331
    317332    "stp       x10, x11, [sp, #" STRINGIZE_VALUE_OF(PROBE_CPU_X10_OFFSET) "]" "\n"
    … …  
    322337    "stp       x20, x21, [sp, #" STRINGIZE_VALUE_OF(PROBE_CPU_X20_OFFSET) "]" "\n"
    323338    "stp       x22, x23, [sp, #" STRINGIZE_VALUE_OF(PROBE_CPU_X22_OFFSET) "]" "\n"
    324     "stp       x24, x25, [sp, #" STRINGIZE_VALUE_OF(PROBE_CPU_X24_OFFSET) "]" "\n"
     339    "stp       x2, x3, [sp, #" STRINGIZE_VALUE_OF(PROBE_CPU_X24_OFFSET) "]" "\n" // Store saved r24 and r25 (preloaded into x2 and x3 above).
    325340    "stp       x4, x5, [sp, #" STRINGIZE_VALUE_OF(PROBE_CPU_X26_OFFSET) "]" "\n" // Store saved r26 and r27 (preloaded into x4 and x5 above).
    326     "stp       x28, x29, [sp, #" STRINGIZE_VALUE_OF(PROBE_CPU_X28_OFFSET) "]" "\n"
    327     "stp       x6, x7, [sp, #" STRINGIZE_VALUE_OF(PROBE_CPU_LR_OFFSET) "]" "\n" // Save values lr and sp (preloaded into x6 and x7 above).
    328 
    329     "str       x6, [sp, #" STRINGIZE_VALUE_OF(SAVED_PROBE_LR_OFFSET) "]" "\n" // Save a duplicate copy of lr (in x6).
     341    "stp       x6, x29, [sp, #" STRINGIZE_VALUE_OF(PROBE_CPU_X28_OFFSET) "]" "\n"
     342    "stp       x7, x26, [sp, #" STRINGIZE_VALUE_OF(PROBE_CPU_LR_OFFSET) "]" "\n" // Save values lr and sp (original sp value computed into x26 above).
     343
    330344    "str       x30, [sp, #" STRINGIZE_VALUE_OF(SAVED_PROBE_RETURN_PC_OFFSET) "]" "\n" // Save a duplicate copy of return pc (in lr).
    331345
    … …  
    334348
    335349    "stp       x0, x1, [sp, #" STRINGIZE_VALUE_OF(PROBE_CPU_NZCV_OFFSET) "]" "\n" // Store nzcv and fpsr (preloaded into x0 and x1 above).
    336 
    337     "stp       x2, x3, [sp, #" STRINGIZE_VALUE_OF(PROBE_PROBE_FUNCTION_OFFSET) "]" "\n" // Store the probe handler function and arg (preloaded into x2 and x3 above).
    338     "str       x8, [sp, #" STRINGIZE_VALUE_OF(SAVED_PROBE_ERROR_FUNCTION_OFFSET) "]" "\n" // Store the probe handler function and arg (preloaded into x8 above).
    339350
    340351    "add       x9, sp, #" STRINGIZE_VALUE_OF(PROBE_CPU_Q0_OFFSET) "\n"
    … …  
    358369    "mov       x27, sp" "\n" // Save the Probe::State* in a callee saved register.
    359370
    360     // Initialize Probe::State::initializeStackFunction to zero.
    361     "str       xzr, [x27, #" STRINGIZE_VALUE_OF(PROBE_INIT_STACK_FUNCTION_OFFSET) "]" "\n"
    362 
    363371    // Note: we haven't changed the value of fp. Hence, it is still pointing to the frame of
    364372    // the caller of the probe (which is what we want in order to play nice with debuggers e.g. lldb).
    365373    "mov       x0, sp" "\n" // Set the Probe::State* arg.
    366     "blr       x2" "\n" // Call the probe handler function (loaded into x2 above).
     374    "blr       x28" "\n" // Call the probe handler.
    367375
    368376    // Make sure the Probe::State is entirely below the result stack pointer so
    … …  
    447455    // Remaining registers to restore are: fpsr, nzcv, x27, x28, fp, lr, sp, and pc.
    448456
    449     "mov       x30, #" STRINGIZE_VALUE_OF(STATE_PC_NOT_CHANGED) "\n"
    450 
    451457    // The only way to set the pc on ARM64 (from user space) is via an indirect branch
    452458    // or a ret, which means we'll need a free register to do so. For our purposes, lr
    … …  
    455461    // returns. So, the ARM64 probe implementation will allow the probe handler to
    456462    // either modify lr or pc, but not both in the same probe invocation. The probe
    457     // mechanism ensures that we never try to modify both lr and pc, else it will
    458     // fail with a RELEASE_ASSERT_NOT_REACHED in arm64ProbeError().
     463    // mechanism ensures that we never try to modify both lr and pc with a RELEASE_ASSERT
     464    // in Probe::executeProbe().
    459465
    460466    // Determine if the probe handler changed the pc.
     467    "ldr       x30, [sp, #" STRINGIZE_VALUE_OF(PROBE_CPU_SP_OFFSET) "]" "\n" // preload the target sp.
    461468    "ldr       x27, [sp, #" STRINGIZE_VALUE_OF(SAVED_PROBE_RETURN_PC_OFFSET) "]" "\n"
    462469    "ldr       x28, [sp, #" STRINGIZE_VALUE_OF(PROBE_CPU_PC_OFFSET) "]" "\n"
    463470    "add       x27, x27, #" STRINGIZE_VALUE_OF(2 * PTR_SIZE) "\n"
    464471    "cmp       x27, x28" "\n"
    465     "beq     " LOCAL_LABEL_STRING(ctiMasmProbeTrampolinePrepareOutgoingRecords) "\n"
    466 
    467     // pc was changed. Determine if the probe handler also changed lr.
    468     "ldr       x27, [sp, #" STRINGIZE_VALUE_OF(SAVED_PROBE_LR_OFFSET) "]" "\n"
    469     "ldr       x28, [sp, #" STRINGIZE_VALUE_OF(PROBE_CPU_LR_OFFSET) "]" "\n"
    470     "cmp       x27, x28" "\n"
    471     "bne     " LOCAL_LABEL_STRING(ctiMasmProbeTrampolineError) "\n"
    472 
    473     "mov       x30, #" STRINGIZE_VALUE_OF(STATE_PC_CHANGED) "\n"
    474 
    475     LOCAL_LABEL_STRING(ctiMasmProbeTrampolinePrepareOutgoingRecords) ":" "\n"
    476 
    477     "ldr       x29, [sp, #" STRINGIZE_VALUE_OF(SAVED_PROBE_RETURN_PC_OFFSET) "]" "\n" // Preload the probe return site pc.
    478 
    479     LOCAL_LABEL_STRING(ctiMasmProbeTrampolineFillOutgoingProbeRecords) ":" "\n"
    480 
    481     "cbnz       x30, " LOCAL_LABEL_STRING(ctiMasmProbeTrampolineEnd) "\n" // Skip lr restoration setup if state (in lr) == STATE_PC_CHANGED.
    482 
    483     // In order to restore lr, we need to do the restoration at the probe return site.
    484     // The probe return site expects sp to be pointing at an OutgoingProbeRecord such that
    485     // popping the OutgoingProbeRecord will yield the desired sp. The probe return site
    486     // also expects the lr value to be restored is stashed in the OutgoingProbeRecord.
    487     // We can make this happen by pushing 2 OutgoingProbeRecords instead of 1:
    488     // 1 for the probe return site, and 1 at ctiMasmProbeTrampolineEnd for returning from
    489     // this probe.
    490 
    491     // Fill in the OutgoingProbeStack for the probe return site.
    492     "ldr       x30, [sp, #" STRINGIZE_VALUE_OF(PROBE_CPU_SP_OFFSET) "]" "\n"
    493     "sub       x30, x30, #" STRINGIZE_VALUE_OF(OUT_SIZE) "\n"
    494 
     472    "bne     " LOCAL_LABEL_STRING(ctiMasmProbeTrampolineEnd) "\n"
     473
     474     // We didn't change the PC. So, let's prepare for setting a potentially new lr value.
     475
     476     // 1. Make room for the LRRestorationRecord. The probe site will pop this off later.
     477    "sub       x30, x30, #" STRINGIZE_VALUE_OF(LR_RESTORATION_SIZE) "\n"
     478     // 2. Store the lp value to restore at the probe return site.
    495479    "ldr       x27, [sp, #" STRINGIZE_VALUE_OF(PROBE_CPU_LR_OFFSET) "]" "\n"
    496     "str       x27, [x30, #" STRINGIZE_VALUE_OF(OUT_LR_OFFSET) "]" "\n"
    497 
    498     // Set up the sp and pc values so that ctiMasmProbeTrampolineEnd will return to the probe return site.
    499     "str       x30, [sp, #" STRINGIZE_VALUE_OF(PROBE_CPU_SP_OFFSET) "]" "\n"
    500     "str       x29, [sp, #" STRINGIZE_VALUE_OF(PROBE_CPU_PC_OFFSET) "]" "\n" // Store the probe return site pc (preloaded into fp above).
     480    "str       x27, [x30, #" STRINGIZE_VALUE_OF(LR_RESTORATION_LR_OFFSET) "]" "\n"
     481     // 3. Force the return ramp to return to the probe return site.
     482    "ldr       x27, [sp, #" STRINGIZE_VALUE_OF(SAVED_PROBE_RETURN_PC_OFFSET) "]" "\n"
     483    "str       x27, [sp, #" STRINGIZE_VALUE_OF(PROBE_CPU_PC_OFFSET) "]" "\n"
    501484
    502485    LOCAL_LABEL_STRING(ctiMasmProbeTrampolineEnd) ":" "\n"
    503486
    504     // Fill in the OutgoingProbeStack.
    505     "ldr       x30, [sp, #" STRINGIZE_VALUE_OF(PROBE_CPU_SP_OFFSET) "]" "\n"
     487    // Fill in the OutgoingProbeRecord.
    506488    "sub       x30, x30, #" STRINGIZE_VALUE_OF(OUT_SIZE) "\n"
    507489
    … …  
    511493    "stp       x27, x28, [x30, #" STRINGIZE_VALUE_OF(OUT_X27_OFFSET) "]" "\n"
    512494    "ldr       x27, [sp, #" STRINGIZE_VALUE_OF(PROBE_CPU_FP_OFFSET) "]" "\n"
    513     "ldr       x28, [sp, #" STRINGIZE_VALUE_OF(PROBE_CPU_PC_OFFSET) "]" "\n"
     495    "ldr       x28, [sp, #" STRINGIZE_VALUE_OF(PROBE_CPU_PC_OFFSET) "]" "\n" // Set up the outgoing record so that we'll jump to the new PC.
    514496    "stp       x27, x28, [x30, #" STRINGIZE_VALUE_OF(OUT_FP_OFFSET) "]" "\n"
    515497    "mov       sp, x30" "\n"
    516498
    517     // Restore the remaining registers and pop the OutgoingProbeStack.
     499    // Restore the remaining registers and pop the OutgoingProbeRecord.
    518500    "ldp       x27, x28, [sp], #" STRINGIZE_VALUE_OF(2 * PTR_SIZE) "\n"
    519501    "msr       nzcv, x27" "\n"
    … …  
    522504    "ldp       x29, x30, [sp], #" STRINGIZE_VALUE_OF(2 * PTR_SIZE) "\n"
    523505    "ret" "\n"
    524 
    525     LOCAL_LABEL_STRING(ctiMasmProbeTrampolineError) ":" "\n"
    526     // The probe handler changed both lr and pc. This is not supported for ARM64.
    527     "ldr       x1, [sp, #" STRINGIZE_VALUE_OF(SAVED_PROBE_ERROR_FUNCTION_OFFSET) "]" "\n"
    528     "mov       x0, sp" "\n" // Set the Probe::State* arg.
    529     "blr       x1" "\n"
    530     "brk       #0x1000" // Should never return here.
    531506);
    532507#endif // COMPILER(GCC_OR_CLANG)
    533 
    534 static NO_RETURN_DUE_TO_CRASH void arm64ProbeError(Probe::State*)
    535 {
    536     dataLog("MacroAssembler probe ERROR: ARM64 does not support the probe changing both LR and PC.\n");
    537     RELEASE_ASSERT_NOT_REACHED();
    538 }
    539508
    540509void MacroAssembler::probe(Probe::Function function, void* arg)
    … …  
    542511    sub64(TrustedImm32(sizeof(IncomingProbeRecord)), sp);
    543512
     513    storePair64(x24, x25, sp, TrustedImm32(offsetof(IncomingProbeRecord, x24)));
    544514    storePair64(x26, x27, sp, TrustedImm32(offsetof(IncomingProbeRecord, x26)));
    545     add64(TrustedImm32(sizeof(IncomingProbeRecord)), sp, x26);
    546     storePair64(lr, x26, sp, TrustedImm32(offsetof(IncomingProbeRecord, lr))); // Save lr and original sp value.
    547     move(TrustedImmPtr(reinterpret_cast<void*>(function)), x26);
    548     move(TrustedImmPtr(arg), x27);
    549     storePair64(x26, x27, sp, TrustedImm32(offsetof(IncomingProbeRecord, probeHandlerFunction)));
    550     move(TrustedImmPtr(reinterpret_cast<void*>(arm64ProbeError)), x27);
    551     store64(x27, Address(sp, offsetof(IncomingProbeRecord, probeErrorFunction)));
    552 
     515    storePair64(x28, x30, sp, TrustedImm32(offsetof(IncomingProbeRecord, x28))); // Note: x30 is lr.
    553516    move(TrustedImmPtr(reinterpret_cast<void*>(ctiMasmProbeTrampoline)), x26);
     517    move(TrustedImmPtr(reinterpret_cast<void*>(Probe::executeProbe)), x28);
     518    move(TrustedImmPtr(reinterpret_cast<void*>(function)), x24);
     519    move(TrustedImmPtr(arg), x25);
    554520    m_assembler.blr(x26);
    555521
    556522    // ctiMasmProbeTrampoline should have restored every register except for lr and the sp.
    557     load64(Address(sp, offsetof(OutgoingProbeRecord, lr)), lr);
    558     add64(TrustedImm32(sizeof(OutgoingProbeRecord)), sp);
     523    load64(Address(sp, offsetof(LRRestorationRecord, lr)), lr);
     524    add64(TrustedImm32(sizeof(LRRestorationRecord)), sp);
    559525}
    560526

  • trunk/Source/JavaScriptCore/assembler/MacroAssemblerARMv7.cpp

    r220921 r220958  
    2929#include "MacroAssembler.h"
    3030
     31#include "ProbeContext.h"
    3132#include <wtf/InlineASM.h>
    3233
    … …  
    3637
    3738extern "C" void ctiMasmProbeTrampoline();
     39
     40using namespace ARMRegisters;
    3841
    3942#if COMPILER(GCC_OR_CLANG)
    … …  
    187190#undef PROBE_OFFSETOF
    188191
     192struct IncomingRecord {
     193    uintptr_t lr;
     194    uintptr_t ip;
     195    uintptr_t apsr;
     196    uintptr_t r0;
     197    uintptr_t r1;
     198    uintptr_t r2;
     199};
     200
     201#define IN_LR_OFFSET (0 * PTR_SIZE)
     202#define IN_IP_OFFSET (1 * PTR_SIZE)
     203#define IN_APSR_OFFSET (2 * PTR_SIZE)
     204#define IN_R0_OFFSET (3 * PTR_SIZE)
     205#define IN_R1_OFFSET (4 * PTR_SIZE)
     206#define IN_R2_OFFSET (5 * PTR_SIZE)
     207#define IN_SIZE      (6 * PTR_SIZE)
     208
     209static_assert(IN_LR_OFFSET == offsetof(IncomingRecord, lr), "IN_LR_OFFSET is incorrect");
     210static_assert(IN_IP_OFFSET == offsetof(IncomingRecord, ip), "IN_IP_OFFSET is incorrect");
     211static_assert(IN_APSR_OFFSET == offsetof(IncomingRecord, apsr), "IN_APSR_OFFSET is incorrect");
     212static_assert(IN_R0_OFFSET == offsetof(IncomingRecord, r0), "IN_R0_OFFSET is incorrect");
     213static_assert(IN_R1_OFFSET == offsetof(IncomingRecord, r1), "IN_R1_OFFSET is incorrect");
     214static_assert(IN_R2_OFFSET == offsetof(IncomingRecord, r2), "IN_R2_OFFSET is incorrect");
     215static_assert(IN_SIZE == sizeof(IncomingRecord), "IN_SIZE is incorrect");
     216
    189217asm (
    190218    ".text" "\n"
    … …  
    197225
    198226    // MacroAssemblerARMv7::probe() has already generated code to store some values.
    199     // The top of stack now looks like this:
    200     //     esp[0 * ptrSize]: probe handler function
    201     //     esp[1 * ptrSize]: probe arg
    202     //     esp[2 * ptrSize]: saved r0
    203     //     esp[3 * ptrSize]: saved ip
    204     //     esp[4 * ptrSize]: saved lr
    205     //     esp[5 * ptrSize]: saved sp
     227    // The top of stack now contains the IncomingRecord.
     228    //
     229    // Incoming register values:
     230    //     r0: probe function
     231    //     r1: probe arg
     232    //     r2: Probe::executeProbe
     233    //     ip: scratch, was ctiMasmProbeTrampoline
     234    //     lr: return address
    206235
    207236    "mov       ip, sp" "\n"
    208     "mov       r0, sp" "\n"
    209     "sub       r0, r0, #" STRINGIZE_VALUE_OF(PROBE_SIZE + OUT_SIZE) "\n"
     237    "str       r2, [ip, #-" STRINGIZE_VALUE_OF(PTR_SIZE) "]" "\n" // Stash Probe::executeProbe.
     238
     239    "mov       r2, sp" "\n"
     240    "sub       r2, r2, #" STRINGIZE_VALUE_OF(PROBE_SIZE + OUT_SIZE) "\n"
    210241
    211242    // The ARM EABI specifies that the stack needs to be 16 byte aligned.
    212     "bic       r0, r0, #0xf" "\n"
    213     "mov       sp, r0" "\n" // Set the sp to protect the Probe::State from interrupts before we initialize it.
    214 
     243    "bic       r2, r2, #0xf" "\n"
     244    "mov       sp, r2" "\n" // Set the sp to protect the Probe::State from interrupts before we initialize it.
     245    "ldr       r2, [ip, #-" STRINGIZE_VALUE_OF(PTR_SIZE) "]" "\n" // Reload Probe::executeProbe.
     246
     247    "str       r0, [sp, #" STRINGIZE_VALUE_OF(PROBE_PROBE_FUNCTION_OFFSET) "]" "\n"
     248    "str       r1, [sp, #" STRINGIZE_VALUE_OF(PROBE_ARG_OFFSET) "]" "\n"
    215249    "str       lr, [sp, #" STRINGIZE_VALUE_OF(PROBE_CPU_PC_OFFSET) "]" "\n"
    216     "add       lr, sp, #" STRINGIZE_VALUE_OF(PROBE_CPU_R1_OFFSET) "\n"
    217     "stmia     lr, { r1-r11 }" "\n"
    218     "mrs       lr, APSR" "\n"
    219     "str       lr, [sp, #" STRINGIZE_VALUE_OF(PROBE_CPU_APSR_OFFSET) "]" "\n"
     250
     251    "add       r0, ip, #" STRINGIZE_VALUE_OF(IN_SIZE) "\n"
     252    "str       r0, [sp, #" STRINGIZE_VALUE_OF(PROBE_CPU_SP_OFFSET) "]" "\n"
     253
     254    "add       lr, sp, #" STRINGIZE_VALUE_OF(PROBE_CPU_R3_OFFSET) "\n"
     255    "stmia     lr, { r3-r11 }" "\n"
     256
    220257    "vmrs      lr, FPSCR" "\n"
    221258    "str       lr, [sp, #" STRINGIZE_VALUE_OF(PROBE_CPU_FPSCR_OFFSET) "]" "\n"
    222259
    223     "ldr       lr, [ip, #0 * " STRINGIZE_VALUE_OF(PTR_SIZE) "]" "\n"
    224     "str       lr, [sp, #" STRINGIZE_VALUE_OF(PROBE_PROBE_FUNCTION_OFFSET) "]" "\n"
    225     "ldr       lr, [ip, #1 * " STRINGIZE_VALUE_OF(PTR_SIZE) "]" "\n"
    226     "str       lr, [sp, #" STRINGIZE_VALUE_OF(PROBE_ARG_OFFSET) "]" "\n"
    227     "ldr       lr, [ip, #2 * " STRINGIZE_VALUE_OF(PTR_SIZE) "]" "\n"
    228     "str       lr, [sp, #" STRINGIZE_VALUE_OF(PROBE_CPU_R0_OFFSET) "]" "\n"
    229     "ldr       lr, [ip, #3 * " STRINGIZE_VALUE_OF(PTR_SIZE) "]" "\n"
    230     "str       lr, [sp, #" STRINGIZE_VALUE_OF(PROBE_CPU_IP_OFFSET) "]" "\n"
    231     "ldr       lr, [ip, #4 * " STRINGIZE_VALUE_OF(PTR_SIZE) "]" "\n"
    232     "str       lr, [sp, #" STRINGIZE_VALUE_OF(PROBE_CPU_LR_OFFSET) "]" "\n"
    233     "ldr       lr, [ip, #5 * " STRINGIZE_VALUE_OF(PTR_SIZE) "]" "\n"
    234     "str       lr, [sp, #" STRINGIZE_VALUE_OF(PROBE_CPU_SP_OFFSET) "]" "\n"
    235 
    236     "ldr       lr, [sp, #" STRINGIZE_VALUE_OF(PROBE_CPU_PC_OFFSET) "]" "\n"
     260    "ldr       r4, [ip, #" STRINGIZE_VALUE_OF(IN_LR_OFFSET) "]" "\n"
     261    "ldr       r5, [ip, #" STRINGIZE_VALUE_OF(IN_IP_OFFSET) "]" "\n"
     262    "ldr       r6, [ip, #" STRINGIZE_VALUE_OF(IN_APSR_OFFSET) "]" "\n"
     263    "ldr       r7, [ip, #" STRINGIZE_VALUE_OF(IN_R0_OFFSET) "]" "\n"
     264    "ldr       r8, [ip, #" STRINGIZE_VALUE_OF(IN_R1_OFFSET) "]" "\n"
     265    "ldr       r9, [ip, #" STRINGIZE_VALUE_OF(IN_R2_OFFSET) "]" "\n"
     266    "str       r4, [sp, #" STRINGIZE_VALUE_OF(PROBE_CPU_LR_OFFSET) "]" "\n"
     267    "str       r5, [sp, #" STRINGIZE_VALUE_OF(PROBE_CPU_IP_OFFSET) "]" "\n"
     268    "str       r6, [sp, #" STRINGIZE_VALUE_OF(PROBE_CPU_APSR_OFFSET) "]" "\n"
     269    "str       r7, [sp, #" STRINGIZE_VALUE_OF(PROBE_CPU_R0_OFFSET) "]" "\n"
     270    "str       r8, [sp, #" STRINGIZE_VALUE_OF(PROBE_CPU_R1_OFFSET) "]" "\n"
     271    "str       r9, [sp, #" STRINGIZE_VALUE_OF(PROBE_CPU_R2_OFFSET) "]" "\n"
    237272
    238273    "add       ip, sp, #" STRINGIZE_VALUE_OF(PROBE_CPU_D0_OFFSET) "\n"
    … …  
    241276    "vstmia.64 ip!, { d16-d31 }" "\n"
    242277#endif
    243     "mov       fp, sp" "\n" // Save the Probe::State*.
    244 
    245     // Initialize Probe::State::initializeStackFunction to zero.
    246     "mov       r0, #0" "\n"
    247     "str       r0, [fp, #" STRINGIZE_VALUE_OF(PROBE_INIT_STACK_FUNCTION_OFFSET) "]" "\n"
    248 
    249     "ldr       ip, [sp, #" STRINGIZE_VALUE_OF(PROBE_PROBE_FUNCTION_OFFSET) "]" "\n"
     278
     279    // r5 is a callee saved register. We'll use it for preserving the Probe::State*.
     280    // https://stackoverflow.com/questions/261419/arm-to-c-calling-convention-registers-to-save#261496
     281    "mov       r5, sp" "\n"
     282
    250283    "mov       r0, sp" "\n" // the Probe::State* arg.
    251     "blx       ip" "\n"
     284    "blx       r2" "\n" // Call Probe::executeProbe.
    252285
    253286    // Make sure the Probe::State is entirely below the result stack pointer so
    254287    // that register values are still preserved when we call the initializeStack
    255288    // function.
    256     "ldr       r1, [fp, #" STRINGIZE_VALUE_OF(PROBE_CPU_SP_OFFSET) "]" "\n" // Result sp.
    257     "add       r2, fp, #" STRINGIZE_VALUE_OF(PROBE_SIZE + OUT_SIZE) "\n" // End of ProveContext + buffer.
     289    "ldr       r1, [r5, #" STRINGIZE_VALUE_OF(PROBE_CPU_SP_OFFSET) "]" "\n" // Result sp.
     290    "add       r2, r5, #" STRINGIZE_VALUE_OF(PROBE_SIZE + OUT_SIZE) "\n" // End of ProveContext + buffer.
    258291    "cmp       r1, r2" "\n"
    259292    "it        ge" "\n"
    … …  
    268301    // Note: we have to copy from low address to higher address because we're moving the
    269302    // Probe::State to a lower address.
    270     "mov       r5, fp" "\n"
    271     "mov       r6, r1" "\n"
    272     "add       r7, fp, #" STRINGIZE_VALUE_OF(PROBE_SIZE) "\n"
     303    "add       r7, r5, #" STRINGIZE_VALUE_OF(PROBE_SIZE) "\n"
    273304
    274305    LOCAL_LABEL_STRING(ctiMasmProbeTrampolineCopyLoop) ":" "\n"
    275306    "ldr       r3, [r5], #4" "\n"
    276307    "ldr       r4, [r5], #4" "\n"
    277     "str       r3, [r6], #4" "\n"
    278     "str       r4, [r6], #4" "\n"
     308    "str       r3, [r1], #4" "\n"
     309    "str       r4, [r1], #4" "\n"
    279310    "cmp       r5, r7" "\n"
    280311    "it        lt" "\n"
    281312    "blt     " LOCAL_LABEL_STRING(ctiMasmProbeTrampolineCopyLoop) "\n"
    282313
    283     "mov       fp, r1" "\n"
     314    "mov       r5, sp" "\n"
    284315
    285316    // Call initializeStackFunction if present.
    286317    LOCAL_LABEL_STRING(ctiMasmProbeTrampolineProbeStateIsSafe) ":" "\n"
    287     "ldr       r2, [fp, #" STRINGIZE_VALUE_OF(PROBE_INIT_STACK_FUNCTION_OFFSET) "]" "\n"
     318    "ldr       r2, [r5, #" STRINGIZE_VALUE_OF(PROBE_INIT_STACK_FUNCTION_OFFSET) "]" "\n"
    288319    "cbz       r2, " LOCAL_LABEL_STRING(ctiMasmProbeTrampolineRestoreRegisters) "\n"
    289320
    290     "mov       r0, fp" "\n" // Set the Probe::State* arg.
     321    "mov       r0, r5" "\n" // Set the Probe::State* arg.
    291322    "blx       r2" "\n" // Call the initializeStackFunction (loaded into r2 above).
    292323
    293324    LOCAL_LABEL_STRING(ctiMasmProbeTrampolineRestoreRegisters) ":" "\n"
    294325
    295     "mov       sp, fp" "\n"
     326    "mov       sp, r5" "\n" // Ensure that sp points to the Probe::State*.
    296327
    297328    // To enable probes to modify register state, we copy all registers
    … …  
    339370void MacroAssembler::probe(Probe::Function function, void* arg)
    340371{
    341     push(RegisterID::lr);
    342     push(RegisterID::lr);
    343     add32(TrustedImm32(8), RegisterID::sp, RegisterID::lr);
    344     store32(RegisterID::lr, ArmAddress(RegisterID::sp, 4));
    345     push(RegisterID::ip);
    346     push(RegisterID::r0);
    347     // The following uses RegisterID::ip. So, they must come after we push ip above.
    348     push(trustedImm32FromPtr(arg));
    349     push(trustedImm32FromPtr(function));
    350 
    351     move(trustedImm32FromPtr(ctiMasmProbeTrampoline), RegisterID::ip);
    352     m_assembler.blx(RegisterID::ip);
     372    sub32(TrustedImm32(sizeof(IncomingRecord)), sp);
     373
     374    store32(lr, Address(sp, offsetof(IncomingRecord, lr)));
     375    store32(ip, Address(sp, offsetof(IncomingRecord, ip)));
     376    m_assembler.mrs(ip, apsr);
     377    store32(ip, Address(sp, offsetof(IncomingRecord, apsr)));
     378    store32(r0, Address(sp, offsetof(IncomingRecord, r0)));
     379    store32(r1, Address(sp, offsetof(IncomingRecord, r1)));
     380    store32(r2, Address(sp, offsetof(IncomingRecord, r2)));
     381
     382    // The following may emit a T1 mov instruction, which is effectively a movs.
     383    // This means we must first preserve the apsr flags above first.
     384    move(TrustedImmPtr(reinterpret_cast<void*>(function)), r0);
     385    move(TrustedImmPtr(arg), r1);
     386    move(TrustedImmPtr(reinterpret_cast<void*>(Probe::executeProbe)), r2);
     387    move(TrustedImmPtr(reinterpret_cast<void*>(ctiMasmProbeTrampoline)), ip);
     388    m_assembler.blx(ip);
    353389}
    354390#endif // ENABLE(MASM_PROBE)
    … …  
    356392} // namespace JSC
    357393
    358 #endif // ENABLE(ASSEMBLER)
    359 
     394#endif // ENABLE(ASSEMBLER) && CPU(ARM_THUMB2)

  • trunk/Source/JavaScriptCore/assembler/MacroAssemblerARMv7.h

    r220921 r220958  
    21072107        return static_cast<ARMv7Assembler::Condition>(cond);
    21082108    }
    2109    
    2110 #if ENABLE(MASM_PROBE)
    2111     inline TrustedImm32 trustedImm32FromPtr(void* ptr)
    2112     {
    2113         return TrustedImm32(TrustedImmPtr(ptr));
    2114     }
    2115 
    2116     inline TrustedImm32 trustedImm32FromPtr(Probe::Function function)
    2117     {
    2118         return TrustedImm32(TrustedImmPtr(reinterpret_cast<void*>(function)));
    2119     }
    2120 
    2121     inline TrustedImm32 trustedImm32FromPtr(void (*function)())
    2122     {
    2123         return TrustedImm32(TrustedImmPtr(reinterpret_cast<void*>(function)));
    2124     }
    2125 #endif // ENABLE(MASM_PROBE)
    21262109
    21272110private:

  • trunk/Source/JavaScriptCore/assembler/MacroAssemblerPrinter.cpp

    r220921 r220958  
    3434
    3535namespace JSC {
     36
    3637namespace Printer {
    3738
    38 using CPUState = MacroAssembler::CPUState;
     39using CPUState = Probe::CPUState;
    3940using RegisterID = MacroAssembler::RegisterID;
    4041using FPRegisterID = MacroAssembler::FPRegisterID;
    … …  
    172173}
    173174
    174 void printCallback(Probe::State* probeContext)
     175void printCallback(Probe::Context& probeContext)
    175176{
    176177    auto& out = WTF::dataFile();
    177     PrintRecordList& list = *reinterpret_cast<PrintRecordList*>(probeContext->arg);
     178    PrintRecordList& list = *reinterpret_cast<PrintRecordList*>(probeContext.arg);
    178179    for (size_t i = 0; i < list.size(); i++) {
    179180        auto& record = list[i];
    180         Context context(*probeContext, record.data);
     181        Context context(probeContext, record.data);
    181182        record.printer(out, context);
    182183    }

  • trunk/Source/JavaScriptCore/assembler/MacroAssemblerPrinter.h

    r220921 r220958  
    2828#include "MacroAssembler.h"
    2929#include "Printer.h"
     30#include "ProbeContext.h"
    3031
    3132namespace JSC {
    … …  
    224225};
    225226
    226 void printCallback(Probe::State*);
     227void printCallback(Probe::Context&);
    227228
    228229} // namespace Printer

  • trunk/Source/JavaScriptCore/assembler/MacroAssemblerX86Common.cpp

    r220926 r220958  
    2929#include "MacroAssembler.h"
    3030
     31#include "ProbeContext.h"
    3132#include <wtf/InlineASM.h>
    3233
    … …  
    102103#endif // CPU(X86_64)
    103104
     105#define PROBE_EXECUTOR_OFFSET PROBE_SIZE // Stash the executeProbe function pointer at the end of the ProbeContext.
     106
    104107// The outgoing record to be popped off the stack at the end consists of:
    105108// eflags, eax, ecx, ebp, eip.
    … …  
    160163
    161164COMPILE_ASSERT(sizeof(Probe::State) == PROBE_SIZE, ProbeState_size_matches_ctiMasmProbeTrampoline);
     165COMPILE_ASSERT((PROBE_EXECUTOR_OFFSET + PTR_SIZE) <= (PROBE_SIZE + OUT_SIZE), Must_have_room_after_ProbeContext_to_stash_the_probe_handler);
    162166
    163167#undef PROBE_OFFSETOF
    … …  
    177181    //     esp[0 * ptrSize]: eflags
    178182    //     esp[1 * ptrSize]: return address / saved eip
    179     //     esp[2 * ptrSize]: probe handler function
    180     //     esp[3 * ptrSize]: probe arg
    181     //     esp[4 * ptrSize]: saved eax
    182     //     esp[5 * ptrSize]: saved esp
     183    //     esp[2 * ptrSize]: saved ebx
     184    //     esp[3 * ptrSize]: saved edx
     185    //     esp[4 * ptrSize]: saved ecx
     186    //     esp[5 * ptrSize]: saved eax
     187    //
     188    // Incoming registers contain:
     189    //     ecx: Probe::executeProbe
     190    //     edx: probe function
     191    //     ebx: probe arg
     192    //     eax: scratch (was ctiMasmProbeTrampoline)
    183193
    184194    "movl %esp, %eax" "\n"
    … …  
    191201    "movl %esp, %ebp" "\n" // Save the Probe::State*.
    192202
    193     "movl %ecx, " STRINGIZE_VALUE_OF(PROBE_CPU_ECX_OFFSET) "(%ebp)" "\n"
    194     "movl %edx, " STRINGIZE_VALUE_OF(PROBE_CPU_EDX_OFFSET) "(%ebp)" "\n"
    195     "movl %ebx, " STRINGIZE_VALUE_OF(PROBE_CPU_EBX_OFFSET) "(%ebp)" "\n"
     203    "movl %ecx, " STRINGIZE_VALUE_OF(PROBE_EXECUTOR_OFFSET) "(%ebp)" "\n"
     204    "movl %edx, " STRINGIZE_VALUE_OF(PROBE_PROBE_FUNCTION_OFFSET) "(%ebp)" "\n"
     205    "movl %ebx, " STRINGIZE_VALUE_OF(PROBE_ARG_OFFSET) "(%ebp)" "\n"
    196206    "movl %esi, " STRINGIZE_VALUE_OF(PROBE_CPU_ESI_OFFSET) "(%ebp)" "\n"
    197207    "movl %edi, " STRINGIZE_VALUE_OF(PROBE_CPU_EDI_OFFSET) "(%ebp)" "\n"
    … …  
    202212    "movl %ecx, " STRINGIZE_VALUE_OF(PROBE_CPU_EIP_OFFSET) "(%ebp)" "\n"
    203213    "movl 2 * " STRINGIZE_VALUE_OF(PTR_SIZE) "(%eax), %ecx" "\n"
    204     "movl %ecx, " STRINGIZE_VALUE_OF(PROBE_PROBE_FUNCTION_OFFSET) "(%ebp)" "\n"
     214    "movl %ecx, " STRINGIZE_VALUE_OF(PROBE_CPU_EBX_OFFSET) "(%ebp)" "\n"
    205215    "movl 3 * " STRINGIZE_VALUE_OF(PTR_SIZE) "(%eax), %ecx" "\n"
    206     "movl %ecx, " STRINGIZE_VALUE_OF(PROBE_ARG_OFFSET) "(%ebp)" "\n"
     216    "movl %ecx, " STRINGIZE_VALUE_OF(PROBE_CPU_EDX_OFFSET) "(%ebp)" "\n"
    207217    "movl 4 * " STRINGIZE_VALUE_OF(PTR_SIZE) "(%eax), %ecx" "\n"
     218    "movl %ecx, " STRINGIZE_VALUE_OF(PROBE_CPU_ECX_OFFSET) "(%ebp)" "\n"
     219    "movl 5 * " STRINGIZE_VALUE_OF(PTR_SIZE) "(%eax), %ecx" "\n"
    208220    "movl %ecx, " STRINGIZE_VALUE_OF(PROBE_CPU_EAX_OFFSET) "(%ebp)" "\n"
    209     "movl 5 * " STRINGIZE_VALUE_OF(PTR_SIZE) "(%eax), %ecx" "\n"
     221
     222    "movl %eax, %ecx" "\n"
     223    "addl $" STRINGIZE_VALUE_OF(6 * PTR_SIZE) ", %ecx" "\n"
    210224    "movl %ecx, " STRINGIZE_VALUE_OF(PROBE_CPU_ESP_OFFSET) "(%ebp)" "\n"
    211225
    … …  
    219233    "movq %xmm7, " STRINGIZE_VALUE_OF(PROBE_CPU_XMM7_OFFSET) "(%ebp)" "\n"
    220234
    221     "xorl %eax, %eax" "\n"
    222     "movl %eax, " STRINGIZE_VALUE_OF(PROBE_INIT_STACK_FUNCTION_OFFSET) "(%ebp)" "\n"
    223 
    224235    // Reserve stack space for the arg while maintaining the required stack
    225236    // pointer 32 byte alignment:
    … …  
    227238    "movl %ebp, 0(%esp)" "\n" // the Probe::State* arg.
    228239
    229     "call *" STRINGIZE_VALUE_OF(PROBE_PROBE_FUNCTION_OFFSET) "(%ebp)" "\n"
     240    "call *" STRINGIZE_VALUE_OF(PROBE_EXECUTOR_OFFSET) "(%ebp)" "\n"
    230241
    231242    // Make sure the Probe::State is entirely below the result stack pointer so
    … …  
    337348        //     esp[0 * ptrSize]: eflags
    338349        //     esp[1 * ptrSize]: return address / saved eip
    339         //     esp[2 * ptrSize]: probe handler function
    340         //     esp[3 * ptrSize]: probe arg
    341         //     esp[4 * ptrSize]: saved eax
    342         //     esp[5 * ptrSize]: saved esp
     350        //     esp[2 * ptrSize]: saved ebx
     351        //     esp[3 * ptrSize]: saved edx
     352        //     esp[4 * ptrSize]: saved ecx
     353        //     esp[5 * ptrSize]: saved eax
     354        //
     355        // Incoming registers contain:
     356        //     ecx: Probe::executeProbe
     357        //     edx: probe function
     358        //     ebx: probe arg
     359        //     eax: scratch (was ctiMasmProbeTrampoline)
    343360
    344361        mov eax, esp
    … …  
    351368        mov ebp, esp // Save the ProbeContext*.
    352369
    353         mov [PROBE_CPU_ECX_OFFSET + ebp], ecx
    354         mov [PROBE_CPU_EDX_OFFSET + ebp], edx
    355         mov [PROBE_CPU_EBX_OFFSET + ebp], ebx
     370        mov [PROBE_EXECUTOR_OFFSET + ebp], ecx
     371        mov [PROBE_PROBE_FUNCTION_OFFSET + ebp], edx
     372        mov [PROBE_ARG_OFFSET + ebp], ebx
    356373        mov [PROBE_CPU_ESI_OFFSET + ebp], esi
    357374        mov [PROBE_CPU_EDI_OFFSET + ebp], edi
    … …  
    362379        mov [PROBE_CPU_EIP_OFFSET + ebp], ecx
    363380        mov ecx, [2 * PTR_SIZE + eax]
    364         mov [PROBE_PROBE_FUNCTION_OFFSET + ebp], ecx
     381        mov [PROBE_CPU_EBX_OFFSET + ebp], ecx
    365382        mov ecx, [3 * PTR_SIZE + eax]
    366         mov [PROBE_ARG_OFFSET + ebp], ecx
     383        mov [PROBE_CPU_EDX_OFFSET + ebp], ecx
    367384        mov ecx, [4 * PTR_SIZE + eax]
     385        mov [PROBE_CPU_ECX_OFFSET + ebp], ecx
     386        mov ecx, [5 * PTR_SIZE + eax]
    368387        mov [PROBE_CPU_EAX_OFFSET + ebp], ecx
    369         mov ecx, [5 * PTR_SIZE + eax]
     388
     389        mov ecx, eax
     390        add ecx, 6 * PTR_SIZE
    370391        mov [PROBE_CPU_ESP_OFFSET + ebp], ecx
    371392
    … …  
    379400        movq qword ptr[PROBE_CPU_XMM7_OFFSET + ebp], xmm7
    380401
    381         xor eax, eax
    382         mov [PROBE_INIT_STACK_FUNCTION_OFFSET + ebp], eax
    383 
    384402        // Reserve stack space for the arg while maintaining the required stack
    385403        // pointer 32 byte alignment:
    … …  
    387405        mov [0 + esp], ebp // the ProbeContext* arg.
    388406
    389         call [PROBE_PROBE_FUNCTION_OFFSET + ebp]
     407        call [PROBE_EXECUTOR_OFFSET + ebp]
    390408
    391409        // Make sure the ProbeContext is entirely below the result stack pointer so
    … …  
    485503    }
    486504}
    487 #endif
     505#endif // COMPILER(MSVC)
    488506
    489507#endif // CPU(X86)
    … …  
    499517
    500518    // MacroAssemblerX86Common::probe() has already generated code to store some values.
    501     // Together with the rflags pushed above, the top of stack now looks like
    502     // this:
    503     //     esp[0 * ptrSize]: rflags
    504     //     esp[1 * ptrSize]: return address / saved rip
    505     //     esp[2 * ptrSize]: probe handler function
    506     //     esp[3 * ptrSize]: probe arg
    507     //     esp[4 * ptrSize]: saved rax
    508     //     esp[5 * ptrSize]: saved rsp
     519    // Together with the rflags pushed above, the top of stack now looks like this:
     520    //     rsp[0 * ptrSize]: rflags
     521    //     rsp[1 * ptrSize]: return address / saved rip
     522    //     rsp[2 * ptrSize]: saved rbx
     523    //     rsp[3 * ptrSize]: saved rdx
     524    //     rsp[4 * ptrSize]: saved rcx
     525    //     rsp[5 * ptrSize]: saved rax
     526    //
     527    // Incoming registers contain:
     528    //     rcx: Probe::executeProbe
     529    //     rdx: probe function
     530    //     rbx: probe arg
     531    //     rax: scratch (was ctiMasmProbeTrampoline)
    509532
    510533    "movq %rsp, %rax" "\n"
    … …  
    518541    "movq %rsp, %rbp" "\n" // Save the Probe::State*.
    519542
    520     "movq %rcx, " STRINGIZE_VALUE_OF(PROBE_CPU_ECX_OFFSET) "(%rbp)" "\n"
    521     "movq %rdx, " STRINGIZE_VALUE_OF(PROBE_CPU_EDX_OFFSET) "(%rbp)" "\n"
    522     "movq %rbx, " STRINGIZE_VALUE_OF(PROBE_CPU_EBX_OFFSET) "(%rbp)" "\n"
     543    "movq %rcx, " STRINGIZE_VALUE_OF(PROBE_EXECUTOR_OFFSET) "(%rbp)" "\n"
     544    "movq %rdx, " STRINGIZE_VALUE_OF(PROBE_PROBE_FUNCTION_OFFSET) "(%rbp)" "\n"
     545    "movq %rbx, " STRINGIZE_VALUE_OF(PROBE_ARG_OFFSET) "(%rbp)" "\n"
    523546    "movq %rsi, " STRINGIZE_VALUE_OF(PROBE_CPU_ESI_OFFSET) "(%rbp)" "\n"
    524547    "movq %rdi, " STRINGIZE_VALUE_OF(PROBE_CPU_EDI_OFFSET) "(%rbp)" "\n"
    … …  
    529552    "movq %rcx, " STRINGIZE_VALUE_OF(PROBE_CPU_EIP_OFFSET) "(%rbp)" "\n"
    530553    "movq 2 * " STRINGIZE_VALUE_OF(PTR_SIZE) "(%rax), %rcx" "\n"
    531     "movq %rcx, " STRINGIZE_VALUE_OF(PROBE_PROBE_FUNCTION_OFFSET) "(%rbp)" "\n"
     554    "movq %rcx, " STRINGIZE_VALUE_OF(PROBE_CPU_EBX_OFFSET) "(%rbp)" "\n"
    532555    "movq 3 * " STRINGIZE_VALUE_OF(PTR_SIZE) "(%rax), %rcx" "\n"
    533     "movq %rcx, " STRINGIZE_VALUE_OF(PROBE_ARG_OFFSET) "(%rbp)" "\n"
     556    "movq %rcx, " STRINGIZE_VALUE_OF(PROBE_CPU_EDX_OFFSET) "(%rbp)" "\n"
    534557    "movq 4 * " STRINGIZE_VALUE_OF(PTR_SIZE) "(%rax), %rcx" "\n"
     558    "movq %rcx, " STRINGIZE_VALUE_OF(PROBE_CPU_ECX_OFFSET) "(%rbp)" "\n"
     559    "movq 5 * " STRINGIZE_VALUE_OF(PTR_SIZE) "(%rax), %rcx" "\n"
    535560    "movq %rcx, " STRINGIZE_VALUE_OF(PROBE_CPU_EAX_OFFSET) "(%rbp)" "\n"
    536     "movq 5 * " STRINGIZE_VALUE_OF(PTR_SIZE) "(%rax), %rcx" "\n"
     561
     562    "movq %rax, %rcx" "\n"
     563    "addq $" STRINGIZE_VALUE_OF(6 * PTR_SIZE) ", %rcx" "\n"
    537564    "movq %rcx, " STRINGIZE_VALUE_OF(PROBE_CPU_ESP_OFFSET) "(%rbp)" "\n"
    538565
    … …  
    563590    "movq %xmm15, " STRINGIZE_VALUE_OF(PROBE_CPU_XMM15_OFFSET) "(%rbp)" "\n"
    564591
    565     "xorq %rax, %rax" "\n"
    566     "movq %rax, " STRINGIZE_VALUE_OF(PROBE_INIT_STACK_FUNCTION_OFFSET) "(%rbp)" "\n"
    567 
    568592    "movq %rbp, %rdi" "\n" // the Probe::State* arg.
    569     "call *" STRINGIZE_VALUE_OF(PROBE_PROBE_FUNCTION_OFFSET) "(%rbp)" "\n"
     593    "call *" STRINGIZE_VALUE_OF(PROBE_EXECUTOR_OFFSET) "(%rbp)" "\n"
    570594
    571595    // Make sure the Probe::State is entirely below the result stack pointer so
    … …  
    732756void MacroAssembler::probe(Probe::Function function, void* arg)
    733757{
    734     push(RegisterID::esp);
    735     push(RegisterID::eax);
    736     move(TrustedImmPtr(arg), RegisterID::eax);
    737     push(RegisterID::eax);
    738     move(TrustedImmPtr(reinterpret_cast<void*>(function)), RegisterID::eax);
    739758    push(RegisterID::eax);
    740759    move(TrustedImmPtr(reinterpret_cast<void*>(ctiMasmProbeTrampoline)), RegisterID::eax);
     760    push(RegisterID::ecx);
     761    move(TrustedImmPtr(reinterpret_cast<void*>(Probe::executeProbe)), RegisterID::ecx);
     762    push(RegisterID::edx);
     763    move(TrustedImmPtr(reinterpret_cast<void*>(function)), RegisterID::edx);
     764    push(RegisterID::ebx);
     765    move(TrustedImmPtr(arg), RegisterID::ebx);
    741766    call(RegisterID::eax);
    742767}

  • trunk/Source/JavaScriptCore/assembler/Printer.h

    r220921 r220958  
    3333
    3434namespace Probe {
    35 struct State;
     35class Context;
    3636} // namespace Probe
    3737
    … …  
    8787
    8888struct Context {
    89     Context(Probe::State& probeContext, Data& data)
     89    Context(Probe::Context& probeContext, Data& data)
    9090        : probeContext(probeContext)
    9191        , data(data)
    9292    { }
    9393
    94     Probe::State& probeContext;
     94    Probe::Context& probeContext;
    9595    Data& data;
    9696};

  • trunk/Source/JavaScriptCore/assembler/testmasm.cpp

    r220926 r220958  
    3232#include "InitializeThreading.h"
    3333#include "LinkBuffer.h"
     34#include "ProbeContext.h"
    3435#include <wtf/Compiler.h>
    3536#include <wtf/DataLog.h>
    … …  
    6263#endif // ENABLE(MASM_PROBE)
    6364
     65namespace JSC {
     66namespace Probe {
     67
     68JS_EXPORT_PRIVATE void* probeStateForContext(Probe::Context&);
     69
     70} // namespace Probe
     71} // namespace JSC
     72
    6473using namespace JSC;
    6574
    … …  
    6776
    6877#if ENABLE(MASM_PROBE)
    69 using CPUState = MacroAssembler::CPUState;
     78using CPUState = Probe::CPUState;
    7079#endif
    7180
    … …  
    185194#endif
    186195
    187         jit.probe([&] (Probe::State* context) {
    188             auto& cpu = context->cpu;
     196        jit.probe([&] (Probe::Context& context) {
     197            auto& cpu = context.cpu;
    189198            probeWasCalled = true;
    190199            CHECK_EQ(cpu.gpr(GPRInfo::argumentGPR0), testWord(0));
    … …  
    227236
    228237        // Write expected values.
    229         jit.probe([&] (Probe::State* context) {
    230             auto& cpu = context->cpu;
     238        jit.probe([&] (Probe::Context& context) {
     239            auto& cpu = context.cpu;
    231240            probeCallCount++;
    232241            cpu.gpr(GPRInfo::argumentGPR0) = testWord(0);
    … …  
    240249
    241250        // Validate that expected values were written.
    242         jit.probe([&] (Probe::State* context) {
    243             auto& cpu = context->cpu;
     251        jit.probe([&] (Probe::Context& context) {
     252            auto& cpu = context.cpu;
    244253            probeCallCount++;
    245254            CHECK_EQ(cpu.gpr(GPRInfo::argumentGPR0), testWord(0));
    … …  
    283292
    284293        // Write expected values into the registers (except for sp, fp, and pc).
    285         jit.probe([&] (Probe::State* context) {
    286             auto& cpu = context->cpu;
     294        jit.probe([&] (Probe::Context& context) {
     295            auto& cpu = context.cpu;
    287296            probeCallCount++;
    288297            for (auto id = CCallHelpers::firstRegister(); id <= CCallHelpers::lastRegister(); id = nextID(id)) {
    … …  
    299308
    300309        // Invoke the probe to call a lot of functions and trash register values.
    301         jit.probe([&] (Probe::State*) {
     310        jit.probe([&] (Probe::Context&) {
    302311            probeCallCount++;
    303312            CHECK_EQ(testFunctionToTrashGPRs(0, 1, 2, 3, 4, 5, 6, 7, 8, 9), 10);
    … …  
    306315
    307316        // Validate that the registers have the expected values.
    308         jit.probe([&] (Probe::State* context) {
    309             auto& cpu = context->cpu;
     317        jit.probe([&] (Probe::Context& context) {
     318            auto& cpu = context.cpu;
    310319            probeCallCount++;
    311320            for (auto id = CCallHelpers::firstRegister(); id <= CCallHelpers::lastRegister(); id = nextID(id)) {
    … …  
    323332
    324333        // Restore the original state.
    325         jit.probe([&] (Probe::State* context) {
    326             auto& cpu = context->cpu;
     334        jit.probe([&] (Probe::Context& context) {
     335            auto& cpu = context.cpu;
    327336            probeCallCount++;
    328337            for (auto id = CCallHelpers::firstRegister(); id <= CCallHelpers::lastRegister(); id = nextID(id)) {
    … …  
    336345
    337346        // Validate that the original state was restored.
    338         jit.probe([&] (Probe::State* context) {
    339             auto& cpu = context->cpu;
     347        jit.probe([&] (Probe::Context& context) {
     348            auto& cpu = context.cpu;
    340349            probeCallCount++;
    341350            for (auto id = CCallHelpers::firstRegister(); id <= CCallHelpers::lastRegister(); id = nextID(id)) {
    … …  
    354363}
    355364
    356 void testProbeModifiesStackPointer(WTF::Function<void*(Probe::State*)> computeModifiedStack)
     365void testProbeModifiesStackPointer(WTF::Function<void*(Probe::Context&)> computeModifiedStackPointer)
    357366{
    358367    unsigned probeCallCount = 0;
    … …  
    367376#elif CPU(ARM_THUMB2) || CPU(ARM_TRADITIONAL)
    368377    auto flagsSPR = ARMRegisters::apsr;
    369     uintptr_t flagsMask = 0xf0000000;
     378    uintptr_t flagsMask = 0xf8000000;
    370379#elif CPU(ARM64)
    371380    auto flagsSPR = ARM64Registers::nzcv;
    … …  
    378387        // Preserve original stack pointer and modify the sp, and
    379388        // write expected values into other registers (except for fp, and pc).
    380         jit.probe([&] (Probe::State* context) {
    381             auto& cpu = context->cpu;
     389        jit.probe([&] (Probe::Context& context) {
     390            auto& cpu = context.cpu;
    382391            probeCallCount++;
    383392            for (auto id = CCallHelpers::firstRegister(); id <= CCallHelpers::lastRegister(); id = nextID(id)) {
    … …  
    397406
    398407            originalSP = cpu.sp();
    399             modifiedSP = computeModifiedStack(context);
     408            modifiedSP = computeModifiedStackPointer(context);
    400409            cpu.sp() = modifiedSP;
    401410        });
    402411
    403412        // Validate that the registers have the expected values.
    404         jit.probe([&] (Probe::State* context) {
    405             auto& cpu = context->cpu;
     413        jit.probe([&] (Probe::Context& context) {
     414            auto& cpu = context.cpu;
    406415            probeCallCount++;
    407416            for (auto id = CCallHelpers::firstRegister(); id <= CCallHelpers::lastRegister(); id = nextID(id)) {
    … …  
    416425            for (auto id = CCallHelpers::firstFPRegister(); id <= CCallHelpers::lastFPRegister(); id = nextID(id))
    417426                CHECK_EQ(cpu.fpr<uint64_t>(id), testWord64(id));
    418             CHECK_EQ(cpu.spr(flagsSPR), modifiedFlags);
     427            CHECK_EQ(cpu.spr(flagsSPR) & flagsMask, modifiedFlags & flagsMask);
    419428            CHECK_EQ(cpu.sp(), modifiedSP);
    420429        });
    421430
    422431        // Restore the original state.
    423         jit.probe([&] (Probe::State* context) {
    424             auto& cpu = context->cpu;
     432        jit.probe([&] (Probe::Context& context) {
     433            auto& cpu = context.cpu;
    425434            probeCallCount++;
    426435            for (auto id = CCallHelpers::firstRegister(); id <= CCallHelpers::lastRegister(); id = nextID(id)) {
    … …  
    436445
    437446        // Validate that the original state was restored.
    438         jit.probe([&] (Probe::State* context) {
    439             auto& cpu = context->cpu;
     447        jit.probe([&] (Probe::Context& context) {
     448            auto& cpu = context.cpu;
    440449            probeCallCount++;
    441450            for (auto id = CCallHelpers::firstRegister(); id <= CCallHelpers::lastRegister(); id = nextID(id)) {
    … …  
    446455            for (auto id = CCallHelpers::firstFPRegister(); id <= CCallHelpers::lastFPRegister(); id = nextID(id))
    447456                CHECK_EQ(cpu.fpr<uint64_t>(id), originalState.fpr<uint64_t>(id));
    448             CHECK_EQ(cpu.spr(flagsSPR), originalState.spr(flagsSPR));
     457            CHECK_EQ(cpu.spr(flagsSPR) & flagsMask, originalState.spr(flagsSPR) & flagsMask);
    449458            CHECK_EQ(cpu.sp(), originalSP);
    450459        });
    … …  
    464473#endif
    465474    for (size_t offset = 0; offset < sizeof(Probe::State); offset += increment) {
    466         testProbeModifiesStackPointer([=] (Probe::State* context) -> void* {
    467             return reinterpret_cast<uint8_t*>(context) + offset;
     475        testProbeModifiesStackPointer([=] (Probe::Context& context) -> void* {
     476            return reinterpret_cast<uint8_t*>(probeStateForContext(context)) + offset;
     477
    468478        });
    469479    }
    … …  
    478488#endif
    479489    for (size_t offset = 0; offset < 1 * KB; offset += increment) {
    480         testProbeModifiesStackPointer([=] (Probe::State* context) -> void* {
    481             return reinterpret_cast<uint8_t*>(context->cpu.sp()) - offset;
     490        testProbeModifiesStackPointer([=] (Probe::Context& context) -> void* {
     491            return context.cpu.sp<uint8_t*>() - offset;
    482492        });
    483493    }
    … …  
    494504    MacroAssemblerCodeRef continuation = compile([&] (CCallHelpers& jit) {
    495505        // Validate that we reached the continuation.
    496         jit.probe([&] (Probe::State*) {
     506        jit.probe([&] (Probe::Context&) {
    497507            probeCallCount++;
    498508            continuationWasReached = true;
    … …  
    507517
    508518        // Write expected values into the registers.
    509         jit.probe([&] (Probe::State* context) {
    510             probeCallCount++;
    511             context->pc() = continuation.code().executableAddress();
     519        jit.probe([&] (Probe::Context& context) {
     520            probeCallCount++;
     521            context.cpu.pc() = continuation.code().executableAddress();
    512522        });
    513523
    … …  
    518528}
    519529
    520 struct FillStackData {
     530void testProbeModifiesStackValues()
     531{
     532    unsigned probeCallCount = 0;
    521533    CPUState originalState;
    522534    void* originalSP { nullptr };
    523535    void* newSP { nullptr };
    524536    uintptr_t modifiedFlags { 0 };
    525     MacroAssembler::SPRegisterID flagsSPR;
    526 };
    527 
    528 static void fillStack(Probe::State* context)
    529 {
    530     auto& cpu = context->cpu;
    531 
    532     FillStackData& data = *reinterpret_cast<FillStackData*>(context->initializeStackArg);
    533     CPUState& originalState = data.originalState;
    534     void*& originalSP = data.originalSP;
    535     void*& newSP = data.newSP;
    536     uintptr_t& modifiedFlags = data.modifiedFlags;
    537     MacroAssembler::SPRegisterID& flagsSPR = data.flagsSPR;
    538 
    539     CHECK_EQ(reinterpret_cast<void*>(context->initializeStackFunction), reinterpret_cast<void*>(fillStack));
    540     CHECK_EQ(cpu.sp(), newSP);
    541 
    542     // Verify that the probe has put the Probe::State out of harm's way.
    543     CHECK_EQ((reinterpret_cast<void*>(context + 1) <= cpu.sp()), true);
    544 
    545     // Verify the CPU state.
    546     for (auto id = CCallHelpers::firstRegister(); id <= CCallHelpers::lastRegister(); id = nextID(id)) {
    547         if (isFP(id)) {
    548             CHECK_EQ(cpu.gpr(id), originalState.gpr(id));
    549             continue;
    550         }
    551         if (isSpecialGPR(id))
    552             continue;
    553         CHECK_EQ(cpu.gpr(id), testWord(id));
    554     }
    555     for (auto id = CCallHelpers::firstFPRegister(); id <= CCallHelpers::lastFPRegister(); id = nextID(id))
    556         CHECK_EQ(cpu.fpr<uint64_t>(id), testWord64(id));
    557     CHECK_EQ(cpu.spr(flagsSPR), modifiedFlags);
    558 
    559     // Fill the stack with values.
    560     uintptr_t* p = reinterpret_cast<uintptr_t*>(newSP);
    561     int count = 0;
    562     while (p < reinterpret_cast<uintptr_t*>(originalSP))
    563         *p++ = testWord(count++);
    564 };
    565 
    566 void testProbeModifiesStackWithCallback()
    567 {
    568     unsigned probeCallCount = 0;
    569     FillStackData data;
    570     CPUState& originalState = data.originalState;
    571     void*& originalSP = data.originalSP;
    572     void*& newSP = data.newSP;
    573     uintptr_t& modifiedFlags = data.modifiedFlags;
    574     size_t numberOfExtraEntriesToWrite = 10; // ARM64 requires that this be 2 word aligned.
    575     MacroAssembler::SPRegisterID& flagsSPR = data.flagsSPR;
     537    size_t numberOfExtraEntriesToWrite { 10 }; // ARM64 requires that this be 2 word aligned.
    576538
    577539#if CPU(X86) || CPU(X86_64)
    578     flagsSPR = X86Registers::eflags;
     540    MacroAssembler::SPRegisterID flagsSPR = X86Registers::eflags;
    579541    uintptr_t flagsMask = 0xc5;
    580542#elif CPU(ARM_THUMB2) || CPU(ARM_TRADITIONAL)
    581     flagsSPR = ARMRegisters::apsr;
    582     uintptr_t flagsMask = 0xf0000000;
     543    MacroAssembler::SPRegisterID flagsSPR = ARMRegisters::apsr;
     544    uintptr_t flagsMask = 0xf8000000;
    583545#elif CPU(ARM64)
    584     flagsSPR = ARM64Registers::nzcv;
     546    MacroAssembler::SPRegisterID flagsSPR = ARM64Registers::nzcv;
    585547    uintptr_t flagsMask = 0xf0000000;
    586548#endif
    … …  
    590552
    591553        // Write expected values into the registers.
    592         jit.probe([&] (Probe::State* context) {
    593             auto& cpu = context->cpu;
     554        jit.probe([&] (Probe::Context& context) {
     555            auto& cpu = context.cpu;
     556            auto& stack = context.stack();
    594557            probeCallCount++;
    595558
    … …  
    609572            cpu.spr(flagsSPR) = modifiedFlags;
    610573
    611             CHECK_EQ(reinterpret_cast<void*>(context->initializeStackFunction), 0);
    612 
    613             // Prepare for initializeStack callback.
    614             context->initializeStackFunction = fillStack;
    615             context->initializeStackArg = &data;
    616 
    617574            // Ensure that we'll be writing over the regions of the stack where the Probe::State is.
    618575            originalSP = cpu.sp();
    619             newSP = reinterpret_cast<uintptr_t*>(context) - numberOfExtraEntriesToWrite;
     576            newSP = reinterpret_cast<uintptr_t*>(probeStateForContext(context)) - numberOfExtraEntriesToWrite;
    620577            cpu.sp() = newSP;
     578
     579            // Fill the stack with values.
     580            uintptr_t* p = reinterpret_cast<uintptr_t*>(newSP);
     581            int count = 0;
     582            stack.set<double>(p++, 1.23456789);
     583            if (is32Bit())
     584                p++; // On 32-bit targets, a double takes up 2 uintptr_t.
     585            while (p < reinterpret_cast<uintptr_t*>(originalSP))
     586                stack.set<uintptr_t>(p++, testWord(count++));
    621587        });
    622588
    623589        // Validate that the registers and stack have the expected values.
    624         jit.probe([&] (Probe::State* context) {
    625             auto& cpu = context->cpu;
     590        jit.probe([&] (Probe::Context& context) {
     591            auto& cpu = context.cpu;
     592            auto& stack = context.stack();
    626593            probeCallCount++;
    627594
    … …  
    638605            for (auto id = CCallHelpers::firstFPRegister(); id <= CCallHelpers::lastFPRegister(); id = nextID(id))
    639606                CHECK_EQ(cpu.fpr<uint64_t>(id), testWord64(id));
    640             CHECK_EQ(cpu.spr(flagsSPR), modifiedFlags);
     607            CHECK_EQ(cpu.spr(flagsSPR) & flagsMask, modifiedFlags & flagsMask);
    641608            CHECK_EQ(cpu.sp(), newSP);
    642609
    643             // Validate the stack with values.
     610            // Validate the stack values.
    644611            uintptr_t* p = reinterpret_cast<uintptr_t*>(newSP);
    645612            int count = 0;
     613            CHECK_EQ(stack.get<double>(p++), 1.23456789);
     614            if (is32Bit())
     615                p++; // On 32-bit targets, a double takes up 2 uintptr_t.
    646616            while (p < reinterpret_cast<uintptr_t*>(originalSP))
    647                 CHECK_EQ(*p++, testWord(count++));
     617                CHECK_EQ(stack.get<uintptr_t>(p++), testWord(count++));
    648618        });
    649619
    650620        // Restore the original state.
    651         jit.probe([&] (Probe::State* context) {
    652             auto& cpu = context->cpu;
     621        jit.probe([&] (Probe::Context& context) {
     622            auto& cpu = context.cpu;
    653623            probeCallCount++;
    654624            for (auto id = CCallHelpers::firstRegister(); id <= CCallHelpers::lastRegister(); id = nextID(id)) {
    … …  
    708678    RUN(testProbeModifiesStackPointerToNBytesBelowSP());
    709679    RUN(testProbeModifiesProgramCounter());
    710     RUN(testProbeModifiesStackWithCallback());
     680    RUN(testProbeModifiesStackValues());
    711681#endif // ENABLE(MASM_PROBE)
    712682
Note: See TracChangeset for help on using the changeset viewer.