Ignore:
Timestamp:
Nov 27, 2020, 8:31:41 PM (5 years ago)
Author:
[email protected]
Message:

[JSC] Use ARM atomic ops in wasm
​https://bugs.webkit.org/show_bug.cgi?id=219281

Reviewed by Filip Pizlo.

JSTests:

  • wasm/threads-spec-tests/atomic-signed.wast.js:
  • wasm/threads-spec-tests/resources/atomic-signed.wast:

Source/JavaScriptCore:

This patch uses ARM LSE Atomic instructions in wasm atomic operations. This includes support in MacroAssembler, offlineasm, Air and B3,
so that FTL atomic operations automatically leverage ARM LSE atomic instructions too. Later we can extend DFG JIT to use it too.

One interesting thing is that this includes a fix for cmpxchg wasm operation implementations. Unfortunately, current wasm cmpxchg ops
are not the same to ARM cas / X86 cmpxchg. For example, i64.atomic.rmw8.cmpxchg_u takes i64 expected value. And the spec requires that
we should perform i64-expected-value <cmp> loaded-zero-extended-1byte-value. For example, if the expected value is 0xffffffff_ffffff00,
and the value stored in the memory is 0x00, then the wasm op needs to fail since 0x00 is not 0xffffffff_ffffff00. But x86 and ARM
cmpxchg / cas ops behave differently since it truncates expected value to 1byte when performing 1byte cmpxchg. So we need to have a check
which performs the value is within 1byte range in this operation.

  • assembler/ARM64EAssembler.h:

(JSC::ARM64EAssembler::exoticAtomicLoadStore):
(JSC::ARM64EAssembler::exoticAtomicCAS):
(JSC::ARM64EAssembler::ldaddal):
(JSC::ARM64EAssembler::ldeoral):
(JSC::ARM64EAssembler::ldclral):
(JSC::ARM64EAssembler::ldsetal):
(JSC::ARM64EAssembler::swpal):
(JSC::ARM64EAssembler::casal):

  • assembler/MacroAssemblerARM64E.h:

(JSC::MacroAssemblerARM64E::atomicXchgAdd8):
(JSC::MacroAssemblerARM64E::atomicXchgAdd16):
(JSC::MacroAssemblerARM64E::atomicXchgAdd32):
(JSC::MacroAssemblerARM64E::atomicXchgAdd64):
(JSC::MacroAssemblerARM64E::atomicXchgXor8):
(JSC::MacroAssemblerARM64E::atomicXchgXor16):
(JSC::MacroAssemblerARM64E::atomicXchgXor32):
(JSC::MacroAssemblerARM64E::atomicXchgXor64):
(JSC::MacroAssemblerARM64E::atomicXchgOr8):
(JSC::MacroAssemblerARM64E::atomicXchgOr16):
(JSC::MacroAssemblerARM64E::atomicXchgOr32):
(JSC::MacroAssemblerARM64E::atomicXchgOr64):
(JSC::MacroAssemblerARM64E::atomicXchgClear8):
(JSC::MacroAssemblerARM64E::atomicXchgClear16):
(JSC::MacroAssemblerARM64E::atomicXchgClear32):
(JSC::MacroAssemblerARM64E::atomicXchgClear64):
(JSC::MacroAssemblerARM64E::atomicXchg8):
(JSC::MacroAssemblerARM64E::atomicXchg16):
(JSC::MacroAssemblerARM64E::atomicXchg32):
(JSC::MacroAssemblerARM64E::atomicXchg64):
(JSC::MacroAssemblerARM64E::atomicStrongCAS8):
(JSC::MacroAssemblerARM64E::atomicStrongCAS16):
(JSC::MacroAssemblerARM64E::atomicStrongCAS32):
(JSC::MacroAssemblerARM64E::atomicStrongCAS64):

  • b3/B3LowerMacros.cpp:
  • b3/B3LowerToAir.cpp:
  • b3/air/AirOpcode.opcodes:
  • b3/air/opcode_generator.rb:
  • disassembler/ARM64/A64DOpcode.cpp:

(JSC::ARM64Disassembler::A64DOpcodeLoadAtomic::format):
(JSC::ARM64Disassembler::A64DOpcodeSwapAtomic::format):
(JSC::ARM64Disassembler::A64DOpcodeCAS::format):

  • disassembler/ARM64/A64DOpcode.h:

(JSC::ARM64Disassembler::A64DOpcode::appendInstructionName):
(JSC::ARM64Disassembler::A64DOpcodeLoadAtomic::opName):
(JSC::ARM64Disassembler::A64DOpcodeLoadAtomic::rs):
(JSC::ARM64Disassembler::A64DOpcodeLoadAtomic::opc):
(JSC::ARM64Disassembler::A64DOpcodeLoadAtomic::ar):
(JSC::ARM64Disassembler::A64DOpcodeLoadAtomic::opNumber):
(JSC::ARM64Disassembler::A64DOpcodeSwapAtomic::opName):
(JSC::ARM64Disassembler::A64DOpcodeSwapAtomic::rs):
(JSC::ARM64Disassembler::A64DOpcodeSwapAtomic::ar):
(JSC::ARM64Disassembler::A64DOpcodeSwapAtomic::opNumber):
(JSC::ARM64Disassembler::A64DOpcodeCAS::opName):
(JSC::ARM64Disassembler::A64DOpcodeCAS::rs):
(JSC::ARM64Disassembler::A64DOpcodeCAS::o1):
(JSC::ARM64Disassembler::A64DOpcodeCAS::l):
(JSC::ARM64Disassembler::A64DOpcodeCAS::opNumber):

  • llint/WebAssembly.asm:
  • offlineasm/arm64.rb:
  • offlineasm/instructions.rb:
  • offlineasm/x86.rb:
  • wasm/WasmAirIRGenerator.cpp:

(JSC::Wasm::AirIRGenerator::sanitizeAtomicResult):
(JSC::Wasm::AirIRGenerator::appendGeneralAtomic):
(JSC::Wasm::AirIRGenerator::appendStrongCAS):
(JSC::Wasm::AirIRGenerator::emitAtomicLoadOp):
(JSC::Wasm::AirIRGenerator::emitAtomicStoreOp):
(JSC::Wasm::AirIRGenerator::emitAtomicBinaryRMWOp):
(JSC::Wasm::AirIRGenerator::emitAtomicCompareExchange):

  • wasm/WasmB3IRGenerator.cpp:

(JSC::Wasm::B3IRGenerator::emitAtomicCompareExchange):

File:
1 edited

Legend:

Unmodified
Added
Removed

  • trunk/Source/JavaScriptCore/offlineasm/arm64.rb

    r270208 r270214  
    231231        "[#{base.arm64Operand(:quad)}, \##{offset.value}]"
    232232    end
     233
     234    def arm64SimpleAddressOperand(kind)
     235        raise "Invalid offset #{offset.value} at #{codeOriginString}" if offset.value != 0
     236        "[#{base.arm64Operand(:quad)}]"
     237    end
    233238   
    234239    def arm64EmitLea(destination, kind)
    … …  
    373378            | node, address |
    374379            case node.opcode
    375             when "loadb", "loadbsi", "loadbsq", "storeb", /^bb/, /^btb/, /^cb/, /^tb/, "loadlinkacqb", "storecondrelb"
     380            when "loadb", "loadbsi", "loadbsq", "storeb", /^bb/, /^btb/, /^cb/, /^tb/, "loadlinkacqb", "storecondrelb", /^atomic[a-z]+b$/
    376381                size = 1
    377             when "loadh", "loadhsi", "loadhsq", "orh", "storeh", "loadlinkacqh", "storecondrelh"
     382            when "loadh", "loadhsi", "loadhsq", "orh", "storeh", "loadlinkacqh", "storecondrelh", /^atomic[a-z]+h$/
    378383                size = 2
    379384            when "loadi", "loadis", "storei", "addi", "andi", "lshifti", "muli", "negi",
    380385                "noti", "ori", "rshifti", "urshifti", "subi", "xori", /^bi/, /^bti/,
    381                 /^ci/, /^ti/, "addis", "subis", "mulis", "smulli", "leai", "loadf", "storef", "loadlinkacqi", "storecondreli"
     386                /^ci/, /^ti/, "addis", "subis", "mulis", "smulli", "leai", "loadf", "storef", "loadlinkacqi", "storecondreli",
     387                /^atomic[a-z]+i$/
    382388                size = 4
    383389            when "loadp", "storep", "loadq", "storeq", "loadd", "stored", "lshiftp", "lshiftq", "negp", "negq", "rshiftp", "rshiftq",
    384390                "urshiftp", "urshiftq", "addp", "addq", "mulp", "mulq", "andp", "andq", "orp", "orq", "subp", "subq", "xorp", "xorq", "addd",
    385391                "divd", "subd", "muld", "sqrtd", /^bp/, /^bq/, /^btp/, /^btq/, /^cp/, /^cq/, /^tp/, /^tq/, /^bd/,
    386                 "jmp", "call", "leap", "leaq", "loadlinkacqq", "storecondrelq"
     392                "jmp", "call", "leap", "leaq", "loadlinkacqq", "storecondrelq", /^atomic[a-z]+q$/
    387393                size = $currentSettings["ADDRESS64"] ? 8 : 4
    388394            else
    … …  
    443449                not (address.is_a? Address and address.offset.value < 0)
    444450            when /^lea/
     451                true
     452            when /^atomic/
    445453                true
    446454            else
    … …  
    766774        when "negq"
    767775            $asm.puts "sub #{operands[0].arm64Operand(:quad)}, xzr, #{operands[0].arm64Operand(:quad)}"
     776        when "notq"
     777            $asm.puts "mvn #{operands[0].arm64Operand(:quad)}, #{operands[0].arm64Operand(:quad)}"
    768778        when "loadi"
    769779            emitARM64Access("ldr", "ldur", operands[1], operands[0], :word)
    … …  
    13281338        when "storecondrelq"
    13291339            $asm.puts "stlxr #{operands[0].arm64Operand(:word)}, #{operands[1].arm64Operand(:quad)}, #{operands[2].arm64Operand(:quad)}"
     1340        when "atomicxchgaddb"
     1341            $asm.puts "ldaddalb #{operands[0].arm64Operand(:word)}, #{operands[2].arm64Operand(:word)}, #{operands[1].arm64SimpleAddressOperand(:word)}"
     1342        when "atomicxchgaddh"
     1343            $asm.puts "ldaddalh #{operands[0].arm64Operand(:word)}, #{operands[2].arm64Operand(:word)}, #{operands[1].arm64SimpleAddressOperand(:word)}"
     1344        when "atomicxchgaddi"
     1345            $asm.puts "ldaddal #{operands[0].arm64Operand(:word)}, #{operands[2].arm64Operand(:word)}, #{operands[1].arm64SimpleAddressOperand(:word)}"
     1346        when "atomicxchgaddq"
     1347            $asm.puts "ldaddal #{operands[0].arm64Operand(:quad)}, #{operands[2].arm64Operand(:quad)}, #{operands[1].arm64SimpleAddressOperand(:quad)}"
     1348        when "atomicxchgclearb"
     1349            $asm.puts "ldclralb #{operands[0].arm64Operand(:word)}, #{operands[2].arm64Operand(:word)}, #{operands[1].arm64SimpleAddressOperand(:word)}"
     1350        when "atomicxchgclearh"
     1351            $asm.puts "ldclralh #{operands[0].arm64Operand(:word)}, #{operands[2].arm64Operand(:word)}, #{operands[1].arm64SimpleAddressOperand(:word)}"
     1352        when "atomicxchgcleari"
     1353            $asm.puts "ldclral #{operands[0].arm64Operand(:word)}, #{operands[2].arm64Operand(:word)}, #{operands[1].arm64SimpleAddressOperand(:word)}"
     1354        when "atomicxchgclearq"
     1355            $asm.puts "ldclral #{operands[0].arm64Operand(:quad)}, #{operands[2].arm64Operand(:quad)}, #{operands[1].arm64SimpleAddressOperand(:quad)}"
     1356        when "atomicxchgorb"
     1357            $asm.puts "ldsetalb #{operands[0].arm64Operand(:word)}, #{operands[2].arm64Operand(:word)}, #{operands[1].arm64SimpleAddressOperand(:word)}"
     1358        when "atomicxchgorh"
     1359            $asm.puts "ldsetalh #{operands[0].arm64Operand(:word)}, #{operands[2].arm64Operand(:word)}, #{operands[1].arm64SimpleAddressOperand(:word)}"
     1360        when "atomicxchgori"
     1361            $asm.puts "ldsetal #{operands[0].arm64Operand(:word)}, #{operands[2].arm64Operand(:word)}, #{operands[1].arm64SimpleAddressOperand(:word)}"
     1362        when "atomicxchgorq"
     1363            $asm.puts "ldsetal #{operands[0].arm64Operand(:quad)}, #{operands[2].arm64Operand(:quad)}, #{operands[1].arm64SimpleAddressOperand(:quad)}"
     1364        when "atomicxchgxorb"
     1365            $asm.puts "ldeoralb #{operands[0].arm64Operand(:word)}, #{operands[2].arm64Operand(:word)}, #{operands[1].arm64SimpleAddressOperand(:word)}"
     1366        when "atomicxchgxorh"
     1367            $asm.puts "ldeoralh #{operands[0].arm64Operand(:word)}, #{operands[2].arm64Operand(:word)}, #{operands[1].arm64SimpleAddressOperand(:word)}"
     1368        when "atomicxchgxori"
     1369            $asm.puts "ldeoral #{operands[0].arm64Operand(:word)}, #{operands[2].arm64Operand(:word)}, #{operands[1].arm64SimpleAddressOperand(:word)}"
     1370        when "atomicxchgxorq"
     1371            $asm.puts "ldeoral #{operands[0].arm64Operand(:quad)}, #{operands[2].arm64Operand(:quad)}, #{operands[1].arm64SimpleAddressOperand(:quad)}"
     1372        when "atomicxchgb"
     1373            $asm.puts "swpalb #{operands[0].arm64Operand(:word)}, #{operands[2].arm64Operand(:word)}, #{operands[1].arm64SimpleAddressOperand(:word)}"
     1374        when "atomicxchgh"
     1375            $asm.puts "swpalh #{operands[0].arm64Operand(:word)}, #{operands[2].arm64Operand(:word)}, #{operands[1].arm64SimpleAddressOperand(:word)}"
     1376        when "atomicxchgi"
     1377            $asm.puts "swpal #{operands[0].arm64Operand(:word)}, #{operands[2].arm64Operand(:word)}, #{operands[1].arm64SimpleAddressOperand(:word)}"
     1378        when "atomicxchgq"
     1379            $asm.puts "swpal #{operands[0].arm64Operand(:quad)}, #{operands[2].arm64Operand(:quad)}, #{operands[1].arm64SimpleAddressOperand(:quad)}"
     1380        when "atomicweakcasb"
     1381            $asm.puts "casalb #{operands[0].arm64Operand(:word)}, #{operands[1].arm64Operand(:word)}, #{operands[2].arm64SimpleAddressOperand(:word)}"
     1382        when "atomicweakcash"
     1383            $asm.puts "casalh #{operands[0].arm64Operand(:word)}, #{operands[1].arm64Operand(:word)}, #{operands[2].arm64SimpleAddressOperand(:word)}"
     1384        when "atomicweakcasi"
     1385            $asm.puts "casal #{operands[0].arm64Operand(:word)}, #{operands[1].arm64Operand(:word)}, #{operands[2].arm64SimpleAddressOperand(:word)}"
     1386        when "atomicweakcasq"
     1387            $asm.puts "casal #{operands[0].arm64Operand(:quad)}, #{operands[1].arm64Operand(:quad)}, #{operands[2].arm64SimpleAddressOperand(:quad)}"
     1388        when "atomicloadb"
     1389            $asm.puts "ldarb #{operands[1].arm64Operand(:word)}, #{operands[0].arm64SimpleAddressOperand(:word)}"
     1390        when "atomicloadh"
     1391            $asm.puts "ldarh #{operands[1].arm64Operand(:word)}, #{operands[0].arm64SimpleAddressOperand(:word)}"
     1392        when "atomicloadi"
     1393            $asm.puts "ldar #{operands[1].arm64Operand(:word)}, #{operands[0].arm64SimpleAddressOperand(:word)}"
     1394        when "atomicloadq"
     1395            $asm.puts "ldar #{operands[1].arm64Operand(:quad)}, #{operands[0].arm64SimpleAddressOperand(:quad)}"
    13301396        else
    13311397            lowerDefault
Note: See TracChangeset for help on using the changeset viewer.