diff options
69 files changed, 1227 insertions, 791 deletions
diff --git a/docs/LangRef.rst b/docs/LangRef.rst index df322396702..97ee0e7d358 100644 --- a/docs/LangRef.rst +++ b/docs/LangRef.rst @@ -2209,12 +2209,21 @@ For a simpler introduction to the ordering constraints, see the same address in this global order. This corresponds to the C++0x/C1x ``memory_order_seq_cst`` and Java volatile. -.. _singlethread: +.. _syncscope: -If an atomic operation is marked ``singlethread``, it only *synchronizes -with* or participates in modification and seq\_cst total orderings with -other operations running in the same thread (for example, in signal -handlers). +If an atomic operation is marked ``syncscope("singlethread")``, it only +*synchronizes with* and only participates in the seq\_cst total orderings of +other operations running in the same thread (for example, in signal handlers). + +If an atomic operation is marked ``syncscope("<target-scope>")``, where +``<target-scope>`` is a target specific synchronization scope, then it is target +dependent if it *synchronizes with* and participates in the seq\_cst total +orderings of other operations. + +Otherwise, an atomic operation that is not marked ``syncscope("singlethread")`` +or ``syncscope("<target-scope>")`` *synchronizes with* and participates in the +seq\_cst total orderings of other operations that are not marked +``syncscope("singlethread")`` or ``syncscope("<target-scope>")``. .. _fastmath: @@ -7380,7 +7389,7 @@ Syntax: :: <result> = load [volatile] <ty>, <ty>* <pointer>[, align <alignment>][, !nontemporal !<index>][, !invariant.load !<index>][, !invariant.group !<index>][, !nonnull !<index>][, !dereferenceable !<deref_bytes_node>][, !dereferenceable_or_null !<deref_bytes_node>][, !align !<align_node>] - <result> = load atomic [volatile] <ty>, <ty>* <pointer> [singlethread] <ordering>, align <alignment> [, !invariant.group !<index>] + <result> = load atomic [volatile] <ty>, <ty>* <pointer> [syncscope("<target-scope>")] <ordering>, align <alignment> [, !invariant.group !<index>] !<index> = !{ i32 1 } !<deref_bytes_node> = !{i64 <dereferenceable_bytes>} !<align_node> = !{ i64 <value_alignment> } @@ -7401,14 +7410,14 @@ modify the number or order of execution of this ``load`` with other :ref:`volatile operations <volatile>`. If the ``load`` is marked as ``atomic``, it takes an extra :ref:`ordering -<ordering>` and optional ``singlethread`` argument. The ``release`` and -``acq_rel`` orderings are not valid on ``load`` instructions. Atomic loads -produce :ref:`defined <memmodel>` results when they may see multiple atomic -stores. The type of the pointee must be an integer, pointer, or floating-point -type whose bit width is a power of two greater than or equal to eight and less -than or equal to a target-specific size limit. ``align`` must be explicitly -specified on atomic loads, and the load has undefined behavior if the alignment -is not set to a value which is at least the size in bytes of the +<ordering>` and optional ``syncscope("<target-scope>")`` argument. The +``release`` and ``acq_rel`` orderings are not valid on ``load`` instructions. +Atomic loads produce :ref:`defined <memmodel>` results when they may see +multiple atomic stores. The type of the pointee must be an integer, pointer, or +floating-point type whose bit width is a power of two greater than or equal to +eight and less than or equal to a target-specific size limit. ``align`` must be +explicitly specified on atomic loads, and the load has undefined behavior if the +alignment is not set to a value which is at least the size in bytes of the pointee. ``!nontemporal`` does not have any defined semantics for atomic loads. The optional constant ``align`` argument specifies the alignment of the @@ -7509,7 +7518,7 @@ Syntax: :: store [volatile] <ty> <value>, <ty>* <pointer>[, align <alignment>][, !nontemporal !<index>][, !invariant.group !<index>] ; yields void - store atomic [volatile] <ty> <value>, <ty>* <pointer> [singlethread] <ordering>, align <alignment> [, !invariant.group !<index>] ; yields void + store atomic [volatile] <ty> <value>, <ty>* <pointer> [syncscope("<target-scope>")] <ordering>, align <alignment> [, !invariant.group !<index>] ; yields void Overview: """"""""" @@ -7529,14 +7538,14 @@ allowed to modify the number or order of execution of this ``store`` with other structural type <t_opaque>`) can be stored. If the ``store`` is marked as ``atomic``, it takes an extra :ref:`ordering -<ordering>` and optional ``singlethread`` argument. The ``acquire`` and -``acq_rel`` orderings aren't valid on ``store`` instructions. Atomic loads -produce :ref:`defined <memmodel>` results when they may see multiple atomic -stores. The type of the pointee must be an integer, pointer, or floating-point -type whose bit width is a power of two greater than or equal to eight and less -than or equal to a target-specific size limit. ``align`` must be explicitly -specified on atomic stores, and the store has undefined behavior if the -alignment is not set to a value which is at least the size in bytes of the +<ordering>` and optional ``syncscope("<target-scope>")`` argument. The +``acquire`` and ``acq_rel`` orderings aren't valid on ``store`` instructions. +Atomic loads produce :ref:`defined <memmodel>` results when they may see +multiple atomic stores. The type of the pointee must be an integer, pointer, or +floating-point type whose bit width is a power of two greater than or equal to +eight and less than or equal to a target-specific size limit. ``align`` must be +explicitly specified on atomic stores, and the store has undefined behavior if +the alignment is not set to a value which is at least the size in bytes of the pointee. ``!nontemporal`` does not have any defined semantics for atomic stores. The optional constant ``align`` argument specifies the alignment of the @@ -7597,7 +7606,7 @@ Syntax: :: - fence [singlethread] <ordering> ; yields void + fence [syncscope("<target-scope>")] <ordering> ; yields void Overview: """"""""" @@ -7631,17 +7640,17 @@ A ``fence`` which has ``seq_cst`` ordering, in addition to having both ``acquire`` and ``release`` semantics specified above, participates in the global program order of other ``seq_cst`` operations and/or fences. -The optional ":ref:`singlethread <singlethread>`" argument specifies -that the fence only synchronizes with other fences in the same thread. -(This is useful for interacting with signal handlers.) +A ``fence`` instruction can also take an optional +":ref:`syncscope <syncscope>`" argument. Example: """""""" .. code-block:: llvm - fence acquire ; yields void - fence singlethread seq_cst ; yields void + fence acquire ; yields void + fence syncscope("singlethread") seq_cst ; yields void + fence syncscope("agent") seq_cst ; yields void .. _i_cmpxchg: @@ -7653,7 +7662,7 @@ Syntax: :: - cmpxchg [weak] [volatile] <ty>* <pointer>, <ty> <cmp>, <ty> <new> [singlethread] <success ordering> <failure ordering> ; yields { ty, i1 } + cmpxchg [weak] [volatile] <ty>* <pointer>, <ty> <cmp>, <ty> <new> [syncscope("<target-scope>")] <success ordering> <failure ordering> ; yields { ty, i1 } Overview: """"""""" @@ -7682,10 +7691,8 @@ must be at least ``monotonic``, the ordering constraint on failure must be no stronger than that on success, and the failure ordering cannot be either ``release`` or ``acq_rel``. -The optional "``singlethread``" argument declares that the ``cmpxchg`` -is only atomic with respect to code (usually signal handlers) running in -the same thread as the ``cmpxchg``. Otherwise the cmpxchg is atomic with -respect to all other code in the system. +A ``cmpxchg`` instruction can also take an optional +":ref:`syncscope <syncscope>`" argument. The pointer passed into cmpxchg must have alignment greater than or equal to the size in memory of the operand. @@ -7739,7 +7746,7 @@ Syntax: :: - atomicrmw [volatile] <operation> <ty>* <pointer>, <ty> <value> [singlethread] <ordering> ; yields ty + atomicrmw [volatile] <operation> <ty>* <pointer>, <ty> <value> [syncscope("<target-scope>")] <ordering> ; yields ty Overview: """"""""" @@ -7773,6 +7780,9 @@ be a pointer to that type. If the ``atomicrmw`` is marked as order of execution of this ``atomicrmw`` with other :ref:`volatile operations <volatile>`. +A ``atomicrmw`` instruction can also take an optional +":ref:`syncscope <syncscope>`" argument. + Semantics: """""""""" diff --git a/include/llvm/Bitcode/LLVMBitCodes.h b/include/llvm/Bitcode/LLVMBitCodes.h index 5435e48ff42..3777f956cf2 100644 --- a/include/llvm/Bitcode/LLVMBitCodes.h +++ b/include/llvm/Bitcode/LLVMBitCodes.h @@ -59,6 +59,8 @@ enum BlockIDs { FULL_LTO_GLOBALVAL_SUMMARY_BLOCK_ID, SYMTAB_BLOCK_ID, + + SYNC_SCOPE_NAMES_BLOCK_ID, }; /// Identification block contains a string that describes the producer details, @@ -172,6 +174,10 @@ enum OperandBundleTagCode { OPERAND_BUNDLE_TAG = 1, // TAG: [strchr x N] }; +enum SyncScopeNameCode { + SYNC_SCOPE_NAME = 1, +}; + // Value symbol table codes. enum ValueSymtabCodes { VST_CODE_ENTRY = 1, // VST_ENTRY: [valueid, namechar x N] @@ -404,12 +410,6 @@ enum AtomicOrderingCodes { ORDERING_SEQCST = 6 }; -/// Encoded SynchronizationScope values. -enum AtomicSynchScopeCodes { - SYNCHSCOPE_SINGLETHREAD = 0, - SYNCHSCOPE_CROSSTHREAD = 1 -}; - /// Markers and flags for call instruction. enum CallMarkersFlags { CALL_TAIL = 0, diff --git a/include/llvm/CodeGen/MachineFunction.h b/include/llvm/CodeGen/MachineFunction.h index f67da7b01c5..19173fa39bd 100644 --- a/include/llvm/CodeGen/MachineFunction.h +++ b/include/llvm/CodeGen/MachineFunction.h @@ -650,7 +650,7 @@ public: MachinePointerInfo PtrInfo, MachineMemOperand::Flags f, uint64_t s, unsigned base_alignment, const AAMDNodes &AAInfo = AAMDNodes(), const MDNode *Ranges = nullptr, - SynchronizationScope SynchScope = CrossThread, + SyncScope::ID SSID = SyncScope::System, AtomicOrdering Ordering = AtomicOrdering::NotAtomic, AtomicOrdering FailureOrdering = AtomicOrdering::NotAtomic); diff --git a/include/llvm/CodeGen/MachineMemOperand.h b/include/llvm/CodeGen/MachineMemOperand.h index 78adce507b8..a2b13d9369c 100644 --- a/include/llvm/CodeGen/MachineMemOperand.h +++ b/include/llvm/CodeGen/MachineMemOperand.h @@ -124,8 +124,8 @@ public: private: /// Atomic information for this memory operation. struct MachineAtomicInfo { - /// Synchronization scope for this memory operation. - unsigned SynchScope : 1; // enum SynchronizationScope + /// Synchronization scope ID for this memory operation. + unsigned SSID : 8; // SyncScope::ID /// Atomic ordering requirements for this memory operation. For cmpxchg /// atomic operations, atomic ordering requirements when store occurs. unsigned Ordering : 4; // enum AtomicOrdering @@ -152,7 +152,7 @@ public: unsigned base_alignment, const AAMDNodes &AAInfo = AAMDNodes(), const MDNode *Ranges = nullptr, - SynchronizationScope SynchScope = CrossThread, + SyncScope::ID SSID = SyncScope::System, AtomicOrdering Ordering = AtomicOrdering::NotAtomic, AtomicOrdering FailureOrdering = AtomicOrdering::NotAtomic); @@ -202,9 +202,9 @@ public: /// Return the range tag for the memory reference. const MDNode *getRanges() const { return Ranges; } - /// Return the synchronization scope for this memory operation. - SynchronizationScope getSynchScope() const { - return static_cast<SynchronizationScope>(AtomicInfo.SynchScope); + /// Returns the synchronization scope ID for this memory operation. + SyncScope::ID getSyncScopeID() const { + return static_cast<SyncScope::ID>(AtomicInfo.SSID); } /// Return the atomic ordering requirements for this memory operation. For diff --git a/include/llvm/CodeGen/SelectionDAG.h b/include/llvm/CodeGen/SelectionDAG.h index f3f3003b7e2..55a23c3cca9 100644 --- a/include/llvm/CodeGen/SelectionDAG.h +++ b/include/llvm/CodeGen/SelectionDAG.h @@ -927,7 +927,7 @@ public: SDValue Cmp, SDValue Swp, MachinePointerInfo PtrInfo, unsigned Alignment, AtomicOrdering SuccessOrdering, AtomicOrdering FailureOrdering, - SynchronizationScope SynchScope); + SyncScope::ID SSID); SDValue getAtomicCmpSwap(unsigned Opcode, const SDLoc &dl, EVT MemVT, SDVTList VTs, SDValue Chain, SDValue Ptr, SDValue Cmp, SDValue Swp, MachineMemOperand *MMO); @@ -937,7 +937,7 @@ public: SDValue getAtomic(unsigned Opcode, const SDLoc &dl, EVT MemVT, SDValue Chain, SDValue Ptr, SDValue Val, const Value *PtrVal, unsigned Alignment, AtomicOrdering Ordering, - SynchronizationScope SynchScope); + SyncScope::ID SSID); SDValue getAtomic(unsigned Opcode, const SDLoc &dl, EVT MemVT, SDValue Chain, SDValue Ptr, SDValue Val, MachineMemOperand *MMO); diff --git a/include/llvm/CodeGen/SelectionDAGNodes.h b/include/llvm/CodeGen/SelectionDAGNodes.h index 45ff654fae2..db42fb6c170 100644 --- a/include/llvm/CodeGen/SelectionDAGNodes.h +++ b/include/llvm/CodeGen/SelectionDAGNodes.h @@ -1213,8 +1213,8 @@ public: /// Returns the Ranges that describes the dereference. const MDNode *getRanges() const { return MMO->getRanges(); } - /// Return the synchronization scope for this memory operation. - SynchronizationScope getSynchScope() const { return MMO->getSynchScope(); } + /// Returns the synchronization scope ID for this memory operation. + SyncScope::ID getSyncScopeID() const { return MMO->getSyncScopeID(); } /// Return the atomic ordering requirements for this memory operation. For /// cmpxchg atomic operations, return the atomic ordering requirements when diff --git a/include/llvm/IR/IRBuilder.h b/include/llvm/IR/IRBuilder.h index 8f7b68fbd19..5344a93efb3 100644 --- a/include/llvm/IR/IRBuilder.h +++ b/include/llvm/IR/IRBuilder.h @@ -1203,22 +1203,22 @@ public: return SI; } FenceInst *CreateFence(AtomicOrdering Ordering, - SynchronizationScope SynchScope = CrossThread, + SyncScope::ID SSID = SyncScope::System, const Twine &Name = "") { - return Insert(new FenceInst(Context, Ordering, SynchScope), Name); + return Insert(new FenceInst(Context, Ordering, SSID), Name); } AtomicCmpXchgInst * CreateAtomicCmpXchg(Value *Ptr, Value *Cmp, Value *New, AtomicOrdering SuccessOrdering, AtomicOrdering FailureOrdering, - SynchronizationScope SynchScope = CrossThread) { + SyncScope::ID SSID = SyncScope::System) { return Insert(new AtomicCmpXchgInst(Ptr, Cmp, New, SuccessOrdering, - FailureOrdering, SynchScope)); + FailureOrdering, SSID)); } AtomicRMWInst *CreateAtomicRMW(AtomicRMWInst::BinOp Op, Value *Ptr, Value *Val, AtomicOrdering Ordering, - SynchronizationScope SynchScope = CrossThread) { - return Insert(new AtomicRMWInst(Op, Ptr, Val, Ordering, SynchScope)); + SyncScope::ID SSID = SyncScope::System) { + return Insert(new AtomicRMWInst(Op, Ptr, Val, Ordering, SSID)); } Value *CreateGEP(Value *Ptr, ArrayRef<Value *> IdxList, const Twine &Name = "") { diff --git a/include/llvm/IR/Instructions.h b/include/llvm/IR/Instructions.h index b6e5c887a9c..60ae98869e5 100644 --- a/include/llvm/IR/Instructions.h +++ b/include/llvm/IR/Instructions.h @@ -52,11 +52,6 @@ class ConstantInt; class DataLayout; class LLVMContext; -enum SynchronizationScope { - SingleThread = 0, - CrossThread = 1 -}; - //===----------------------------------------------------------------------===// // AllocaInst Class //===----------------------------------------------------------------------===// @@ -195,17 +190,16 @@ public: LoadInst(Value *Ptr, const Twine &NameStr, bool isVolatile, unsigned Align, BasicBlock *InsertAtEnd); LoadInst(Value *Ptr, const Twine &NameStr, bool isVolatile, unsigned Align, - AtomicOrdering Order, SynchronizationScope SynchScope = CrossThread, + AtomicOrdering Order, SyncScope::ID SSID = SyncScope::System, Instruction *InsertBefore = nullptr) : LoadInst(cast<PointerType>(Ptr->getType())->getElementType(), Ptr, - NameStr, isVolatile, Align, Order, SynchScope, InsertBefore) {} + NameStr, isVolatile, Align, Order, SSID, InsertBefore) {} LoadInst(Type *Ty, Value *Ptr, const Twine &NameStr, bool isVolatile, unsigned Align, AtomicOrdering Order, - SynchronizationScope SynchScope = CrossThread, + SyncScope::ID SSID = SyncScope::System, Instruction *InsertBefore = nullptr); LoadInst(Value *Ptr, const Twine &NameStr, bool isVolatile, - unsigned Align, AtomicOrdering Order, - SynchronizationScope SynchScope, + unsigned Align, AtomicOrdering Order, SyncScope::ID SSID, BasicBlock *InsertAtEnd); LoadInst(Value *Ptr, const char *NameStr, Instruction *InsertBefore); LoadInst(Value *Ptr, const char *NameStr, BasicBlock *InsertAtEnd); @@ -235,34 +229,34 @@ public: void setAlignment(unsigned Align); - /// Returns the ordering effect of this fence. + /// Returns the ordering constraint of this load instruction. AtomicOrdering getOrdering() const { return AtomicOrdering((getSubclassDataFromInstruction() >> 7) & 7); } - /// Set the ordering constraint on this load. May not be Release or - /// AcquireRelease. + /// Sets the ordering constraint of this load instruction. May not be Release + /// or AcquireRelease. void setOrdering(AtomicOrdering Ordering) { setInstructionSubclassData((getSubclassDataFromInstruction() & ~(7 << 7)) | ((unsigned)Ordering << 7)); } - SynchronizationScope getSynchScope() const { - return SynchronizationScope((getSubclassDataFromInstruction() >> 6) & 1); + /// Returns the synchronization scope ID of this load instruction. + SyncScope::ID getSyncScopeID() const { + return SSID; } - /// Specify whether this load is ordered with respect to all - /// concurrently executing threads, or only with respect to signal handlers - /// executing in the same thread. - void setSynchScope(SynchronizationScope xthread) { - setInstructionSubclassData((getSubclassDataFromInstruction() & ~(1 << 6)) | - (xthread << 6)); + /// Sets the synchronization scope ID of this load instruction. + void setSyncScopeID(SyncScope::ID SSID) { + this->SSID = SSID; } + /// Sets the ordering constraint and the synchronization scope ID of this load + /// instruction. void setAtomic(AtomicOrdering Ordering, - SynchronizationScope SynchScope = CrossThread) { + SyncScope::ID SSID = SyncScope::System) { setOrdering(Ordering); - setSynchScope(SynchScope); + setSyncScopeID(SSID); } bool isSimple() const { return !isAtomic() && !isVolatile(); } @@ -297,6 +291,11 @@ private: void setInstructionSubclassData(unsigned short D) { Instruction::setInstructionSubclassData(D); } + + /// The synchronization scope ID of this load instruction. Not quite enough + /// room in SubClassData for everything, so synchronization scope ID gets its + /// own field. + SyncScope::ID SSID; }; //===----------------------------------------------------------------------===// @@ -325,11 +324,10 @@ public: unsigned Align, BasicBlock *InsertAtEnd); StoreInst(Value *Val, Value *Ptr, bool isVolatile, unsigned Align, AtomicOrdering Order, - SynchronizationScope SynchScope = CrossThread, + SyncScope::ID SSID = SyncScope::System, Instruction *InsertBefore = nullptr); StoreInst(Value *Val, Value *Ptr, bool isVolatile, - unsigned Align, AtomicOrdering Order, - SynchronizationScope SynchScope, + unsigned Align, AtomicOrdering Order, SyncScope::ID SSID, BasicBlock *InsertAtEnd); // allocate space for exactly two operands @@ -356,34 +354,34 @@ public: void setAlignment(unsigned Align); - /// Returns the ordering effect of this store. + /// Returns the ordering constraint of this store instruction. AtomicOrdering getOrdering() const { return AtomicOrdering((getSubclassDataFromInstruction() >> 7) & 7); } - /// Set the ordering constraint on this store. May not be Acquire or - /// AcquireRelease. + /// Sets the ordering constraint of this store instruction. May not be + /// Acquire or AcquireRelease. void setOrdering(AtomicOrdering Ordering) { setInstructionSubclassData((getSubclassDataFromInstruction() & ~(7 << 7)) | ((unsigned)Ordering << 7)); } - SynchronizationScope getSynchScope() const { - return SynchronizationScope((getSubclassDataFromInstruction() >> 6) & 1); + /// Returns the synchronization scope ID of this store instruction. + SyncScope::ID getSyncScopeID() const { + return SSID; } - /// Specify whether this store instruction is ordered with respect to all - /// concurrently executing threads, or only with respect to signal handlers - /// executing in the same thread. - void setSynchScope(SynchronizationScope xthread) { - setInstructionSubclassData((getSubclassDataFromInstruction() & ~(1 << 6)) | - (xthread << 6)); + /// Sets the synchronization scope ID of this store instruction. + void setSyncScopeID(SyncScope::ID SSID) { + this->SSID = SSID; } + /// Sets the ordering constraint and the synchronization scope ID of this + /// store instruction. void setAtomic(AtomicOrdering Ordering, - SynchronizationScope SynchScope = CrossThread) { + SyncScope::ID SSID = SyncScope::System) { setOrdering(Ordering); - setSynchScope(SynchScope); + setSyncScopeID(SSID); } bool isSimple() const { return !isAtomic() && !isVolatile(); } @@ -421,6 +419,11 @@ private: void setInstructionSubclassData(unsigned short D) { Instruction::setInstructionSubclassData(D); } + + /// The synchronization scope ID of this store instruction. Not quite enough + /// room in SubClassData for everything, so synchronization scope ID gets its + /// own field. + SyncScope::ID SSID; }; template <> @@ -435,7 +438,7 @@ DEFINE_TRANSPARENT_OPERAND_ACCESSORS(StoreInst, Value) /// An instruction for ordering other memory operations. class FenceInst : public Instruction { - void Init(AtomicOrdering Ordering, SynchronizationScope SynchScope); + void Init(AtomicOrdering Ordering, SyncScope::ID SSID); protected: // Note: Instruction needs to be a friend here to call cloneImpl. @@ -447,10 +450,9 @@ public: // Ordering may only be Acquire, Release, AcquireRelease, or // SequentiallyConsistent. FenceInst(LLVMContext &C, AtomicOrdering Ordering, - SynchronizationScope SynchScope = CrossThread, + SyncScope::ID SSID = SyncScope::System, Instruction *InsertBefore = nullptr); - FenceInst(LLVMContext &C, AtomicOrdering Ordering, - SynchronizationScope SynchScope, + FenceInst(LLVMContext &C, AtomicOrdering Ordering, SyncScope::ID SSID, BasicBlock *InsertAtEnd); // allocate space for exactly zero operands @@ -458,28 +460,26 @@ public: return User::operator new(s, 0); } - /// Returns the ordering effect of this fence. + /// Returns the ordering constraint of this fence instruction. AtomicOrdering getOrdering() const { return AtomicOrdering(getSubclassDataFromInstruction() >> 1); } - /// Set the ordering constraint on this fence. May only be Acquire, Release, - /// AcquireRelease, or SequentiallyConsistent. + /// Sets the ordering constraint of this fence instruction. May only be + /// Acquire, Release, AcquireRelease, or SequentiallyConsistent. void setOrdering(AtomicOrdering Ordering) { setInstructionSubclassData((getSubclassDataFromInstruction() & 1) | ((unsigned)Ordering << 1)); } - SynchronizationScope getSynchScope() const { - return SynchronizationScope(getSubclassDataFromInstruction() & 1); + /// Returns the synchronization scope ID of this fence instruction. + SyncScope::ID getSyncScopeID() const { + return SSID; } - /// Specify whether this fence orders other operations with respect to all - /// concurrently executing threads, or only with respect to signal handlers - /// executing in the same thread. - void setSynchScope(SynchronizationScope xthread) { - setInstructionSubclassData((getSubclassDataFromInstruction() & ~1) | - xthread); + /// Sets the synchronization scope ID of this fence instruction. + void setSyncScopeID(SyncScope::ID SSID) { + this->SSID = SSID; } // Methods for support type inquiry through isa, cast, and dyn_cast: @@ -496,6 +496,11 @@ private: void setInstructionSubclassData(unsigned short D) { Instruction::setInstructionSubclassData(D); } + + /// The synchronization scope ID of this fence instruction. Not quite enough + /// room in SubClassData for everything, so synchronization scope ID gets its + /// own field. + SyncScope::ID SSID; }; //===----------------------------------------------------------------------===// @@ -509,7 +514,7 @@ private: class AtomicCmpXchgInst : public Instruction { void Init(Value *Ptr, Value *Cmp, Value *NewVal, AtomicOrdering SuccessOrdering, AtomicOrdering FailureOrdering, - SynchronizationScope SynchScope); + SyncScope::ID SSID); protected: // Note: Instruction needs to be a friend here to call cloneImpl. @@ -521,13 +526,11 @@ public: AtomicCmpXchgInst(Value *Ptr, Value *Cmp, Value *NewVal, AtomicOrdering SuccessOrdering, AtomicOrdering FailureOrdering, - SynchronizationScope SynchScope, - Instruction *InsertBefore = nullptr); + SyncScope::ID SSID, Instruction *InsertBefore = nullptr); AtomicCmpXchgInst(Value *Ptr, Value *Cmp, Value *NewVal, AtomicOrdering SuccessOrdering, AtomicOrdering FailureOrdering, - SynchronizationScope SynchScope, - BasicBlock *InsertAtEnd); + SyncScope::ID SSID, BasicBlock *InsertAtEnd); // allocate space for exactly three operands void *operator new(size_t s) { @@ -561,7 +564,12 @@ public: /// Transparently provide more efficient getOperand methods. DECLARE_TRANSPARENT_OPERAND_ACCESSORS(Value); - /// Set the ordering constraint on this cmpxchg. + /// Returns the success ordering constraint of this cmpxchg instruction. + AtomicOrdering getSuccessOrdering() const { + return AtomicOrdering((getSubclassDataFromInstruction() >> 2) & 7); + } + + /// Sets the success ordering constraint of this cmpxchg instruction. void setSuccessOrdering(AtomicOrdering Ordering) { assert(Ordering != AtomicOrdering::NotAtomic && "CmpXchg instructions can only be atomic."); @@ -569,6 +577,12 @@ public: ((unsigned)Ordering << 2)); } + /// Returns the failure ordering constraint of this cmpxchg instruction. + AtomicOrdering getFailureOrdering() const { + return AtomicOrdering((getSubclassDataFromInstruction() >> 5) & 7); + } + + /// Sets the failure ordering constraint of this cmpxchg instruction. void setFailureOrdering(AtomicOrdering Ordering) { assert(Ordering != AtomicOrdering::NotAtomic && "CmpXchg instructions can only be atomic."); @@ -576,28 +590,14 @@ public: ((unsigned)Ordering << 5)); } - /// Specify whether this cmpxchg is atomic and orders other operations with - /// respect to all concurrently executing threads, or only with respect to - /// signal handlers executing in the same thread. - void setSynchScope(SynchronizationScope SynchScope) { - setInstructionSubclassData((getSubclassDataFromInstruction() & ~2) | - (SynchScope << 1)); - } - - /// Returns the ordering constraint on this cmpxchg. - AtomicOrdering getSuccessOrdering() const { - return AtomicOrdering((getSubclassDataFromInstruction() >> 2) & 7); - } - - /// Returns the ordering constraint on this cmpxchg. - AtomicOrdering getFailureOrdering() const { - return AtomicOrdering((getSubclassDataFromInstruction() >> 5) & 7); + /// Returns the synchronization scope ID of this cmpxchg instruction. + SyncScope::ID getSyncScopeID() const { + return SSID; } - /// Returns whether this cmpxchg is atomic between threads or only within a - /// single thread. - SynchronizationScope getSynchScope() const { - return SynchronizationScope((getSubclassDataFromInstruction() & 2) >> 1); + /// Sets the synchronization scope ID of this cmpxchg instruction. + void setSyncScopeID(SyncScope::ID SSID) { + this->SSID = SSID; } Value *getPointerOperand() { return getOperand(0); } @@ -652,6 +652,11 @@ private: void setInstructionSubclassData(unsigned short D) { Instruction::setInstructionSubclassData(D); } + + /// The synchronization scope ID of this cmpxchg instruction. Not quite + /// enough room in SubClassData for everything, so synchronization scope ID + /// gets its own field. + SyncScope::ID SSID; }; template <> @@ -711,10 +716,10 @@ public: }; AtomicRMWInst(BinOp Operation, Value *Ptr, Value *Val, - AtomicOrdering Ordering, SynchronizationScope SynchScope, + AtomicOrdering Ordering, SyncScope::ID SSID, Instruction *InsertBefore = nullptr); AtomicRMWInst(BinOp Operation, Value *Ptr, Value *Val, - AtomicOrdering Ordering, SynchronizationScope SynchScope, + AtomicOrdering Ordering, SyncScope::ID SSID, BasicBlock *InsertAtEnd); // allocate space for exactly two operands @@ -748,7 +753,12 @@ public: /// Transparently provide more efficient getOperand methods. DECLARE_TRANSPARENT_OPERAND_ACCESSORS(Value); - /// Set the ordering constraint on this RMW. + /// Returns the ordering constraint of this rmw instruction. + AtomicOrdering getOrdering() const { + return AtomicOrdering((getSubclassDataFromInstruction() >> 2) & 7); + } + + /// Sets the ordering constraint of this rmw instruction. void setOrdering(AtomicOrdering Ordering) { assert(Ordering != AtomicOrdering::NotAtomic && "atomicrmw instructions can only be atomic."); @@ -756,23 +766,14 @@ public: ((unsigned)Ordering << 2)); } - /// Specify whether this RMW orders other operations with respect to all - /// concurrently executing threads, or only with respect to signal handlers - /// executing in the same thread. - void setSynchScope(SynchronizationScope SynchScope) { - setInstructionSubclassData((getSubclassDataFromInstruction() & ~2) | - (SynchScope << 1)); + /// Returns the synchronization scope ID of this rmw instruction. + SyncScope::ID getSyncScopeID() const { + return SSID; } - /// Returns the ordering constraint on this RMW. - AtomicOrdering getOrdering() const { - return AtomicOrdering((getSubclassDataFromInstruction() >> 2) & 7); - } - - /// Returns whether this RMW is atomic between threads or only within a - /// single thread. - SynchronizationScope getSynchScope() const { - return SynchronizationScope((getSubclassDataFromInstruction() & 2) >> 1); + /// Sets the synchronization scope ID of this rmw instruction. + void setSyncScopeID(SyncScope::ID SSID) { + this->SSID = SSID; } Value *getPointerOperand() { return getOperand(0); } @@ -797,13 +798,18 @@ public: private: void Init(BinOp Operation, Value *Ptr, Value *Val, - AtomicOrdering Ordering, SynchronizationScope SynchScope); + AtomicOrdering Ordering, SyncScope::ID SSID); // Shadow Instruction::setInstructionSubclassData with a private forwarding // method so that subclasses cannot accidentally use it. void setInstructionSubclassData(unsigned short D) { Instruction::setInstructionSubclassData(D); } + + /// The synchronization scope ID of this rmw instruction. Not quite enough + /// room in SubClassData for everything, so synchronization scope ID gets its + /// own field. + SyncScope::ID SSID; }; template <> diff --git a/include/llvm/IR/LLVMContext.h b/include/llvm/IR/LLVMContext.h index b27abad618c..4cb77701f76 100644 --- a/include/llvm/IR/LLVMContext.h +++ b/include/llvm/IR/LLVMContext.h @@ -42,6 +42,24 @@ class Output; } // end namespace yaml +namespace SyncScope { + +typedef uint8_t ID; + +/// Known synchronization scope IDs, which always have the same value. All +/// synchronization scope IDs that LLVM has special knowledge of are listed +/// here. Additionally, this scheme allows LLVM to efficiently check for +/// specific synchronization scope ID without comparing strings. +enum { + /// Synchronized with respect to signal handlers executing in the same thread. + SingleThread = 0, + + /// Synchronized with respect to all concurrently executing threads. + System = 1 +}; + +} // end namespace SyncScope + /// This is an important class for using LLVM in a threaded context. It /// (opaquely) owns and manages the core "global" data of LLVM's core /// infrastructure, including the type and constant uniquing tables. @@ -111,6 +129,16 @@ public: /// tag registered with an LLVMContext has an unique ID. uint32_t getOperandBundleTagID(StringRef Tag) const; + /// getOrInsertSyncScopeID - Maps synchronization scope name to + /// synchronization scope ID. Every synchronization scope registered with + /// LLVMContext has unique ID except pre-defined ones. + SyncScope::ID getOrInsertSyncScopeID(StringRef SSN); + + /// getSyncScopeNames - Populates client supplied SmallVector with + /// synchronization scope names registered with LLVMContext. Synchronization + /// scope names are ordered by increasing synchronization scope IDs. + void getSyncScopeNames(SmallVectorImpl<StringRef> &SSNs) const; + /// Define the GC for a function void setGC(const Function &Fn, std::string GCName); diff --git a/lib/AsmParser/LLLexer.cpp b/lib/AsmParser/LLLexer.cpp index a49276099f1..428bb21fbf5 100644 --- a/lib/AsmParser/LLLexer.cpp +++ b/lib/AsmParser/LLLexer.cpp @@ -542,7 +542,7 @@ lltok::Kind LLLexer::LexIdentifier() { KEYWORD(release); KEYWORD(acq_rel); KEYWORD(seq_cst); - KEYWORD(singlethread); + KEYWORD(syncscope); KEYWORD(nnan); KEYWORD(ninf); diff --git a/lib/AsmParser/LLParser.cpp b/lib/AsmParser/LLParser.cpp index 7817449dac7..717eb0e00f4 100644 --- a/lib/AsmParser/LLParser.cpp +++ b/lib/AsmParser/LLParser.cpp @@ -1919,20 +1919,42 @@ bool LLParser::parseAllocSizeArguments(unsigned &BaseSizeArg, } /// ParseScopeAndOrdering -/// if isAtomic: ::= 'singlethread'? AtomicOrdering +/// if isAtomic: ::= SyncScope? AtomicOrdering /// else: ::= /// /// This sets Scope and Ordering to the parsed values. -bool LLParser::ParseScopeAndOrdering(bool isAtomic, SynchronizationScope &Scope, +bool LLParser::ParseScopeAndOrdering(bool isAtomic, SyncScope::ID &SSID, AtomicOrdering &Ordering) { if (!isAtomic) return false; - Scope = CrossThread; - if (EatIfPresent(lltok::kw_singlethread)) - Scope = SingleThread; + return ParseScope(SSID) || ParseOrdering(Ordering); +} + +/// ParseScope +/// ::= syncscope("singlethread" | "<target scope>")? +/// +/// This sets synchronization scope ID to the ID of the parsed value. +bool LLParser::ParseScope(SyncScope::ID &SSID) { + SSID = SyncScope::System; + if (EatIfPresent(lltok::kw_syncscope)) { + auto StartParenAt = Lex.getLoc(); + if (!EatIfPresent(lltok::lparen)) + return Error(StartParenAt, "Expected '(' in syncscope"); + + std::string SSN; + auto SSNAt = Lex.getLoc(); + if (ParseStringConstant(SSN)) + return Error(SSNAt, "Expected synchronization scope name"); - return ParseOrdering(Ordering); + auto EndParenAt = Lex.getLoc(); + if (!EatIfPresent(lltok::rparen)) + return Error(EndParenAt, "Expected ')' in syncscope"); + + SSID = Context.getOrInsertSyncScopeID(SSN); + } + + return false; } /// ParseOrdering @@ -6100,7 +6122,7 @@ int LLParser::ParseLoad(Instruction *&Inst, PerFunctionState &PFS) { bool AteExtraComma = false; bool isAtomic = false; AtomicOrdering Ordering = AtomicOrdering::NotAtomic; - SynchronizationScope Scope = CrossThread; + SyncScope::ID SSID = SyncScope::System; if (Lex.getKind() == lltok::kw_atomic) { isAtomic = true; @@ -6118,7 +6140,7 @@ int LLParser::ParseLoad(Instruction *&Inst, PerFunctionState &PFS) { if (ParseType(Ty) || ParseToken(lltok::comma, "expected comma after load's type") || ParseTypeAndValue(Val, Loc, PFS) || - ParseScopeAndOrdering(isAtomic, Scope, Ordering) || + ParseScopeAndOrdering(isAtomic, SSID, Ordering) || ParseOptionalCommaAlign(Alignment, AteExtraComma)) return true; @@ -6134,7 +6156,7 @@ int LLParser::ParseLoad(Instruction *&Inst, PerFunctionState &PFS) { return Error(ExplicitTypeLoc, "explicit pointee type doesn't match operand's pointee type"); - Inst = new LoadInst(Ty, Val, "", isVolatile, Alignment, Ordering, Scope); + Inst = new LoadInst(Ty, Val, "", isVolatile, Alignment, Ordering, SSID); return AteExtraComma ? InstExtraComma : InstNormal; } @@ -6149,7 +6171,7 @@ int LLParser::ParseStore(Instruction *&Inst, PerFunctionState &PFS) { bool AteExtraComma = false; bool isAtomic = false; AtomicOrdering Ordering = AtomicOrdering::NotAtomic; - SynchronizationScope Scope = CrossThread; + SyncScope::ID SSID = SyncScope::System; if (Lex.getKind() == lltok::kw_atomic) { isAtomic = true; @@ -6165,7 +6187,7 @@ int LLParser::ParseStore(Instruction *&Inst, PerFunctionState &PFS) { if (ParseTypeAndValue(Val, Loc, PFS) || ParseToken(lltok::comma, "expected ',' after store operand") || ParseTypeAndValue(Ptr, PtrLoc, PFS) || - ParseScopeAndOrdering(isAtomic, Scope, Ordering) || + ParseScopeAndOrdering(isAtomic, SSID, Ordering) || ParseOptionalCommaAlign(Alignment, AteExtraComma)) return true; @@ -6181,7 +6203,7 @@ int LLParser::ParseStore(Instruction *&Inst, PerFunctionState &PFS) { Ordering == AtomicOrdering::AcquireRelease) return Error(Loc, "atomic store cannot use Acquire ordering"); - Inst = new StoreInst(Val, Ptr, isVolatile, Alignment, Ordering, Scope); + Inst = new StoreInst(Val, Ptr, isVolatile, Alignment, Ordering, SSID); return AteExtraComma ? InstExtraComma : InstNormal; } @@ -6193,7 +6215,7 @@ int LLParser::ParseCmpXchg(Instruction *&Inst, PerFunctionState &PFS) { bool AteExtraComma = false; AtomicOrdering SuccessOrdering = AtomicOrdering::NotAtomic; AtomicOrdering FailureOrdering = AtomicOrdering::NotAtomic; - SynchronizationScope Scope = CrossThread; + SyncScope::ID SSID = SyncScope::System; bool isVolatile = false; bool isWeak = false; @@ -6208,7 +6230,7 @@ int LLParser::ParseCmpXchg(Instruction *&Inst, PerFunctionState &PFS) { ParseTypeAndValue(Cmp, CmpLoc, PFS) || ParseToken(lltok::comma, "expected ',' after cmpxchg cmp operand") || ParseTypeAndValue(New, NewLoc, PFS) || - ParseScopeAndOrdering(true /*Always atomic*/, Scope, SuccessOrdering) || + ParseScopeAndOrdering(true /*Always atomic*/, SSID, SuccessOrdering) || ParseOrdering(FailureOrdering)) return true; @@ -6231,7 +6253,7 @@ int LLParser::ParseCmpXchg(Instruction *&Inst, PerFunctionState &PFS) { if (!New->getType()->isFirstClassType()) return Error(NewLoc, "cmpxchg operand must be a first class value"); AtomicCmpXchgInst *CXI = new AtomicCmpXchgInst( - Ptr, Cmp, New, SuccessOrdering, FailureOrdering, Scope); + Ptr, Cmp, New, SuccessOrdering, FailureOrdering, SSID); CXI->setVolatile(isVolatile); CXI->setWeak(isWeak); Inst = CXI; @@ -6245,7 +6267,7 @@ int LLParser::ParseAtomicRMW(Instruction *&Inst, PerFunctionState &PFS) { Value *Ptr, *Val; LocTy PtrLoc, ValLoc; bool AteExtraComma = false; AtomicOrdering Ordering = AtomicOrdering::NotAtomic; - SynchronizationScope Scope = CrossThread; + SyncScope::ID SSID = SyncScope::System; bool isVolatile = false; AtomicRMWInst::BinOp Operation; @@ -6271,7 +6293,7 @@ int LLParser::ParseAtomicRMW(Instruction *&Inst, PerFunctionState &PFS) { if (ParseTypeAndValue(Ptr, PtrLoc, PFS) || ParseToken(lltok::comma, "expected ',' after atomicrmw address") || ParseTypeAndValue(Val, ValLoc, PFS) || - ParseScopeAndOrdering(true /*Always atomic*/, Scope, Ordering)) + ParseScopeAndOrdering(true /*Always atomic*/, SSID, Ordering)) return true; if (Ordering == AtomicOrdering::Unordered) @@ -6288,7 +6310,7 @@ int LLParser::ParseAtomicRMW(Instruction *&Inst, PerFunctionState &PFS) { " integer"); AtomicRMWInst *RMWI = - new AtomicRMWInst(Operation, Ptr, Val, Ordering, Scope); + new AtomicRMWInst(Operation, Ptr, Val, Ordering, SSID); RMWI->setVolatile(isVolatile); Inst = RMWI; return AteExtraComma ? InstExtraComma : InstNormal; @@ -6298,8 +6320,8 @@ int LLParser::ParseAtomicRMW(Instruction *&Inst, PerFunctionState &PFS) { /// ::= 'fence' 'singlethread'? AtomicOrdering int LLParser::ParseFence(Instruction *&Inst, PerFunctionState &PFS) { AtomicOrdering Ordering = AtomicOrdering::NotAtomic; - SynchronizationScope Scope = CrossThread; - if (ParseScopeAndOrdering(true /*Always atomic*/, Scope, Ordering)) + SyncScope::ID SSID = SyncScope::System; + if (ParseScopeAndOrdering(true /*Always atomic*/, SSID, Ordering)) return true; if (Ordering == AtomicOrdering::Unordered) @@ -6307,7 +6329,7 @@ int LLParser::ParseFence(Instruction *&Inst, PerFunctionState &PFS) { if (Ordering == AtomicOrdering::Monotonic) return TokError("fence cannot be monotonic"); - Inst = new FenceInst(Context, Ordering, Scope); + Inst = new FenceInst(Context, Ordering, SSID); return InstNormal; } diff --git a/lib/AsmParser/LLParser.h b/lib/AsmParser/LLParser.h index 4616c2e8694..d5b059355c4 100644 --- a/lib/AsmParser/LLParser.h +++ b/lib/AsmParser/LLParser.h @@ -241,8 +241,9 @@ namespace llvm { bool ParseOptionalCallingConv(unsigned &CC); bool ParseOptionalAlignment(unsigned &Alignment); bool ParseOptionalDerefAttrBytes(lltok::Kind AttrKind, uint64_t &Bytes); - bool ParseScopeAndOrdering(bool isAtomic, SynchronizationScope &Scope, + bool ParseScopeAndOrdering(bool isAtomic, SyncScope::ID &SSID, AtomicOrdering &Ordering); + bool ParseScope(SyncScope::ID &SSID); bool ParseOrdering(AtomicOrdering &Ordering); bool ParseOptionalStackAlignment(unsigned &Alignment); bool ParseOptionalCommaAlign(unsigned &Alignment, bool &AteExtraComma); diff --git a/lib/AsmParser/LLToken.h b/lib/AsmParser/LLToken.h index 6c8ed7da495..9c7a06de81b 100644 --- a/lib/AsmParser/LLToken.h +++ b/lib/AsmParser/LLToken.h @@ -93,7 +93,7 @@ enum Kind { kw_release, kw_acq_rel, kw_seq_cst, - kw_singlethread, + kw_syncscope, kw_nnan, kw_ninf, kw_nsz, diff --git a/lib/Bitcode/Reader/BitcodeReader.cpp b/lib/Bitcode/Reader/BitcodeReader.cpp index 1ebef317313..2b4970a80cd 100644 --- a/lib/Bitcode/Reader/BitcodeReader.cpp +++ b/lib/Bitcode/Reader/BitcodeReader.cpp @@ -513,6 +513,7 @@ class BitcodeReader : public BitcodeReaderBase, public GVMaterializer { TBAAVerifier TBAAVerifyHelper; std::vector<std::string> BundleTags; + SmallVector<SyncScope::ID, 8> SSIDs; public: BitcodeReader(BitstreamCursor Stream, StringRef Strtab, @@ -648,6 +649,7 @@ private: Error parseTypeTable(); Error parseTypeTableBody(); Error parseOperandBundleTags(); + Error parseSyncScopeNames(); Expected<Value *> recordValue(SmallVectorImpl<uint64_t> &Record, unsigned NameIndex, Triple &TT); @@ -668,6 +670,8 @@ private: Error findFunctionInStream( Function *F, DenseMap<Function *, uint64_t>::iterator DeferredFunctionInfoIterator); + + SyncScope::ID getDecodedSyncScopeID(unsigned Val); }; /// Class to manage reading and parsing function summary index bitcode @@ -998,14 +1002,6 @@ static AtomicOrdering getDecodedOrdering(unsigned Val) { } } -static SynchronizationScope getDecodedSynchScope(unsigned Val) { - switch (Val) { - case bitc::SYNCHSCOPE_SINGLETHREAD: return SingleThread; - default: // Map unknown scopes to cross-thread. - case bitc::SYNCHSCOPE_CROSSTHREAD: return CrossThread; - } -} - static Comdat::SelectionKind getDecodedComdatSelectionKind(unsigned Val) { switch (Val) { default: // Map unknown selection kinds to any. @@ -1745,6 +1741,44 @@ Error BitcodeReader::parseOperandBundleTags() { } } +Error BitcodeReader::parseSyncScopeNames() { + if (Stream.EnterSubBlock(bitc::SYNC_SCOPE_NAMES_BLOCK_ID)) + return error("Invalid record"); + + if (!SSIDs.empty()) + return error("Invalid multiple synchronization scope names blocks"); + + SmallVector<uint64_t, 64> Record; + while (true) { + BitstreamEntry Entry = Stream.advanceSkippingSubblocks(); + switch (Entry.Kind) { + case BitstreamEntry::SubBlock: // Handled for us already. + case BitstreamEntry::Error: + return error("Malformed block"); + case BitstreamEntry::EndBlock: + if (SSIDs.empty()) + return error("Invalid empty synchronization scope names block"); + return Error::success(); + case BitstreamEntry::Record: + // The interesting case. + break; + } + + // Synchronization scope names are implicitly mapped to synchronization + // scope IDs by their order. + + if (Stream.readRecord(Entry.ID, Record) != bitc::SYNC_SCOPE_NAME) + return error("Invalid record"); + + SmallString<16> SSN; + if (convertToString(Record, 0, SSN)) + return error("Invalid record"); + + SSIDs.push_back(Context.getOrInsertSyncScopeID(SSN)); + Record.clear(); + } +} + /// Associate a value with its name from the given index in the provided record. Expected<Value *> BitcodeReader::recordValue(SmallVectorImpl<uint64_t> &Record, unsigned NameIndex, Triple &TT) { @@ -3132,6 +3166,10 @@ Error BitcodeReader::parseModule(uint64_t ResumeBit, if (Error Err = parseOperandBundleTags()) return Err; break; + case bitc::SYNC_SCOPE_NAMES_BLOCK_ID: + if (Error Err = parseSyncScopeNames()) + return Err; + break; } continue; @@ -4204,7 +4242,7 @@ Error BitcodeReader::parseFunctionBody(Function *F) { break; } case bitc::FUNC_CODE_INST_LOADATOMIC: { - // LOADATOMIC: [opty, op, align, vol, ordering, synchscope] + // LOADATOMIC: [opty, op, align, vol, ordering, ssid] unsigned OpNum = 0; Value *Op; if (getValueTypePair(Record, OpNum, NextValueNo, Op) || @@ -4226,12 +4264,12 @@ Error BitcodeReader::parseFunctionBody(Function *F) { return error("Invalid record"); if (Ordering != AtomicOrdering::NotAtomic && Record[OpNum] == 0) return error("Invalid record"); - SynchronizationScope SynchScope = getDecodedSynchScope(Record[OpNum + 3]); + SyncScope::ID SSID = getDecodedSyncScopeID(Record[OpNum + 3]); unsigned Align; if (Error Err = parseAlignmentValue(Record[OpNum], Align)) return Err; - I = new LoadInst(Op, "", Record[OpNum+1], Align, Ordering, SynchScope); + I = new LoadInst(Op, "", Record[OpNum+1], Align, Ordering, SSID); InstructionList.push_back(I); break; @@ -4260,7 +4298,7 @@ Error BitcodeReader::parseFunctionBody(Function *F) { } case bitc::FUNC_CODE_INST_STOREATOMIC: case bitc::FUNC_CODE_INST_STOREATOMIC_OLD: { - // STOREATOMIC: [ptrty, ptr, val, align, vol, ordering, synchscope] + // STOREATOMIC: [ptrty, ptr, val, align, vol, ordering, ssid] unsigned OpNum = 0; Value *Val, *Ptr; if (getValueTypePair(Record, OpNum, NextValueNo, Ptr) || @@ -4280,20 +4318,20 @@ Error BitcodeReader::parseFunctionBody(Function *F) { Ordering == AtomicOrdering::Acquire || Ordering == AtomicOrdering::AcquireRelease) return error("Invalid record"); - SynchronizationScope SynchScope = getDecodedSynchScope(Record[OpNum + 3]); + SyncScope::ID SSID = getDecodedSyncScopeID(Record[OpNum + 3]); if (Ordering != AtomicOrdering::NotAtomic && Record[OpNum] == 0) return error("Invalid record"); unsigned Align; if (Error Err = parseAlignmentValue(Record[OpNum], Align)) return Err; - I = new StoreInst(Val, Ptr, Record[OpNum+1], Align, Ordering, SynchScope); + I = new StoreInst(Val, Ptr, Record[OpNum+1], Align, Ordering, SSID); InstructionList.push_back(I); break; } case bitc::FUNC_CODE_INST_CMPXCHG_OLD: case bitc::FUNC_CODE_INST_CMPXCHG: { - // CMPXCHG:[ptrty, ptr, cmp, new, vol, successordering, synchscope, + // CMPXCHG:[ptrty, ptr, cmp, new, vol, successordering, ssid, // failureordering?, isweak?] unsigned OpNum = 0; Value *Ptr, *Cmp, *New; @@ -4310,7 +4348,7 @@ Error BitcodeReader::parseFunctionBody(Function *F) { if (SuccessOrdering == AtomicOrdering::NotAtomic || SuccessOrdering == AtomicOrdering::Unordered) return error("Invalid record"); - SynchronizationScope SynchScope = getDecodedSynchScope(Record[OpNum + 2]); + SyncScope::ID SSID = getDecodedSyncScopeID(Record[OpNum + 2]); if (Error Err = typeCheckLoadStoreInst(Cmp->getType(), Ptr->getType())) return Err; @@ -4322,7 +4360,7 @@ Error BitcodeReader::parseFunctionBody(Function *F) { FailureOrdering = getDecodedOrdering(Record[OpNum + 3]); I = new AtomicCmpXchgInst(Ptr, Cmp, New, SuccessOrdering, FailureOrdering, - SynchScope); + SSID); cast<AtomicCmpXchgInst>(I)->setVolatile(Record[OpNum]); if (Record.size() < 8) { @@ -4339,7 +4377,7 @@ Error BitcodeReader::parseFunctionBody(Function *F) { break; } case bitc::FUNC_CODE_INST_ATOMICRMW: { - // ATOMICRMW:[ptrty, ptr, val, op, vol, ordering, synchscope] + // ATOMICRMW:[ptrty, ptr, val, op, vol, ordering, ssid] unsigned OpNum = 0; Value *Ptr, *Val; if (getValueTypePair(Record, OpNum, NextValueNo, Ptr) || @@ -4356,13 +4394,13 @@ Error BitcodeReader::parseFunctionBody(Function *F) { if (Ordering == AtomicOrdering::NotAtomic || Ordering == AtomicOrdering::Unordered) return error("Invalid record"); - SynchronizationScope SynchScope = getDecodedSynchScope(Record[OpNum + 3]); - I = new AtomicRMWInst(Operation, Ptr, Val, Ordering, SynchScope); + SyncScope::ID SSID = getDecodedSyncScopeID(Record[OpNum + 3]); + I = new AtomicRMWInst(Operation, Ptr, Val, Ordering, SSID); cast<AtomicRMWInst>(I)->setVolatile(Record[OpNum+1]); InstructionList.push_back(I); break; } - case bitc::FUNC_CODE_INST_FENCE: { // FENCE:[ordering, synchscope] + case bitc::FUNC_CODE_INST_FENCE: { // FENCE:[ordering, ssid] if (2 != Record.size()) return error("Invalid record"); AtomicOrdering Ordering = getDecodedOrdering(Record[0]); @@ -4370,8 +4408,8 @@ Error BitcodeReader::parseFunctionBody(Function *F) { Ordering == AtomicOrdering::Unordered || Ordering == AtomicOrdering::Monotonic) return error("Invalid record"); - SynchronizationScope SynchScope = getDecodedSynchScope(Record[1]); - I = new FenceInst(Context, Ordering, SynchScope); + SyncScope::ID SSID = getDecodedSyncScopeID(Record[1]); + I = new FenceInst(Context, Ordering, SSID); InstructionList.push_back(I); break; } @@ -4567,6 +4605,14 @@ Error BitcodeReader::findFunctionInStream( return Error::success(); } +SyncScope::ID BitcodeReader::getDecodedSyncScopeID(unsigned Val) { + if (Val == SyncScope::SingleThread || Val == SyncScope::System) + return SyncScope::ID(Val); + if (Val >= SSIDs.size()) + return SyncScope::System; // Map unknown synchronization scopes to system. + return SSIDs[Val]; +} + //===----------------------------------------------------------------------===// // GVMaterializer implementation //===----------------------------------------------------------------------===// diff --git a/lib/Bitcode/Writer/BitcodeWriter.cpp b/lib/Bitcode/Writer/BitcodeWriter.cpp index 862d43381f3..0e518d2bbc8 100644 --- a/lib/Bitcode/Writer/BitcodeWriter.cpp +++ b/lib/Bitcode/Writer/BitcodeWriter.cpp @@ -266,6 +266,7 @@ private: const GlobalObject &GO); void writeModuleMetadataKinds(); void writeOperandBundleTags(); + void writeSyncScopeNames(); void writeConstants(unsigned FirstVal, unsigned LastVal, bool isGlobal); void writeModuleConstants(); bool pushValueAndType(const Value *V, unsigned InstID, @@ -316,6 +317,10 @@ private: return VE.getValueID(VI.getValue()); } std::map<GlobalValue::GUID, unsigned> &valueIds() { return GUIDToValueIdMap; } + + unsigned getEncodedSyncScopeID(SyncScope::ID SSID) { + return unsigned(SSID); + } }; /// Class to manage the bitcode writing for a combined index. @@ -485,14 +490,6 @@ static unsigned getEncodedOrdering(AtomicOrdering Ordering) { llvm_unreachable("Invalid ordering"); } -static unsigned getEncodedSynchScope(SynchronizationScope SynchScope) { - switch (SynchScope) { - case SingleThread: return bitc::SYNCHSCOPE_SINGLETHREAD; - case CrossThread: return bitc::SYNCHSCOPE_CROSSTHREAD; - } - llvm_unreachable("Invalid synch scope"); -} - static void writeStringRecord(BitstreamWriter &Stream, unsigned Code, StringRef Str, unsigned AbbrevToUse) { SmallVector<unsigned, 64> Vals; @@ -2042,6 +2039,24 @@ void ModuleBitcodeWriter::writeOperandBundleTags() { Stream.ExitBlock(); } +void ModuleBitcodeWriter::writeSyncScopeNames() { + SmallVector<StringRef, 8> SSNs; + M.getContext().getSyncScopeNames(SSNs); + if (SSNs.empty()) + return; + + Stream.EnterSubblock(bitc::SYNC_SCOPE_NAMES_BLOCK_ID, 2); + + SmallVector<uint64_t, 64> Record; + for (auto SSN : SSNs) { + Record.append(SSN.begin(), SSN.end()); + Stream.EmitRecord(bitc::SYNC_SCOPE_NAME, Record, 0); + Record.clear(); + } + + Stream.ExitBlock(); +} + static void emitSignedInt64(SmallVectorImpl<uint64_t> &Vals, uint64_t V) { if ((int64_t)V >= 0) Vals.push_back(V << 1); @@ -2658,7 +2673,7 @@ void ModuleBitcodeWriter::writeInstruction(const Instruction &I, Vals.push_back(cast<LoadInst>(I).isVolatile()); if (cast<LoadInst>(I).isAtomic()) { Vals.push_back(getEncodedOrdering(cast<LoadInst>(I).getOrdering())); - Vals.push_back(getEncodedSynchScope(cast<LoadInst>(I).getSynchScope())); + Vals.push_back(getEncodedSyncScopeID(cast<LoadInst>(I).getSyncScopeID())); } break; case Instruction::Store: @@ -2672,7 +2687,8 @@ void ModuleBitcodeWriter::writeInstruction(const Instruction &I, Vals.push_back(cast<StoreInst>(I).isVolatile()); if (cast<StoreInst>(I).isAtomic()) { Vals.push_back(getEncodedOrdering(cast<StoreInst>(I).getOrdering())); - Vals.push_back(getEncodedSynchScope(cast<StoreInst>(I).getSynchScope())); + Vals.push_back( + getEncodedSyncScopeID(cast<StoreInst>(I).getSyncScopeID())); } break; case Instruction::AtomicCmpXchg: @@ -2684,7 +2700,7 @@ void ModuleBitcodeWriter::writeInstruction(const Instruction &I, Vals.push_back( getEncodedOrdering(cast<AtomicCmpXchgInst>(I).getSuccessOrdering())); Vals.push_back( - getEncodedSynchScope(cast<AtomicCmpXchgInst>(I).getSynchScope())); + getEncodedSyncScopeID(cast<AtomicCmpXchgInst>(I).getSyncScopeID())); Vals.push_back( getEncodedOrdering(cast<AtomicCmpXchgInst>(I).getFailureOrdering())); Vals.push_back(cast<AtomicCmpXchgInst>(I).isWeak()); @@ -2698,12 +2714,12 @@ void ModuleBitcodeWriter::writeInstruction(const Instruction &I, Vals.push_back(cast<AtomicRMWInst>(I).isVolatile()); Vals.push_back(getEncodedOrdering(cast<AtomicRMWInst>(I).getOrdering())); Vals.push_back( - getEncodedSynchScope(cast<AtomicRMWInst>(I).getSynchScope())); + getEncodedSyncScopeID(cast<AtomicRMWInst>(I).getSyncScopeID())); break; case Instruction::Fence: Code = bitc::FUNC_CODE_INST_FENCE; Vals.push_back(getEncodedOrdering(cast<FenceInst>(I).getOrdering())); - Vals.push_back(getEncodedSynchScope(cast<FenceInst>(I).getSynchScope())); + Vals.push_back(getEncodedSyncScopeID(cast<FenceInst>(I).getSyncScopeID())); break; case Instruction::Call: { const CallInst &CI = cast<CallInst>(I); @@ -3716,6 +3732,7 @@ void ModuleBitcodeWriter::write() { writeUseListBlock(nullptr); writeOperandBundleTags(); + writeSyncScopeNames(); // Emit function bodies. DenseMap<const Function *, uint64_t> FunctionToBitcodeIndex; diff --git a/lib/CodeGen/AtomicExpandPass.cpp b/lib/CodeGen/AtomicExpandPass.cpp index 344136b1f19..aa9c8e94d08 100644 --- a/lib/CodeGen/AtomicExpandPass.cpp +++ b/lib/CodeGen/AtomicExpandPass.cpp @@ -361,7 +361,7 @@ LoadInst *AtomicExpand::convertAtomicLoadToIntegerType(LoadInst *LI) { auto *NewLI = Builder.CreateLoad(NewAddr); NewLI->setAlignment(LI->getAlignment()); NewLI->setVolatile(LI->isVolatile()); - NewLI->setAtomic(LI->getOrdering(), LI->getSynchScope()); + NewLI->setAtomic(LI->getOrdering(), LI->getSyncScopeID()); DEBUG(dbgs() << "Replaced " << *LI << " with " << *NewLI << "\n"); Value *NewVal = Builder.CreateBitCast(NewLI, LI->getType()); @@ -444,7 +444,7 @@ StoreInst *AtomicExpand::convertAtomicStoreToIntegerType(StoreInst *SI) { StoreInst *NewSI = Builder.CreateStore(NewVal, NewAddr); NewSI->setAlignment(SI->getAlignment()); NewSI->setVolatile(SI->isVolatile()); - NewSI->setAtomic(SI->getOrdering(), SI->getSynchScope()); + NewSI->setAtomic(SI->getOrdering(), SI->getSyncScopeID()); DEBUG(dbgs() << "Replaced " << *SI << " with " << *NewSI << "\n"); SI->eraseFromParent(); return NewSI; @@ -801,7 +801,7 @@ void AtomicExpand::expandPartwordCmpXchg(AtomicCmpXchgInst *CI) { Value *FullWord_Cmp = Builder.CreateOr(Loaded_MaskOut, Cmp_Shifted); AtomicCmpXchgInst *NewCI = Builder.CreateAtomicCmpXchg( PMV.AlignedAddr, FullWord_Cmp, FullWord_NewVal, CI->getSuccessOrdering(), - CI->getFailureOrdering(), CI->getSynchScope()); + CI->getFailureOrdering(), CI->getSyncScopeID()); NewCI->setVolatile(CI->isVolatile()); // When we're building a strong cmpxchg, we need a loop, so you // might think we could use a weak cmpxchg inside. But, using strong @@ -924,7 +924,7 @@ AtomicCmpXchgInst *AtomicExpand::convertCmpXchgToIntegerType(AtomicCmpXchgInst * auto *NewCI = Builder.CreateAtomicCmpXchg(NewAddr, NewCmp, NewNewVal, CI->getSuccessOrdering(), CI->getFailureOrdering(), - CI->getSynchScope()); + CI->getSyncScopeID()); NewCI->setVolatile(CI->isVolatile()); NewCI->setWeak(CI->isWeak()); DEBUG(dbgs() << "Replaced " << *CI << " with " << *NewCI << "\n"); diff --git a/lib/CodeGen/GlobalISel/IRTranslator.cpp b/lib/CodeGen/GlobalISel/IRTranslator.cpp index 521037f9d20..ed1bd995e60 100644 --- a/lib/CodeGen/GlobalISel/IRTranslator.cpp +++ b/lib/CodeGen/GlobalISel/IRTranslator.cpp @@ -345,7 +345,7 @@ bool IRTranslator::translateLoad(const User &U, MachineIRBuilder &MIRBuilder) { *MF->getMachineMemOperand(MachinePointerInfo(LI.getPointerOperand()), Flags, DL->getTypeStoreSize(LI.getType()), getMemOpAlignment(LI), AAMDNodes(), nullptr, - LI.getSynchScope(), LI.getOrdering())); + LI.getSyncScopeID(), LI.getOrdering())); return true; } @@ -363,7 +363,7 @@ bool IRTranslator::translateStore(const User &U, MachineIRBuilder &MIRBuilder) { *MF->getMachineMemOperand( MachinePointerInfo(SI.getPointerOperand()), Flags, DL->getTypeStoreSize(SI.getValueOperand()->getType()), - getMemOpAlignment(SI), AAMDNodes(), nullptr, SI.getSynchScope(), + getMemOpAlignment(SI), AAMDNodes(), nullptr, SI.getSyncScopeID(), SI.getOrdering())); return true; } diff --git a/lib/CodeGen/MIRParser/MILexer.cpp b/lib/CodeGen/MIRParser/MILexer.cpp index 1f1ce6e8d72..58a655a4dee 100644 --- a/lib/CodeGen/MIRParser/MILexer.cpp +++ b/lib/CodeGen/MIRParser/MILexer.cpp @@ -365,6 +365,14 @@ static Cursor maybeLexIRValue(Cursor C, MIToken &Token, return lexName(C, Token, MIToken::NamedIRValue, Rule.size(), ErrorCallback); } +static Cursor maybeLexStringConstant(Cursor C, MIToken &Token, + ErrorCallbackType ErrorCallback) { + if (C.peek() != '"') + return None; + return lexName(C, Token, MIToken::StringConstant, /*PrefixLength=*/0, + ErrorCallback); +} + static Cursor lexVirtualRegister(Cursor C, MIToken &Token) { auto Range = C; C.advance(); // Skip '%' @@ -630,6 +638,8 @@ StringRef llvm::lexMIToken(StringRef Source, MIToken &Token, return R.remaining(); if (Cursor R = maybeLexEscapedIRValue(C, Token, ErrorCallback)) return R.remaining(); + if (Cursor R = maybeLexStringConstant(C, Token, ErrorCallback)) + return R.remaining(); Token.reset(MIToken::Error, C.remaining()); ErrorCallback(C.location(), diff --git a/lib/CodeGen/MIRParser/MILexer.h b/lib/CodeGen/MIRParser/MILexer.h index 3e9513111bf..ed41e07e3c1 100644 --- a/lib/CodeGen/MIRParser/MILexer.h +++ b/lib/CodeGen/MIRParser/MILexer.h @@ -127,7 +127,8 @@ struct MIToken { NamedIRValue, IRValue, QuotedIRValue, // `<constant value>` - SubRegisterIndex + SubRegisterIndex, + StringConstant }; private: diff --git a/lib/CodeGen/MIRParser/MIParser.cpp b/lib/CodeGen/MIRParser/MIParser.cpp index c58d192284d..70dca277733 100644 --- a/lib/CodeGen/MIRParser/MIParser.cpp +++ b/lib/CodeGen/MIRParser/MIParser.cpp @@ -229,6 +229,7 @@ public: bool parseMemoryOperandFlag(MachineMemOperand::Flags &Flags); bool parseMemoryPseudoSourceValue(const PseudoSourceValue *&PSV); bool parseMachinePointerInfo(MachinePointerInfo &Dest); + bool parseOptionalScope(LLVMContext &Context, SyncScope::ID &SSID); bool parseOptionalAtomicOrdering(AtomicOrdering &Order); bool parseMachineMemoryOperand(MachineMemOperand *&Dest); @@ -318,6 +319,10 @@ private: /// /// Return true if the name isn't a name of a bitmask target flag. bool getBitmaskTargetFlag(StringRef Name, unsigned &Flag); + + /// parseStringConstant + /// ::= StringConstant + bool parseStringConstant(std::string &Result); }; } // end anonymous namespace @@ -2135,6 +2140,26 @@ bool MIParser::parseMachinePointerInfo(MachinePointerInfo &Dest) { return false; } +bool MIParser::parseOptionalScope(LLVMContext &Context, + SyncScope::ID &SSID) { + SSID = SyncScope::System; + if (Token.is(MIToken::Identifier) && Token.stringValue() == "syncscope") { + lex(); + if (expectAndConsume(MIToken::lparen)) + return error("expected '(' in syncscope"); + + std::string SSN; + if (parseStringConstant(SSN)) + return true; + + SSID = Context.getOrInsertSyncScopeID(SSN); + if (expectAndConsume(MIToken::rparen)) + return error("expected ')' in syncscope"); + } + + return false; +} + bool MIParser::parseOptionalAtomicOrdering(AtomicOrdering &Order) { Order = AtomicOrdering::NotAtomic; if (Token.isNot(MIToken::Identifier)) @@ -2174,12 +2199,10 @@ bool MIParser::parseMachineMemoryOperand(MachineMemOperand *&Dest) { Flags |= MachineMemOperand::MOStore; lex(); - // Optional "singlethread" scope. - SynchronizationScope Scope = SynchronizationScope::CrossThread; - if (Token.is(MIToken::Identifier) && Token.stringValue() == "singlethread") { - Scope = SynchronizationScope::SingleThread; - lex(); - } + // Optional synchronization scope. + SyncScope::ID SSID; + if (parseOptionalScope(MF.getFunction()->getContext(), SSID)) + return true; // Up to two atomic orderings (cmpxchg provides guarantees on failure). AtomicOrdering Order, FailureOrder; @@ -2244,7 +2267,7 @@ bool MIParser::parseMachineMemoryOperand(MachineMemOperand *&Dest) { if (expectAndConsume(MIToken::rparen)) return true; Dest = MF.getMachineMemOperand(Ptr, Flags, Size, BaseAlignment, AAInfo, Range, - Scope, Order, FailureOrder); + SSID, Order, FailureOrder); return false; } @@ -2457,6 +2480,14 @@ bool MIParser::getBitmaskTargetFlag(StringRef Name, unsigned &Flag) { return false; } +bool MIParser::parseStringConstant(std::string &Result) { + if (Token.isNot(MIToken::StringConstant)) + return error("expected string constant"); + Result = Token.stringValue(); + lex(); + return false; +} + bool llvm::parseMachineBasicBlockDefinitions(PerFunctionMIParsingState &PFS, StringRef Src, SMDiagnostic &Error) { diff --git a/lib/CodeGen/MIRPrinter.cpp b/lib/CodeGen/MIRPrinter.cpp index c524a9835f3..4cc61420ab4 100644 --- a/lib/CodeGen/MIRPrinter.cpp +++ b/lib/CodeGen/MIRPrinter.cpp @@ -18,6 +18,7 @@ #include "llvm/ADT/SmallPtrSet.h" #include "llvm/ADT/SmallVector.h" #include "llvm/ADT/STLExtras.h" +#include "llvm/ADT/StringExtras.h" #include "llvm/ADT/StringRef.h" #include "llvm/ADT/Twine.h" #include "llvm/CodeGen/GlobalISel/RegisterBank.h" @@ -139,6 +140,8 @@ class MIPrinter { ModuleSlotTracker &MST; const DenseMap<const uint32_t *, unsigned> &RegisterMaskIds; const DenseMap<int, FrameIndexOperand> &StackObjectOperandMapping; + /// Synchronization scope names registered with LLVMContext. + SmallVector<StringRef, 8> SSNs; bool canPredictBranchProbabilities(const MachineBasicBlock &MBB) const; bool canPredictSuccessors(const MachineBasicBlock &MBB) const; @@ -162,7 +165,8 @@ public: void print(const MachineOperand &Op, const TargetRegisterInfo *TRI, unsigned I, bool ShouldPrintRegisterTies, LLT TypeToPrint, bool IsDef = false); - void print(const MachineMemOperand &Op); + void print(const LLVMContext &Context, const MachineMemOperand &Op); + void printSyncScope(const LLVMContext &Context, SyncScope::ID SSID); void print(const MCCFIInstruction &CFI, const TargetRegisterInfo *TRI); }; @@ -731,11 +735,12 @@ void MIPrinter::print(const MachineInstr &MI) { if (!MI.memoperands_empty()) { OS << " :: "; + const LLVMContext &Context = MF->getFunction()->getContext(); bool NeedComma = false; for (const auto *Op : MI.memoperands()) { if (NeedComma) OS << ", "; - print(*Op); + print(Context, *Op); NeedComma = true; } } @@ -1031,7 +1036,7 @@ void MIPrinter::print(const MachineOperand &Op, const TargetRegisterInfo *TRI, } } -void MIPrinter::print(const MachineMemOperand &Op) { +void MIPrinter::print(const LLVMContext &Context, const MachineMemOperand &Op) { OS << '('; // TODO: Print operand's target specific flags. if (Op.isVolatile()) @@ -1049,8 +1054,7 @@ void MIPrinter::print(const MachineMemOperand &Op) { OS << "store "; } - if (Op.getSynchScope() == SynchronizationScope::SingleThread) - OS << "singlethread "; + printSyncScope(Context, Op.getSyncScopeID()); if (Op.getOrdering() != AtomicOrdering::NotAtomic) OS << toIRString(Op.getOrdering()) << ' '; @@ -1119,6 +1123,23 @@ void MIPrinter::print(const MachineMemOperand &Op) { OS << ')'; } +void MIPrinter::printSyncScope(const LLVMContext &Context, SyncScope::ID SSID) { + switch (SSID) { + case SyncScope::System: { + break; + } + default: { + if (SSNs.empty()) + Context.getSyncScopeNames(SSNs); + + OS << "syncscope(\""; + PrintEscapedString(SSNs[SSID], OS); + OS << "\") "; + break; + } + } +} + static void printCFIRegister(unsigned DwarfReg, raw_ostream &OS, const TargetRegisterInfo *TRI) { int Reg = TRI->getLLVMRegNum(DwarfReg, true); diff --git a/lib/CodeGen/MachineFunction.cpp b/lib/CodeGen/MachineFunction.cpp index bbdae6e1a49..f88e175a977 100644 --- a/lib/CodeGen/MachineFunction.cpp +++ b/lib/CodeGen/MachineFunction.cpp @@ -305,11 +305,11 @@ MachineFunction::DeleteMachineBasicBlock(MachineBasicBlock *MBB) { MachineMemOperand *MachineFunction::getMachineMemOperand( MachinePointerInfo PtrInfo, MachineMemOperand::Flags f, uint64_t s, unsigned base_alignment, const AAMDNodes &AAInfo, const MDNode *Ranges, - SynchronizationScope SynchScope, AtomicOrdering Ordering, + SyncScope::ID SSID, AtomicOrdering Ordering, AtomicOrdering FailureOrdering) { return new (Allocator) MachineMemOperand(PtrInfo, f, s, base_alignment, AAInfo, Ranges, - SynchScope, Ordering, FailureOrdering); + SSID, Ordering, FailureOrdering); } MachineMemOperand * @@ -320,13 +320,13 @@ MachineFunction::getMachineMemOperand(const MachineMemOperand *MMO, MachineMemOperand(MachinePointerInfo(MMO->getValue(), MMO->getOffset()+Offset), MMO->getFlags(), Size, MMO->getBaseAlignment(), - AAMDNodes(), nullptr, MMO->getSynchScope(), + AAMDNodes(), nullptr, MMO->getSyncScopeID(), MMO->getOrdering(), MMO->getFailureOrdering()); return new (Allocator) MachineMemOperand(MachinePointerInfo(MMO->getPseudoValue(), MMO->getOffset()+Offset), MMO->getFlags(), Size, MMO->getBaseAlignment(), - AAMDNodes(), nullptr, MMO->getSynchScope(), + AAMDNodes(), nullptr, MMO->getSyncScopeID(), MMO->getOrdering(), MMO->getFailureOrdering()); } @@ -359,7 +359,7 @@ MachineFunction::extractLoadMemRefs(MachineInstr::mmo_iterator Begin, (*I)->getFlags() & ~MachineMemOperand::MOStore, (*I)->getSize(), (*I)->getBaseAlignment(), (*I)->getAAInfo(), nullptr, - (*I)->getSynchScope(), (*I)->getOrdering(), + (*I)->getSyncScopeID(), (*I)->getOrdering(), (*I)->getFailureOrdering()); Result[Index] = JustLoad; } @@ -393,7 +393,7 @@ MachineFunction::extractStoreMemRefs(MachineInstr::mmo_iterator Begin, (*I)->getFlags() & ~MachineMemOperand::MOLoad, (*I)->getSize(), (*I)->getBaseAlignment(), (*I)->getAAInfo(), nullptr, - (*I)->getSynchScope(), (*I)->getOrdering(), + (*I)->getSyncScopeID(), (*I)->getOrdering(), (*I)->getFailureOrdering()); Result[Index] = JustStore; } diff --git a/lib/CodeGen/MachineInstr.cpp b/lib/CodeGen/MachineInstr.cpp index 7be3b1a4894..ca39db49bdb 100644 --- a/lib/CodeGen/MachineInstr.cpp +++ b/lib/CodeGen/MachineInstr.cpp @@ -614,7 +614,7 @@ MachineMemOperand::MachineMemOperand(MachinePointerInfo ptrinfo, Flags f, uint64_t s, unsigned int a, const AAMDNodes &AAInfo, const MDNode *Ranges, - SynchronizationScope SynchScope, + SyncScope::ID SSID, AtomicOrdering Ordering, AtomicOrdering FailureOrdering) : PtrInfo(ptrinfo), Size(s), FlagVals(f), BaseAlignLog2(Log2_32(a) + 1), @@ -625,8 +625,8 @@ MachineMemOperand::MachineMemOperand(MachinePointerInfo ptrinfo, Flags f, assert(getBaseAlignment() == a && "Alignment is not a power of 2!"); assert((isLoad() || isStore()) && "Not a load/store!"); - AtomicInfo.SynchScope = static_cast<unsigned>(SynchScope); - assert(getSynchScope() == SynchScope && "Value truncated"); + AtomicInfo.SSID = static_cast<unsigned>(SSID); + assert(getSyncScopeID() == SSID && "Value truncated"); AtomicInfo.Ordering = static_cast<unsigned>(Ordering); assert(getOrdering() == Ordering && "Value truncated"); AtomicInfo.FailureOrdering = static_cast<unsigned>(FailureOrdering); diff --git a/lib/CodeGen/SelectionDAG/SelectionDAG.cpp b/lib/CodeGen/SelectionDAG/SelectionDAG.cpp index f41323bd08b..823e77850c4 100644 --- a/lib/CodeGen/SelectionDAG/SelectionDAG.cpp +++ b/lib/CodeGen/SelectionDAG/SelectionDAG.cpp @@ -5443,7 +5443,7 @@ SDValue SelectionDAG::getAtomicCmpSwap( unsigned Opcode, const SDLoc &dl, EVT MemVT, SDVTList VTs, SDValue Chain, SDValue Ptr, SDValue Cmp, SDValue Swp, MachinePointerInfo PtrInfo, unsigned Alignment, AtomicOrdering SuccessOrdering, - AtomicOrdering FailureOrdering, SynchronizationScope SynchScope) { + AtomicOrdering FailureOrdering, SyncScope::ID SSID) { assert(Opcode == ISD::ATOMIC_CMP_SWAP || Opcode == ISD::ATOMIC_CMP_SWAP_WITH_SUCCESS); assert(Cmp.getValueType() == Swp.getValueType() && "Invalid Atomic Op Types"); @@ -5459,7 +5459,7 @@ SDValue SelectionDAG::getAtomicCmpSwap( MachineMemOperand::MOStore; MachineMemOperand *MMO = MF.getMachineMemOperand(PtrInfo, Flags, MemVT.getStoreSize(), Alignment, - AAMDNodes(), nullptr, SynchScope, SuccessOrdering, + AAMDNodes(), nullptr, SSID, SuccessOrdering, FailureOrdering); return getAtomicCmpSwap(Opcode, dl, MemVT, VTs, Chain, Ptr, Cmp, Swp, MMO); @@ -5481,7 +5481,7 @@ SDValue SelectionDAG::getAtomic(unsigned Opcode, const SDLoc &dl, EVT MemVT, SDValue Chain, SDValue Ptr, SDValue Val, const Value *PtrVal, unsigned Alignment, AtomicOrdering Ordering, - SynchronizationScope SynchScope) { + SyncScope::ID SSID) { if (Alignment == 0) // Ensure that codegen never sees alignment 0 Alignment = getEVTAlignment(MemVT); @@ -5501,7 +5501,7 @@ SDValue SelectionDAG::getAtomic(unsigned Opcode, const SDLoc &dl, EVT MemVT, MachineMemOperand *MMO = MF.getMachineMemOperand(MachinePointerInfo(PtrVal), Flags, MemVT.getStoreSize(), Alignment, AAMDNodes(), - nullptr, SynchScope, Ordering); + nullptr, SSID, Ordering); return getAtomic(Opcode, dl, MemVT, Chain, Ptr, Val, MMO); } diff --git a/lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp b/lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp index dbec2796ed7..4046b90f3df 100644 --- a/lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp +++ b/lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp @@ -3990,7 +3990,7 @@ void SelectionDAGBuilder::visitAtomicCmpXchg(const AtomicCmpXchgInst &I) { SDLoc dl = getCurSDLoc(); AtomicOrdering SuccessOrder = I.getSuccessOrdering(); AtomicOrdering FailureOrder = I.getFailureOrdering(); - SynchronizationScope Scope = I.getSynchScope(); + SyncScope::ID SSID = I.getSyncScopeID(); SDValue InChain = getRoot(); @@ -4000,7 +4000,7 @@ void SelectionDAGBuilder::visitAtomicCmpXchg(const AtomicCmpXchgInst &I) { ISD::ATOMIC_CMP_SWAP_WITH_SUCCESS, dl, MemVT, VTs, InChain, getValue(I.getPointerOperand()), getValue(I.getCompareOperand()), getValue(I.getNewValOperand()), MachinePointerInfo(I.getPointerOperand()), - /*Alignment=*/ 0, SuccessOrder, FailureOrder, Scope); + /*Alignment=*/ 0, SuccessOrder, FailureOrder, SSID); SDValue OutChain = L.getValue(2); @@ -4026,7 +4026,7 @@ void SelectionDAGBuilder::visitAtomicRMW(const AtomicRMWInst &I) { case AtomicRMWInst::UMin: NT = ISD::ATOMIC_LOAD_UMIN; break; } AtomicOrdering Order = I.getOrdering(); - SynchronizationScope Scope = I.getSynchScope(); + SyncScope::ID SSID = I.getSyncScopeID(); SDValue InChain = getRoot(); @@ -4037,7 +4037,7 @@ void SelectionDAGBuilder::visitAtomicRMW(const AtomicRMWInst &I) { getValue(I.getPointerOperand()), getValue(I.getValOperand()), I.getPointerOperand(), - /* Alignment=*/ 0, Order, Scope); + /* Alignment=*/ 0, Order, SSID); SDValue OutChain = L.getValue(1); @@ -4052,7 +4052,7 @@ void SelectionDAGBuilder::visitFence(const FenceInst &I) { Ops[0] = getRoot(); Ops[1] = DAG.getConstant((unsigned)I.getOrdering(), dl, TLI.getFenceOperandTy(DAG.getDataLayout())); - Ops[2] = DAG.getConstant(I.getSynchScope(), dl, + Ops[2] = DAG.getConstant(I.getSyncScopeID(), dl, TLI.getFenceOperandTy(DAG.getDataLayout())); DAG.setRoot(DAG.getNode(ISD::ATOMIC_FENCE, dl, MVT::Other, Ops)); } @@ -4060,7 +4060,7 @@ void SelectionDAGBuilder::visitFence(const FenceInst &I) { void SelectionDAGBuilder::visitAtomicLoad(const LoadInst &I) { SDLoc dl = getCurSDLoc(); AtomicOrdering Order = I.getOrdering(); - SynchronizationScope Scope = I.getSynchScope(); + SyncScope::ID SSID = I.getSyncScopeID(); SDValue InChain = getRoot(); @@ -4078,7 +4078,7 @@ void SelectionDAGBuilder::visitAtomicLoad(const LoadInst &I) { VT.getStoreSize(), I.getAlignment() ? I.getAlignment() : DAG.getEVTAlignment(VT), - AAMDNodes(), nullptr, Scope, Order); + AAMDNodes(), nullptr, SSID, Order); InChain = TLI.prepareVolatileOrAtomicLoad(InChain, dl, DAG); SDValue L = @@ -4095,7 +4095,7 @@ void SelectionDAGBuilder::visitAtomicStore(const StoreInst &I) { SDLoc dl = getCurSDLoc(); AtomicOrdering Order = I.getOrdering(); - SynchronizationScope Scope = I.getSynchScope(); + SyncScope::ID SSID = I.getSyncScopeID(); SDValue InChain = getRoot(); @@ -4112,7 +4112,7 @@ void SelectionDAGBuilder::visitAtomicStore(const StoreInst &I) { getValue(I.getPointerOperand()), getValue(I.getValueOperand()), I.getPointerOperand(), I.getAlignment(), - Order, Scope); + Order, SSID); DAG.setRoot(OutChain); } diff --git a/lib/IR/AsmWriter.cpp b/lib/IR/AsmWriter.cpp index c7f112887a3..80371780fb6 100644 --- a/lib/IR/AsmWriter.cpp +++ b/lib/IR/AsmWriter.cpp @@ -2119,6 +2119,8 @@ class AssemblyWriter { bool ShouldPreserveUseListOrder; UseListOrderStack UseListOrders; SmallVector<StringRef, 8> MDNames; + /// Synchronization scope names registered with LLVMContext. + SmallVector<StringRef, 8> SSNs; public: /// Construct an AssemblyWriter with an external SlotTracker @@ -2134,10 +2136,15 @@ public: void writeOperand(const Value *Op, bool PrintType); void writeParamOperand(const Value *Operand, AttributeSet Attrs); void writeOperandBundles(ImmutableCallSite CS); - void writeAtomic(AtomicOrdering Ordering, SynchronizationScope SynchScope); - void writeAtomicCmpXchg(AtomicOrdering SuccessOrdering, + void writeSyncScope(const LLVMContext &Context, + SyncScope::ID SSID); + void writeAtomic(const LLVMContext &Context, + AtomicOrdering Ordering, + SyncScope::ID SSID); + void writeAtomicCmpXchg(const LLVMContext &Context, + AtomicOrdering SuccessOrdering, AtomicOrdering FailureOrdering, - SynchronizationScope SynchScope); + SyncScope::ID SSID); void writeAllMDNodes(); void writeMDNode(unsigned Slot, const MDNode *Node); @@ -2199,30 +2206,42 @@ void AssemblyWriter::writeOperand(const Value *Operand, bool PrintType) { WriteAsOperandInternal(Out, Operand, &TypePrinter, &Machine, TheModule); } -void AssemblyWriter::writeAtomic(AtomicOrdering Ordering, - SynchronizationScope SynchScope) { - if (Ordering == AtomicOrdering::NotAtomic) - return; +void AssemblyWriter::writeSyncScope(const LLVMContext &Context, + SyncScope::ID SSID) { + switch (SSID) { + case SyncScope::System: { + break; + } + default: { + if (SSNs.empty()) + Context.getSyncScopeNames(SSNs); - switch (SynchScope) { - case SingleThread: Out << " singlethread"; break; - case CrossThread: break; + Out << " syncscope(\""; + PrintEscapedString(SSNs[SSID], Out); + Out << "\")"; + break; + } } +} + +void AssemblyWriter::writeAtomic(const LLVMContext &Context, + AtomicOrdering Ordering, + SyncScope::ID SSID) { + if (Ordering == AtomicOrdering::NotAtomic) + return; + writeSyncScope(Context, SSID); Out << " " << toIRString(Ordering); } -void AssemblyWriter::writeAtomicCmpXchg(AtomicOrdering SuccessOrdering, +void AssemblyWriter::writeAtomicCmpXchg(const LLVMContext &Context, + AtomicOrdering SuccessOrdering, AtomicOrdering FailureOrdering, - SynchronizationScope SynchScope) { + SyncScope::ID SSID) { assert(SuccessOrdering != AtomicOrdering::NotAtomic && FailureOrdering != AtomicOrdering::NotAtomic); - switch (SynchScope) { - case SingleThread: Out << " singlethread"; break; - case CrossThread: break; - } - + writeSyncScope(Context, SSID); Out << " " << toIRString(SuccessOrdering); Out << " " << toIRString(FailureOrdering); } @@ -3215,21 +3234,22 @@ void AssemblyWriter::printInstruction(const Instruction &I) { // Print atomic ordering/alignment for memory operations if (const LoadInst *LI = dyn_cast<LoadInst>(&I)) { if (LI->isAtomic()) - writeAtomic(LI->getOrdering(), LI->getSynchScope()); + writeAtomic(LI->getContext(), LI->getOrdering(), LI->getSyncScopeID()); if (LI->getAlignment()) Out << ", align " << LI->getAlignment(); } else if (const StoreInst *SI = dyn_cast<StoreInst>(&I)) { if (SI->isAtomic()) - writeAtomic(SI->getOrdering(), SI->getSynchScope()); + writeAtomic(SI->getContext(), SI->getOrdering(), SI->getSyncScopeID()); if (SI->getAlignment()) Out << ", align " << SI->getAlignment(); } else if (const AtomicCmpXchgInst *CXI = dyn_cast<AtomicCmpXchgInst>(&I)) { - writeAtomicCmpXchg(CXI->getSuccessOrdering(), CXI->getFailureOrdering(), - CXI->getSynchScope()); + writeAtomicCmpXchg(CXI->getContext(), CXI->getSuccessOrdering(), + CXI->getFailureOrdering(), CXI->getSyncScopeID()); } else if (const AtomicRMWInst *RMWI = dyn_cast<AtomicRMWInst>(&I)) { - writeAtomic(RMWI->getOrdering(), RMWI->getSynchScope()); + writeAtomic(RMWI->getContext(), RMWI->getOrdering(), + RMWI->getSyncScopeID()); } else if (const FenceInst *FI = dyn_cast<FenceInst>(&I)) { - writeAtomic(FI->getOrdering(), FI->getSynchScope()); + writeAtomic(FI->getContext(), FI->getOrdering(), FI->getSyncScopeID()); } // Print Metadata info. diff --git a/lib/IR/Core.cpp b/lib/IR/Core.cpp index 192a565de0d..2165ae5a947 100644 --- a/lib/IR/Core.cpp +++ b/lib/IR/Core.cpp @@ -2756,11 +2756,14 @@ static LLVMAtomicOrdering mapToLLVMOrdering(AtomicOrdering Ordering) { llvm_unreachable("Invalid AtomicOrdering value!"); } +// TODO: Should this and other atomic instructions support building with +// "syncscope"? LLVMValueRef LLVMBuildFence(LLVMBuilderRef B, LLVMAtomicOrdering Ordering, LLVMBool isSingleThread, const char *Name) { return wrap( unwrap(B)->CreateFence(mapFromLLVMOrdering(Ordering), - isSingleThread ? SingleThread : CrossThread, + isSingleThread ? SyncScope::SingleThread + : SyncScope::System, Name)); } @@ -3042,7 +3045,8 @@ LLVMValueRef LLVMBuildAtomicRMW(LLVMBuilderRef B,LLVMAtomicRMWBinOp op, case LLVMAtomicRMWBinOpUMin: intop = AtomicRMWInst::UMin; break; } return wrap(unwrap(B)->CreateAtomicRMW(intop, unwrap(PTR), unwrap(Val), - mapFromLLVMOrdering(ordering), singleThread ? SingleThread : CrossThread)); + mapFromLLVMOrdering(ordering), singleThread ? SyncScope::SingleThread + : SyncScope::System)); } LLVMValueRef LLVMBuildAtomicCmpXchg(LLVMBuilderRef B, LLVMValueRef Ptr, @@ -3054,7 +3058,7 @@ LLVMValueRef LLVMBuildAtomicCmpXchg(LLVMBuilderRef B, LLVMValueRef Ptr, return wrap(unwrap(B)->CreateAtomicCmpXchg(unwrap(Ptr), unwrap(Cmp), unwrap(New), mapFromLLVMOrdering(SuccessOrdering), mapFromLLVMOrdering(FailureOrdering), - singleThread ? SingleThread : CrossThread)); + singleThread ? SyncScope::SingleThread : SyncScope::System)); } @@ -3062,17 +3066,18 @@ LLVMBool LLVMIsAtomicSingleThread(LLVMValueRef AtomicInst) { Value *P = unwrap<Value>(AtomicInst); if (AtomicRMWInst *I = dyn_cast<AtomicRMWInst>(P)) - return I->getSynchScope() == SingleThread; - return cast<AtomicCmpXchgInst>(P)->getSynchScope() == SingleThread; + return I->getSyncScopeID() == SyncScope::SingleThread; + return cast<AtomicCmpXchgInst>(P)->getSyncScopeID() == + SyncScope::SingleThread; } void LLVMSetAtomicSingleThread(LLVMValueRef AtomicInst, LLVMBool NewValue) { Value *P = unwrap<Value>(AtomicInst); - SynchronizationScope Sync = NewValue ? SingleThread : CrossThread; + SyncScope::ID SSID = NewValue ? SyncScope::SingleThread : SyncScope::System; if (AtomicRMWInst *I = dyn_cast<AtomicRMWInst>(P)) - return I->setSynchScope(Sync); - return cast<AtomicCmpXchgInst>(P)->setSynchScope(Sync); + return I->setSyncScopeID(SSID); + return cast<AtomicCmpXchgInst>(P)->setSyncScopeID(SSID); } LLVMAtomicOrdering LLVMGetCmpXchgSuccessOrdering(LLVMValueRef CmpXchgInst) { diff --git a/lib/IR/Instruction.cpp b/lib/IR/Instruction.cpp index 3dd653d2d04..365cb019aec 100644 --- a/lib/IR/Instruction.cpp +++ b/lib/IR/Instruction.cpp @@ -362,13 +362,13 @@ static bool haveSameSpecialState(const Instruction *I1, const Instruction *I2, (LI->getAlignment() == cast<LoadInst>(I2)->getAlignment() || IgnoreAlignment) && LI->getOrdering() == cast<LoadInst>(I2)->getOrdering() && - LI->getSynchScope() == cast<LoadInst>(I2)->getSynchScope(); + LI->getSyncScopeID() == cast<LoadInst>(I2)->getSyncScopeID(); if (const StoreInst *SI = dyn_cast<StoreInst>(I1)) return SI->isVolatile() == cast<StoreInst>(I2)->isVolatile() && (SI->getAlignment() == cast<StoreInst>(I2)->getAlignment() || IgnoreAlignment) && SI->getOrdering() == cast<StoreInst>(I2)->getOrdering() && - SI->getSynchScope() == cast<StoreInst>(I2)->getSynchScope(); + SI->getSyncScopeID() == cast<StoreInst>(I2)->getSyncScopeID(); if (const CmpInst *CI = dyn_cast<CmpInst>(I1)) return CI->getPredicate() == cast<CmpInst>(I2)->getPredicate(); if (const CallInst *CI = dyn_cast<CallInst>(I1)) @@ -386,7 +386,7 @@ static bool haveSameSpecialState(const Instruction *I1, const Instruction *I2, return EVI->getIndices() == cast<ExtractValueInst>(I2)->getIndices(); if (const FenceInst *FI = dyn_cast<FenceInst>(I1)) return FI->getOrdering() == cast<FenceInst>(I2)->getOrdering() && - FI->getSynchScope() == cast<FenceInst>(I2)->getSynchScope(); + FI->getSyncScopeID() == cast<FenceInst>(I2)->getSyncScopeID(); if (const AtomicCmpXchgInst *CXI = dyn_cast<AtomicCmpXchgInst>(I1)) return CXI->isVolatile() == cast<AtomicCmpXchgInst>(I2)->isVolatile() && CXI->isWeak() == cast<AtomicCmpXchgInst>(I2)->isWeak() && @@ -394,12 +394,13 @@ static bool haveSameSpecialState(const Instruction *I1, const Instruction *I2, cast<AtomicCmpXchgInst>(I2)->getSuccessOrdering() && CXI->getFailureOrdering() == cast<AtomicCmpXchgInst>(I2)->getFailureOrdering() && - CXI->getSynchScope() == cast<AtomicCmpXchgInst>(I2)->getSynchScope(); + CXI->getSyncScopeID() == + cast<AtomicCmpXchgInst>(I2)->getSyncScopeID(); if (const AtomicRMWInst *RMWI = dyn_cast<AtomicRMWInst>(I1)) return RMWI->getOperation() == cast<AtomicRMWInst>(I2)->getOperation() && RMWI->isVolatile() == cast<AtomicRMWInst>(I2)->isVolatile() && RMWI->getOrdering() == cast<AtomicRMWInst>(I2)->getOrdering() && - RMWI->getSynchScope() == cast<AtomicRMWInst>(I2)->getSynchScope(); + RMWI->getSyncScopeID() == cast<AtomicRMWInst>(I2)->getSyncScopeID(); return true; } diff --git a/lib/IR/Instructions.cpp b/lib/IR/Instructions.cpp index af58040eb27..2c49564e328 100644 --- a/lib/IR/Instructions.cpp +++ b/lib/IR/Instructions.cpp @@ -1304,34 +1304,34 @@ LoadInst::LoadInst(Value *Ptr, const Twine &Name, bool isVolatile, LoadInst::LoadInst(Type *Ty, Value *Ptr, const Twine &Name, bool isVolatile, unsigned Align, Instruction *InsertBef) : LoadInst(Ty, Ptr, Name, isVolatile, Align, AtomicOrdering::NotAtomic, - CrossThread, InsertBef) {} + SyncScope::System, InsertBef) {} LoadInst::LoadInst(Value *Ptr, const Twine &Name, bool isVolatile, unsigned Align, BasicBlock *InsertAE) : LoadInst(Ptr, Name, isVolatile, Align, AtomicOrdering::NotAtomic, - CrossThread, InsertAE) {} + SyncScope::System, InsertAE) {} LoadInst::LoadInst(Type *Ty, Value *Ptr, const Twine &Name, bool isVolatile, unsigned Align, AtomicOrdering Order, - SynchronizationScope SynchScope, Instruction *InsertBef) + SyncScope::ID SSID, Instruction *InsertBef) : UnaryInstruction(Ty, Load, Ptr, InsertBef) { assert(Ty == cast<PointerType>(Ptr->getType())->getElementType()); setVolatile(isVolatile); setAlignment(Align); - setAtomic(Order, SynchScope); + setAtomic(Order, SSID); AssertOK(); setName(Name); } LoadInst::LoadInst(Value *Ptr, const Twine &Name, bool isVolatile, unsigned Align, AtomicOrdering Order, - SynchronizationScope SynchScope, + SyncScope::ID SSID, BasicBlock *InsertAE) : UnaryInstruction(cast<PointerType>(Ptr->getType())->getElementType(), Load, Ptr, InsertAE) { setVolatile(isVolatile); setAlignment(Align); - setAtomic(Order, SynchScope); + setAtomic(Order, SSID); AssertOK(); setName(Name); } @@ -1419,16 +1419,16 @@ StoreInst::StoreInst(Value *val, Value *addr, bool isVolatile, StoreInst::StoreInst(Value *val, Value *addr, bool isVolatile, unsigned Align, Instruction *InsertBefore) : StoreInst(val, addr, isVolatile, Align, AtomicOrdering::NotAtomic, - CrossThread, InsertBefore) {} + SyncScope::System, InsertBefore) {} StoreInst::StoreInst(Value *val, Value *addr, bool isVolatile, unsigned Align, BasicBlock *InsertAtEnd) : StoreInst(val, addr, isVolatile, Align, AtomicOrdering::NotAtomic, - CrossThread, InsertAtEnd) {} + SyncScope::System, InsertAtEnd) {} StoreInst::StoreInst(Value *val, Value *addr, bool isVolatile, unsigned Align, AtomicOrdering Order, - SynchronizationScope SynchScope, + SyncScope::ID SSID, Instruction *InsertBefore) : Instruction(Type::getVoidTy(val->getContext()), Store, OperandTraits<StoreInst>::op_begin(this), @@ -1438,13 +1438,13 @@ StoreInst::StoreInst(Value *val, Value *addr, bool isVolatile, Op<1>() = addr; setVolatile(isVolatile); setAlignment(Align); - setAtomic(Order, SynchScope); + setAtomic(Order, SSID); AssertOK(); } StoreInst::StoreInst(Value *val, Value *addr, bool isVolatile, unsigned Align, AtomicOrdering Order, - SynchronizationScope SynchScope, + SyncScope::ID SSID, BasicBlock *InsertAtEnd) : Instruction(Type::getVoidTy(val->getContext()), Store, OperandTraits<StoreInst>::op_begin(this), @@ -1454,7 +1454,7 @@ StoreInst::StoreInst(Value *val, Value *addr, bool isVolatile, Op<1>() = addr; setVolatile(isVolatile); setAlignment(Align); - setAtomic(Order, SynchScope); + setAtomic(Order, SSID); AssertOK(); } @@ -1474,13 +1474,13 @@ void StoreInst::setAlignment(unsigned Align) { void AtomicCmpXchgInst::Init(Value *Ptr, Value *Cmp, Value *NewVal, AtomicOrdering SuccessOrdering, AtomicOrdering FailureOrdering, - SynchronizationScope SynchScope) { + SyncScope::ID SSID) { Op<0>() = Ptr; Op<1>() = Cmp; Op<2>() = NewVal; setSuccessOrdering(SuccessOrdering); setFailureOrdering(FailureOrdering); - setSynchScope(SynchScope); + setSyncScopeID(SSID); assert(getOperand(0) && getOperand(1) && getOperand(2) && "All operands must be non-null!"); @@ -1507,25 +1507,25 @@ void AtomicCmpXchgInst::Init(Value *Ptr, Value *Cmp, Value *NewVal, AtomicCmpXchgInst::AtomicCmpXchgInst(Value *Ptr, Value *Cmp, Value *NewVal, AtomicOrdering SuccessOrdering, AtomicOrdering FailureOrdering, - SynchronizationScope SynchScope, + SyncScope::ID SSID, Instruction *InsertBefore) : Instruction( StructType::get(Cmp->getType(), Type::getInt1Ty(Cmp->getContext())), AtomicCmpXchg, OperandTraits<AtomicCmpXchgInst>::op_begin(this), OperandTraits<AtomicCmpXchgInst>::operands(this), InsertBefore) { - Init(Ptr, Cmp, NewVal, SuccessOrdering, FailureOrdering, SynchScope); + Init(Ptr, Cmp, NewVal, SuccessOrdering, FailureOrdering, SSID); } AtomicCmpXchgInst::AtomicCmpXchgInst(Value *Ptr, Value *Cmp, Value *NewVal, AtomicOrdering SuccessOrdering, AtomicOrdering FailureOrdering, - SynchronizationScope SynchScope, + SyncScope::ID SSID, BasicBlock *InsertAtEnd) : Instruction( StructType::get(Cmp->getType(), Type::getInt1Ty(Cmp->getContext())), AtomicCmpXchg, OperandTraits<AtomicCmpXchgInst>::op_begin(this), OperandTraits<AtomicCmpXchgInst>::operands(this), InsertAtEnd) { - Init(Ptr, Cmp, NewVal, SuccessOrdering, FailureOrdering, SynchScope); + Init(Ptr, Cmp, NewVal, SuccessOrdering, FailureOrdering, SSID); } //===----------------------------------------------------------------------===// @@ -1534,12 +1534,12 @@ AtomicCmpXchgInst::AtomicCmpXchgInst(Value *Ptr, Value *Cmp, Value *NewVal, void AtomicRMWInst::Init(BinOp Operation, Value *Ptr, Value *Val, AtomicOrdering Ordering, - SynchronizationScope SynchScope) { + SyncScope::ID SSID) { Op<0>() = Ptr; Op<1>() = Val; setOperation(Operation); setOrdering(Ordering); - setSynchScope(SynchScope); + setSyncScopeID(SSID); assert(getOperand(0) && getOperand(1) && "All operands must be non-null!"); @@ -1554,24 +1554,24 @@ void AtomicRMWInst::Init(BinOp Operation, Value *Ptr, Value *Val, AtomicRMWInst::AtomicRMWInst(BinOp Operation, Value *Ptr, Value *Val, AtomicOrdering Ordering, - SynchronizationScope SynchScope, + SyncScope::ID SSID, Instruction *InsertBefore) : Instruction(Val->getType(), AtomicRMW, OperandTraits<AtomicRMWInst>::op_begin(this), OperandTraits<AtomicRMWInst>::operands(this), InsertBefore) { - Init(Operation, Ptr, Val, Ordering, SynchScope); + Init(Operation, Ptr, Val, Ordering, SSID); } AtomicRMWInst::AtomicRMWInst(BinOp Operation, Value *Ptr, Value *Val, AtomicOrdering Ordering, - SynchronizationScope SynchScope, + SyncScope::ID SSID, BasicBlock *InsertAtEnd) : Instruction(Val->getType(), AtomicRMW, OperandTraits<AtomicRMWInst>::op_begin(this), OperandTraits<AtomicRMWInst>::operands(this), InsertAtEnd) { - Init(Operation, Ptr, Val, Ordering, SynchScope); + Init(Operation, Ptr, Val, Ordering, SSID); } //===----------------------------------------------------------------------===// @@ -1579,19 +1579,19 @@ AtomicRMWInst::AtomicRMWInst(BinOp Operation, Value *Ptr, Value *Val, //===----------------------------------------------------------------------===// FenceInst::FenceInst(LLVMContext &C, AtomicOrdering Ordering, - SynchronizationScope SynchScope, + SyncScope::ID SSID, Instruction *InsertBefore) : Instruction(Type::getVoidTy(C), Fence, nullptr, 0, InsertBefore) { setOrdering(Ordering); - setSynchScope(SynchScope); + setSyncScopeID(SSID); } FenceInst::FenceInst(LLVMContext &C, AtomicOrdering Ordering, - SynchronizationScope SynchScope, + SyncScope::ID SSID, BasicBlock *InsertAtEnd) : Instruction(Type::getVoidTy(C), Fence, nullptr, 0, InsertAtEnd) { setOrdering(Ordering); - setSynchScope(SynchScope); + setSyncScopeID(SSID); } //===----------------------------------------------------------------------===// @@ -3795,12 +3795,12 @@ AllocaInst *AllocaInst::cloneImpl() const { LoadInst *LoadInst::cloneImpl() const { return new LoadInst(getOperand(0), Twine(), isVolatile(), - getAlignment(), getOrdering(), getSynchScope()); + getAlignment(), getOrdering(), getSyncScopeID()); } StoreInst *StoreInst::cloneImpl() const { return new StoreInst(getOperand(0), getOperand(1), isVolatile(), - getAlignment(), getOrdering(), getSynchScope()); + getAlignment(), getOrdering(), getSyncScopeID()); } @@ -3808,7 +3808,7 @@ AtomicCmpXchgInst *AtomicCmpXchgInst::cloneImpl() const { AtomicCmpXchgInst *Result = new AtomicCmpXchgInst(getOperand(0), getOperand(1), getOperand(2), getSuccessOrdering(), getFailureOrdering(), - getSynchScope()); + getSyncScopeID()); Result->setVolatile(isVolatile()); Result->setWeak(isWeak()); return Result; @@ -3816,14 +3816,14 @@ AtomicCmpXchgInst *AtomicCmpXchgInst::cloneImpl() const { AtomicRMWInst *AtomicRMWInst::cloneImpl() const { AtomicRMWInst *Result = - new AtomicRMWInst(getOperation(),getOperand(0), getOperand(1), - getOrdering(), getSynchScope()); + new AtomicRMWInst(getOperation(), getOperand(0), getOperand(1), + getOrdering(), getSyncScopeID()); Result->setVolatile(isVolatile()); return Result; } FenceInst *FenceInst::cloneImpl() const { - return new FenceInst(getContext(), getOrdering(), getSynchScope()); + return new FenceInst(getContext(), getOrdering(), getSyncScopeID()); } TruncInst *TruncInst::cloneImpl() const { diff --git a/lib/IR/LLVMContext.cpp b/lib/IR/LLVMContext.cpp index 2e13f362344..1463cdb60fb 100644 --- a/lib/IR/LLVMContext.cpp +++ b/lib/IR/LLVMContext.cpp @@ -81,6 +81,16 @@ LLVMContext::LLVMContext() : pImpl(new LLVMContextImpl(*this)) { assert(GCTransitionEntry->second == LLVMContext::OB_gc_transition && "gc-transition operand bundle id drifted!"); (void)GCTransitionEntry; + + SyncScope::ID SingleThreadSSID = + pImpl->getOrInsertSyncScopeID("singlethread"); + assert(SingleThreadSSID == SyncScope::SingleThread && + "singlethread synchronization scope ID drifted!"); + + SyncScope::ID SystemSSID = + pImpl->getOrInsertSyncScopeID(""); + assert(SystemSSID == SyncScope::System && + "system synchronization scope ID drifted!"); } LLVMContext::~LLVMContext() { delete pImpl; } @@ -255,6 +265,14 @@ uint32_t LLVMContext::getOperandBundleTagID(StringRef Tag) const { return pImpl->getOperandBundleTagID(Tag); } +SyncScope::ID LLVMContext::getOrInsertSyncScopeID(StringRef SSN) { + return pImpl->getOrInsertSyncScopeID(SSN); +} + +void LLVMContext::getSyncScopeNames(SmallVectorImpl<StringRef> &SSNs) const { + pImpl->getSyncScopeNames(SSNs); +} + void LLVMContext::setGC(const Function &Fn, std::string GCName) { auto It = pImpl->GCNames.find(&Fn); diff --git a/lib/IR/LLVMContextImpl.cpp b/lib/IR/LLVMContextImpl.cpp index c19e1be44fd..57dd08b36fe 100644 --- a/lib/IR/LLVMContextImpl.cpp +++ b/lib/IR/LLVMContextImpl.cpp @@ -205,6 +205,20 @@ uint32_t LLVMContextImpl::getOperandBundleTagID(StringRef Tag) const { return I->second; } +SyncScope::ID LLVMContextImpl::getOrInsertSyncScopeID(StringRef SSN) { + auto NewSSID = SSC.size(); + assert(NewSSID < std::numeric_limits<SyncScope::ID>::max() && + "Hit the maximum number of synchronization scopes allowed!"); + return SSC.insert(std::make_pair(SSN, SyncScope::ID(NewSSID))).first->second; +} + +void LLVMContextImpl::getSyncScopeNames( + SmallVectorImpl<StringRef> &SSNs) const { + SSNs.resize(SSC.size()); + for (const auto &SSE : SSC) + SSNs[SSE.second] = SSE.first(); +} + /// Singleton instance of the OptBisect class. /// /// This singleton is accessed via the LLVMContext::getOptBisect() function. It diff --git a/lib/IR/LLVMContextImpl.h b/lib/IR/LLVMContextImpl.h index 395beb57fe3..e413a4f3443 100644 --- a/lib/IR/LLVMContextImpl.h +++ b/lib/IR/LLVMContextImpl.h @@ -1297,6 +1297,20 @@ public: void getOperandBundleTags(SmallVectorImpl<StringRef> &Tags) const; uint32_t getOperandBundleTagID(StringRef Tag) const; + /// A set of interned synchronization scopes. The StringMap maps + /// synchronization scope names to their respective synchronization scope IDs. + StringMap<SyncScope::ID> SSC; + + /// getOrInsertSyncScopeID - Maps synchronization scope name to + /// synchronization scope ID. Every synchronization scope registered with + /// LLVMContext has unique ID except pre-defined ones. + SyncScope::ID getOrInsertSyncScopeID(StringRef SSN); + + /// getSyncScopeNames - Populates client supplied SmallVector with + /// synchronization scope names registered with LLVMContext. Synchronization + /// scope names are ordered by increasing synchronization scope IDs. + void getSyncScopeNames(SmallVectorImpl<StringRef> &SSNs) const; + /// Maintain the GC name for each function. /// /// This saves allocating an additional word in Function for programs which diff --git a/lib/IR/Verifier.cpp b/lib/IR/Verifier.cpp index 7039bd34988..49384bf975a 100644 --- a/lib/IR/Verifier.cpp +++ b/lib/IR/Verifier.cpp @@ -3108,7 +3108,7 @@ void Verifier::visitLoadInst(LoadInst &LI) { ElTy, &LI); checkAtomicMemAccessSize(ElTy, &LI); } else { - Assert(LI.getSynchScope() == CrossThread, + Assert(LI.getSyncScopeID() == SyncScope::System, "Non-atomic load cannot have SynchronizationScope specified", &LI); } @@ -3137,7 +3137,7 @@ void Verifier::visitStoreInst(StoreInst &SI) { ElTy, &SI); checkAtomicMemAccessSize(ElTy, &SI); } else { - Assert(SI.getSynchScope() == CrossThread, + Assert(SI.getSyncScopeID() == SyncScope::System, "Non-atomic store cannot have SynchronizationScope specified", &SI); } visitInstruction(SI); diff --git a/lib/Target/ARM/ARMISelLowering.cpp b/lib/Target/ARM/ARMISelLowering.cpp index 4ff9f79898f..6ba7593543a 100644 --- a/lib/Target/ARM/ARMISelLowering.cpp +++ b/lib/Target/ARM/ARMISelLowering.cpp @@ -3398,9 +3398,9 @@ ARMTargetLowering::LowerINTRINSIC_WO_CHAIN(SDValue Op, SelectionDAG &DAG, static SDValue LowerATOMIC_FENCE(SDValue Op, SelectionDAG &DAG, const ARMSubtarget *Subtarget) { SDLoc dl(Op); - ConstantSDNode *ScopeN = cast<ConstantSDNode>(Op.getOperand(2)); - auto Scope = static_cast<SynchronizationScope>(ScopeN->getZExtValue()); - if (Scope == SynchronizationScope::SingleThread) + ConstantSDNode *SSIDNode = cast<ConstantSDNode>(Op.getOperand(2)); + auto SSID = static_cast<SyncScope::ID>(SSIDNode->getZExtValue()); + if (SSID == SyncScope::SingleThread) return Op; if (!Subtarget->hasDataBarrier()) { diff --git a/lib/Target/SystemZ/SystemZISelLowering.cpp b/lib/Target/SystemZ/SystemZISelLowering.cpp index 937bb6d229d..2801141cd95 100644 --- a/lib/Target/SystemZ/SystemZISelLowering.cpp +++ b/lib/Target/SystemZ/SystemZISelLowering.cpp @@ -3182,13 +3182,13 @@ SDValue SystemZTargetLowering::lowerATOMIC_FENCE(SDValue Op, SDLoc DL(Op); AtomicOrdering FenceOrdering = static_cast<AtomicOrdering>( cast<ConstantSDNode>(Op.getOperand(1))->getZExtValue()); - SynchronizationScope FenceScope = static_cast<SynchronizationScope>( + SyncScope::ID FenceSSID = static_cast<SyncScope::ID>( cast<ConstantSDNode>(Op.getOperand(2))->getZExtValue()); // The only fence that needs an instruction is a sequentially-consistent // cross-thread fence. if (FenceOrdering == AtomicOrdering::SequentiallyConsistent && - FenceScope == CrossThread) { + FenceSSID == SyncScope::System) { return SDValue(DAG.getMachineNode(SystemZ::Serialize, DL, MVT::Other, Op.getOperand(0)), 0); diff --git a/lib/Target/X86/X86ISelLowering.cpp b/lib/Target/X86/X86ISelLowering.cpp index 4e160eccb3a..3d08037e728 100644 --- a/lib/Target/X86/X86ISelLowering.cpp +++ b/lib/Target/X86/X86ISelLowering.cpp @@ -22850,7 +22850,7 @@ X86TargetLowering::lowerIdempotentRMWIntoFencedLoad(AtomicRMWInst *AI) const { auto Builder = IRBuilder<>(AI); Module *M = Builder.GetInsertBlock()->getParent()->getParent(); - auto SynchScope = AI->getSynchScope(); + auto SSID = AI->getSyncScopeID(); // We must restrict the ordering to avoid generating loads with Release or // ReleaseAcquire orderings. auto Order = AtomicCmpXchgInst::getStrongestFailureOrdering(AI->getOrdering()); @@ -22872,7 +22872,7 @@ X86TargetLowering::lowerIdempotentRMWIntoFencedLoad(AtomicRMWInst *AI) const { // otherwise, we might be able to be more aggressive on relaxed idempotent // rmw. In practice, they do not look useful, so we don't try to be // especially clever. - if (SynchScope == SingleThread) + if (SSID == SyncScope::SingleThread) // FIXME: we could just insert an X86ISD::MEMBARRIER here, except we are at // the IR level, so we must wrap it in an intrinsic. return nullptr; @@ -22891,7 +22891,7 @@ X86TargetLowering::lowerIdempotentRMWIntoFencedLoad(AtomicRMWInst *AI) const { // Finally we can emit the atomic load. LoadInst *Loaded = Builder.CreateAlignedLoad(Ptr, AI->getType()->getPrimitiveSizeInBits()); - Loaded->setAtomic(Order, SynchScope); + Loaded->setAtomic(Order, SSID); AI->replaceAllUsesWith(Loaded); AI->eraseFromParent(); return Loaded; @@ -22902,13 +22902,13 @@ static SDValue LowerATOMIC_FENCE(SDValue Op, const X86Subtarget &Subtarget, SDLoc dl(Op); AtomicOrdering FenceOrdering = static_cast<AtomicOrdering>( cast<ConstantSDNode>(Op.getOperand(1))->getZExtValue()); - SynchronizationScope FenceScope = static_cast<SynchronizationScope>( + SyncScope::ID FenceSSID = static_cast<SyncScope::ID>( cast<ConstantSDNode>(Op.getOperand(2))->getZExtValue()); // The only fence that needs an instruction is a sequentially-consistent // cross-thread fence. if (FenceOrdering == AtomicOrdering::SequentiallyConsistent && - FenceScope == CrossThread) { + FenceSSID == SyncScope::System) { if (Subtarget.hasMFence()) return DAG.getNode(X86ISD::MFENCE, dl, MVT::Other, Op.getOperand(0)); diff --git a/lib/Transforms/IPO/GlobalOpt.cpp b/lib/Transforms/IPO/GlobalOpt.cpp index 2ce868995bd..93eab680ca6 100644 --- a/lib/Transforms/IPO/GlobalOpt.cpp +++ b/lib/Transforms/IPO/GlobalOpt.cpp @@ -837,7 +837,7 @@ OptimizeGlobalAddressOfMalloc(GlobalVariable *GV, CallInst *CI, Type *AllocTy, if (StoreInst *SI = dyn_cast<StoreInst>(GV->user_back())) { // The global is initialized when the store to it occurs. new StoreInst(ConstantInt::getTrue(GV->getContext()), InitBool, false, 0, - SI->getOrdering(), SI->getSynchScope(), SI); + SI->getOrdering(), SI->getSyncScopeID(), SI); SI->eraseFromParent(); continue; } @@ -854,7 +854,7 @@ OptimizeGlobalAddressOfMalloc(GlobalVariable *GV, CallInst *CI, Type *AllocTy, // Replace the cmp X, 0 with a use of the bool value. // Sink the load to where the compare was, if atomic rules allow us to. Value *LV = new LoadInst(InitBool, InitBool->getName()+".val", false, 0, - LI->getOrdering(), LI->getSynchScope(), + LI->getOrdering(), LI->getSyncScopeID(), LI->isUnordered() ? (Instruction*)ICI : LI); InitBoolUsed = true; switch (ICI->getPredicate()) { @@ -1605,7 +1605,7 @@ static bool TryToShrinkGlobalToBoolean(GlobalVariable *GV, Constant *OtherVal) { assert(LI->getOperand(0) == GV && "Not a copy!"); // Insert a new load, to preserve the saved value. StoreVal = new LoadInst(NewGV, LI->getName()+".b", false, 0, - LI->getOrdering(), LI->getSynchScope(), LI); + LI->getOrdering(), LI->getSyncScopeID(), LI); } else { assert((isa<CastInst>(StoredVal) || isa<SelectInst>(StoredVal)) && "This is not a form that we understand!"); @@ -1614,12 +1614,12 @@ static bool TryToShrinkGlobalToBoolean(GlobalVariable *GV, Constant *OtherVal) { } } new StoreInst(StoreVal, NewGV, false, 0, - SI->getOrdering(), SI->getSynchScope(), SI); + SI->getOrdering(), SI->getSyncScopeID(), SI); } else { // Change the load into a load of bool then a select. LoadInst *LI = cast<LoadInst>(UI); LoadInst *NLI = new LoadInst(NewGV, LI->getName()+".b", false, 0, - LI->getOrdering(), LI->getSynchScope(), LI); + LI->getOrdering(), LI->getSyncScopeID(), LI); Value *NSI; if (IsOneZero) NSI = new ZExtInst(NLI, LI->getType(), "", LI); diff --git a/lib/Transforms/InstCombine/InstCombineLoadStoreAlloca.cpp b/lib/Transforms/InstCombine/InstCombineLoadStoreAlloca.cpp index 05936a94629..c59e1ce69ac 100644 --- a/lib/Transforms/InstCombine/InstCombineLoadStoreAlloca.cpp +++ b/lib/Transforms/InstCombine/InstCombineLoadStoreAlloca.cpp @@ -461,7 +461,7 @@ static LoadInst *combineLoadToNewType(InstCombiner &IC, LoadInst &LI, Type *NewT LoadInst *NewLoad = IC.Builder.CreateAlignedLoad( IC.Builder.CreateBitCast(Ptr, NewTy->getPointerTo(AS)), LI.getAlignment(), LI.isVolatile(), LI.getName() + Suffix); - NewLoad->setAtomic(LI.getOrdering(), LI.getSynchScope()); + NewLoad->setAtomic(LI.getOrdering(), LI.getSyncScopeID()); MDBuilder MDB(NewLoad->getContext()); for (const auto &MDPair : MD) { unsigned ID = MDPair.first; @@ -521,7 +521,7 @@ static StoreInst *combineStoreToNewValue(InstCombiner &IC, StoreInst &SI, Value StoreInst *NewStore = IC.Builder.CreateAlignedStore( V, IC.Builder.CreateBitCast(Ptr, V->getType()->getPointerTo(AS)), SI.getAlignment(), SI.isVolatile()); - NewStore->setAtomic(SI.getOrdering(), SI.getSynchScope()); + NewStore->setAtomic(SI.getOrdering(), SI.getSyncScopeID()); for (const auto &MDPair : MD) { unsigned ID = MDPair.first; MDNode *N = MDPair.second; @@ -1025,9 +1025,9 @@ Instruction *InstCombiner::visitLoadInst(LoadInst &LI) { SI->getOperand(2)->getName()+".val"); assert(LI.isUnordered() && "implied by above"); V1->setAlignment(Align); - V1->setAtomic(LI.getOrdering(), LI.getSynchScope()); + V1->setAtomic(LI.getOrdering(), LI.getSyncScopeID()); V2->setAlignment(Align); - V2->setAtomic(LI.getOrdering(), LI.getSynchScope()); + V2->setAtomic(LI.getOrdering(), LI.getSyncScopeID()); return SelectInst::Create(SI->getCondition(), V1, V2); } @@ -1540,7 +1540,7 @@ bool InstCombiner::SimplifyStoreAtEndOfBlock(StoreInst &SI) { SI.isVolatile(), SI.getAlignment(), SI.getOrdering(), - SI.getSynchScope()); + SI.getSyncScopeID()); InsertNewInstBefore(NewSI, *BBI); // The debug locations of the original instructions might differ; merge them. NewSI->setDebugLoc(DILocation::getMergedLocation(SI.getDebugLoc(), diff --git a/lib/Transforms/Instrumentation/ThreadSanitizer.cpp b/lib/Transforms/Instrumentation/ThreadSanitizer.cpp index a991792bf5a..ec6904486e1 100644 --- a/lib/Transforms/Instrumentation/ThreadSanitizer.cpp +++ b/lib/Transforms/Instrumentation/ThreadSanitizer.cpp @@ -379,10 +379,11 @@ void ThreadSanitizer::chooseInstructionsToInstrument( } static bool isAtomic(Instruction *I) { + // TODO: Ask TTI whether synchronization scope is between threads. if (LoadInst *LI = dyn_cast<LoadInst>(I)) - return LI->isAtomic() && LI->getSynchScope() == CrossThread; + return LI->isAtomic() && LI->getSyncScopeID() != SyncScope::SingleThread; if (StoreInst *SI = dyn_cast<StoreInst>(I)) - return SI->isAtomic() && SI->getSynchScope() == CrossThread; + return SI->isAtomic() && SI->getSyncScopeID() != SyncScope::SingleThread; if (isa<AtomicRMWInst>(I)) return true; if (isa<AtomicCmpXchgInst>(I)) @@ -676,7 +677,7 @@ bool ThreadSanitizer::instrumentAtomic(Instruction *I, const DataLayout &DL) { I->eraseFromParent(); } else if (FenceInst *FI = dyn_cast<FenceInst>(I)) { Value *Args[] = {createOrdering(&IRB, FI->getOrdering())}; - Function *F = FI->getSynchScope() == SingleThread ? + Function *F = FI->getSyncScopeID() == SyncScope::SingleThread ? TsanAtomicSignalFence : TsanAtomicThreadFence; CallInst *C = CallInst::Create(F, Args); ReplaceInstWithInst(I, C); diff --git a/lib/Transforms/Scalar/GVN.cpp b/lib/Transforms/Scalar/GVN.cpp index f6783f5c3db..0fe72f3f733 100644 --- a/lib/Transforms/Scalar/GVN.cpp +++ b/lib/Transforms/Scalar/GVN.cpp @@ -1166,7 +1166,7 @@ bool GVN::PerformLoadPRE(LoadInst *LI, AvailValInBlkVect &ValuesPerBlock, auto *NewLoad = new LoadInst(LoadPtr, LI->getName()+".pre", LI->isVolatile(), LI->getAlignment(), - LI->getOrdering(), LI->getSynchScope(), + LI->getOrdering(), LI->getSyncScopeID(), UnavailablePred->getTerminator()); // Transfer the old load's AA tags to the new load. diff --git a/lib/Transforms/Scalar/JumpThreading.cpp b/lib/Transforms/Scalar/JumpThreading.cpp index 05293eb0079..ee3de51b136 100644 --- a/lib/Transforms/Scalar/JumpThreading.cpp +++ b/lib/Transforms/Scalar/JumpThreading.cpp @@ -1212,7 +1212,7 @@ bool JumpThreadingPass::SimplifyPartiallyRedundantLoad(LoadInst *LI) { LoadInst *NewVal = new LoadInst( LoadedPtr->DoPHITranslation(LoadBB, UnavailablePred), LI->getName() + ".pr", false, LI->getAlignment(), LI->getOrdering(), - LI->getSynchScope(), UnavailablePred->getTerminator()); + LI->getSyncScopeID(), UnavailablePred->getTerminator()); NewVal->setDebugLoc(LI->getDebugLoc()); if (AATags) NewVal->setAAMetadata(AATags); diff --git a/lib/Transforms/Scalar/SROA.cpp b/lib/Transforms/Scalar/SROA.cpp index f8497e9c5fe..b9cee5b2ba9 100644 --- a/lib/Transforms/Scalar/SROA.cpp +++ b/lib/Transforms/Scalar/SROA.cpp @@ -2398,7 +2398,7 @@ private: LoadInst *NewLI = IRB.CreateAlignedLoad(&NewAI, NewAI.getAlignment(), LI.isVolatile(), LI.getName()); if (LI.isVolatile()) - NewLI->setAtomic(LI.getOrdering(), LI.getSynchScope()); + NewLI->setAtomic(LI.getOrdering(), LI.getSyncScopeID()); // Any !nonnull metadata or !range metadata on the old load is also valid // on the new load. This is even true in some cases even when the loads @@ -2433,7 +2433,7 @@ private: getSliceAlign(TargetTy), LI.isVolatile(), LI.getName()); if (LI.isVolatile()) - NewLI->setAtomic(LI.getOrdering(), LI.getSynchScope()); + NewLI->setAtomic(LI.getOrdering(), LI.getSyncScopeID()); V = NewLI; IsPtrAdjusted = true; @@ -2576,7 +2576,7 @@ private: } NewSI->copyMetadata(SI, LLVMContext::MD_mem_parallel_loop_access); if (SI.isVolatile()) - NewSI->setAtomic(SI.getOrdering(), SI.getSynchScope()); + NewSI->setAtomic(SI.getOrdering(), SI.getSyncScopeID()); Pass.DeadInsts.insert(&SI); deleteIfTriviallyDead(OldOp); diff --git a/lib/Transforms/Utils/FunctionComparator.cpp b/lib/Transforms/Utils/FunctionComparator.cpp index 0457294361b..4a2be3a5317 100644 --- a/lib/Transforms/Utils/FunctionComparator.cpp +++ b/lib/Transforms/Utils/FunctionComparator.cpp @@ -513,8 +513,8 @@ int FunctionComparator::cmpOperations(const Instruction *L, if (int Res = cmpOrderings(LI->getOrdering(), cast<LoadInst>(R)->getOrdering())) return Res; - if (int Res = - cmpNumbers(LI->getSynchScope(), cast<LoadInst>(R)->getSynchScope())) + if (int Res = cmpNumbers(LI->getSyncScopeID(), + cast<LoadInst>(R)->getSyncScopeID())) return Res; return cmpRangeMetadata(LI->getMetadata(LLVMContext::MD_range), cast<LoadInst>(R)->getMetadata(LLVMContext::MD_range)); @@ -529,7 +529,8 @@ int FunctionComparator::cmpOperations(const Instruction *L, if (int Res = cmpOrderings(SI->getOrdering(), cast<StoreInst>(R)->getOrdering())) return Res; - return cmpNumbers(SI->getSynchScope(), cast<StoreInst>(R)->getSynchScope()); + return cmpNumbers(SI->getSyncScopeID(), + cast<StoreInst>(R)->getSyncScopeID()); } if (const CmpInst *CI = dyn_cast<CmpInst>(L)) return cmpNumbers(CI->getPredicate(), cast<CmpInst>(R)->getPredicate()); @@ -584,7 +585,8 @@ int FunctionComparator::cmpOperations(const Instruction *L, if (int Res = cmpOrderings(FI->getOrdering(), cast<FenceInst>(R)->getOrdering())) return Res; - return cmpNumbers(FI->getSynchScope(), cast<FenceInst>(R)->getSynchScope()); + return cmpNumbers(FI->getSyncScopeID(), + cast<FenceInst>(R)->getSyncScopeID()); } if (const AtomicCmpXchgInst *CXI = dyn_cast<AtomicCmpXchgInst>(L)) { if (int Res = cmpNumbers(CXI->isVolatile(), @@ -601,8 +603,8 @@ int FunctionComparator::cmpOperations(const Instruction *L, cmpOrderings(CXI->getFailureOrdering(), cast<AtomicCmpXchgInst>(R)->getFailureOrdering())) return Res; - return cmpNumbers(CXI->getSynchScope(), - cast<AtomicCmpXchgInst>(R)->getSynchScope()); + return cmpNumbers(CXI->getSyncScopeID(), + cast<AtomicCmpXchgInst>(R)->getSyncScopeID()); } if (const AtomicRMWInst *RMWI = dyn_cast<AtomicRMWInst>(L)) { if (int Res = cmpNumbers(RMWI->getOperation(), @@ -614,8 +616,8 @@ int FunctionComparator::cmpOperations(const Instruction *L, if (int Res = cmpOrderings(RMWI->getOrdering(), cast<AtomicRMWInst>(R)->getOrdering())) return Res; - return cmpNumbers(RMWI->getSynchScope(), - cast<AtomicRMWInst>(R)->getSynchScope()); + return cmpNumbers(RMWI->getSyncScopeID(), + cast<AtomicRMWInst>(R)->getSyncScopeID()); } if (const PHINode *PNL = dyn_cast<PHINode>(L)) { const PHINode *PNR = cast<PHINode>(R); diff --git a/test/Assembler/atomic.ll b/test/Assembler/atomic.ll index 148b95d88e3..a8b527f2f86 100644 --- a/test/Assembler/atomic.ll +++ b/test/Assembler/atomic.ll @@ -5,14 +5,20 @@ define void @f(i32* %x) { ; CHECK: load atomic i32, i32* %x unordered, align 4 load atomic i32, i32* %x unordered, align 4 - ; CHECK: load atomic volatile i32, i32* %x singlethread acquire, align 4 - load atomic volatile i32, i32* %x singlethread acquire, align 4 + ; CHECK: load atomic volatile i32, i32* %x syncscope("singlethread") acquire, align 4 + load atomic volatile i32, i32* %x syncscope("singlethread") acquire, align 4 + ; CHECK: load atomic volatile i32, i32* %x syncscope("agent") acquire, align 4 + load atomic volatile i32, i32* %x syncscope("agent") acquire, align 4 ; CHECK: store atomic i32 3, i32* %x release, align 4 store atomic i32 3, i32* %x release, align 4 - ; CHECK: store atomic volatile i32 3, i32* %x singlethread monotonic, align 4 - store atomic volatile i32 3, i32* %x singlethread monotonic, align 4 - ; CHECK: cmpxchg i32* %x, i32 1, i32 0 singlethread monotonic monotonic - cmpxchg i32* %x, i32 1, i32 0 singlethread monotonic monotonic + ; CHECK: store atomic volatile i32 3, i32* %x syncscope("singlethread") monotonic, align 4 + store atomic volatile i32 3, i32* %x syncscope("singlethread") monotonic, align 4 + ; CHECK: store atomic volatile i32 3, i32* %x syncscope("workgroup") monotonic, align 4 + store atomic volatile i32 3, i32* %x syncscope("workgroup") monotonic, align 4 + ; CHECK: cmpxchg i32* %x, i32 1, i32 0 syncscope("singlethread") monotonic monotonic + cmpxchg i32* %x, i32 1, i32 0 syncscope("singlethread") monotonic monotonic + ; CHECK: cmpxchg i32* %x, i32 1, i32 0 syncscope("workitem") monotonic monotonic + cmpxchg i32* %x, i32 1, i32 0 syncscope("workitem") monotonic monotonic ; CHECK: cmpxchg volatile i32* %x, i32 0, i32 1 acq_rel acquire cmpxchg volatile i32* %x, i32 0, i32 1 acq_rel acquire ; CHECK: cmpxchg i32* %x, i32 42, i32 0 acq_rel monotonic @@ -23,9 +29,13 @@ define void @f(i32* %x) { atomicrmw add i32* %x, i32 10 seq_cst ; CHECK: atomicrmw volatile xchg i32* %x, i32 10 monotonic atomicrmw volatile xchg i32* %x, i32 10 monotonic - ; CHECK: fence singlethread release - fence singlethread release + ; CHECK: atomicrmw volatile xchg i32* %x, i32 10 syncscope("agent") monotonic + atomicrmw volatile xchg i32* %x, i32 10 syncscope("agent") monotonic + ; CHECK: fence syncscope("singlethread") release + fence syncscope("singlethread") release ; CHECK: fence seq_cst fence seq_cst + ; CHECK: fence syncscope("device") seq_cst + fence syncscope("device") seq_cst ret void } diff --git a/test/Bitcode/atomic-no-syncscope.ll b/test/Bitcode/atomic-no-syncscope.ll new file mode 100644 index 00000000000..a57507bc814 --- /dev/null +++ b/test/Bitcode/atomic-no-syncscope.ll @@ -0,0 +1,17 @@ +; RUN: llvm-dis -o - %s.bc | FileCheck %s + +; Backwards compatibility test: make sure we can process bitcode without +; synchronization scope names encoded in it. + +; CHECK: load atomic i32, i32* %x unordered, align 4 +; CHECK: load atomic volatile i32, i32* %x syncscope("singlethread") acquire, align 4 +; CHECK: store atomic i32 3, i32* %x release, align 4 +; CHECK: store atomic volatile i32 3, i32* %x syncscope("singlethread") monotonic, align 4 +; CHECK: cmpxchg i32* %x, i32 1, i32 0 syncscope("singlethread") monotonic monotonic +; CHECK: cmpxchg volatile i32* %x, i32 0, i32 1 acq_rel acquire +; CHECK: cmpxchg i32* %x, i32 42, i32 0 acq_rel monotonic +; CHECK: cmpxchg weak i32* %x, i32 13, i32 0 seq_cst monotonic +; CHECK: atomicrmw add i32* %x, i32 10 seq_cst +; CHECK: atomicrmw volatile xchg i32* %x, i32 10 monotonic +; CHECK: fence syncscope("singlethread") release +; CHECK: fence seq_cst diff --git a/test/Bitcode/atomic-no-syncscope.ll.bc b/test/Bitcode/atomic-no-syncscope.ll.bc Binary files differnew file mode 100644 index 00000000000..01d565eb442 --- /dev/null +++ b/test/Bitcode/atomic-no-syncscope.ll.bc diff --git a/test/Bitcode/atomic.ll b/test/Bitcode/atomic.ll index c09e74c1c2f..bef3f271293 100644 --- a/test/Bitcode/atomic.ll +++ b/test/Bitcode/atomic.ll @@ -11,8 +11,8 @@ define void @test_cmpxchg(i32* %addr, i32 %desired, i32 %new) { cmpxchg weak i32* %addr, i32 %desired, i32 %new acq_rel acquire ; CHECK: cmpxchg weak i32* %addr, i32 %desired, i32 %new acq_rel acquire - cmpxchg weak volatile i32* %addr, i32 %desired, i32 %new singlethread release monotonic - ; CHECK: cmpxchg weak volatile i32* %addr, i32 %desired, i32 %new singlethread release monotonic + cmpxchg weak volatile i32* %addr, i32 %desired, i32 %new syncscope("singlethread") release monotonic + ; CHECK: cmpxchg weak volatile i32* %addr, i32 %desired, i32 %new syncscope("singlethread") release monotonic ret void } diff --git a/test/Bitcode/compatibility-3.6.ll b/test/Bitcode/compatibility-3.6.ll index 8d51ee11a20..cf6c30e7c26 100644 --- a/test/Bitcode/compatibility-3.6.ll +++ b/test/Bitcode/compatibility-3.6.ll @@ -551,8 +551,8 @@ define void @atomics(i32* %word) { ; CHECK: %cmpxchg.5 = cmpxchg weak i32* %word, i32 0, i32 9 seq_cst monotonic %cmpxchg.6 = cmpxchg volatile i32* %word, i32 0, i32 10 seq_cst monotonic ; CHECK: %cmpxchg.6 = cmpxchg volatile i32* %word, i32 0, i32 10 seq_cst monotonic - %cmpxchg.7 = cmpxchg weak volatile i32* %word, i32 0, i32 11 singlethread seq_cst monotonic - ; CHECK: %cmpxchg.7 = cmpxchg weak volatile i32* %word, i32 0, i32 11 singlethread seq_cst monotonic + %cmpxchg.7 = cmpxchg weak volatile i32* %word, i32 0, i32 11 syncscope("singlethread") seq_cst monotonic + ; CHECK: %cmpxchg.7 = cmpxchg weak volatile i32* %word, i32 0, i32 11 syncscope("singlethread") seq_cst monotonic %atomicrmw.xchg = atomicrmw xchg i32* %word, i32 12 monotonic ; CHECK: %atomicrmw.xchg = atomicrmw xchg i32* %word, i32 12 monotonic %atomicrmw.add = atomicrmw add i32* %word, i32 13 monotonic @@ -571,33 +571,33 @@ define void @atomics(i32* %word) { ; CHECK: %atomicrmw.max = atomicrmw max i32* %word, i32 19 monotonic %atomicrmw.min = atomicrmw volatile min i32* %word, i32 20 monotonic ; CHECK: %atomicrmw.min = atomicrmw volatile min i32* %word, i32 20 monotonic - %atomicrmw.umax = atomicrmw umax i32* %word, i32 21 singlethread monotonic - ; CHECK: %atomicrmw.umax = atomicrmw umax i32* %word, i32 21 singlethread monotonic - %atomicrmw.umin = atomicrmw volatile umin i32* %word, i32 22 singlethread monotonic - ; CHECK: %atomicrmw.umin = atomicrmw volatile umin i32* %word, i32 22 singlethread monotonic + %atomicrmw.umax = atomicrmw umax i32* %word, i32 21 syncscope("singlethread") monotonic + ; CHECK: %atomicrmw.umax = atomicrmw umax i32* %word, i32 21 syncscope("singlethread") monotonic + %atomicrmw.umin = atomicrmw volatile umin i32* %word, i32 22 syncscope("singlethread") monotonic + ; CHECK: %atomicrmw.umin = atomicrmw volatile umin i32* %word, i32 22 syncscope("singlethread") monotonic fence acquire ; CHECK: fence acquire fence release ; CHECK: fence release fence acq_rel ; CHECK: fence acq_rel - fence singlethread seq_cst - ; CHECK: fence singlethread seq_cst + fence syncscope("singlethread") seq_cst + ; CHECK: fence syncscope("singlethread") seq_cst ; XXX: The parser spits out the load type here. %ld.1 = load atomic i32* %word monotonic, align 4 ; CHECK: %ld.1 = load atomic i32, i32* %word monotonic, align 4 %ld.2 = load atomic volatile i32* %word acquire, align 8 ; CHECK: %ld.2 = load atomic volatile i32, i32* %word acquire, align 8 - %ld.3 = load atomic volatile i32* %word singlethread seq_cst, align 16 - ; CHECK: %ld.3 = load atomic volatile i32, i32* %word singlethread seq_cst, align 16 + %ld.3 = load atomic volatile i32* %word syncscope("singlethread") seq_cst, align 16 + ; CHECK: %ld.3 = load atomic volatile i32, i32* %word syncscope("singlethread") seq_cst, align 16 store atomic i32 23, i32* %word monotonic, align 4 ; CHECK: store atomic i32 23, i32* %word monotonic, align 4 store atomic volatile i32 24, i32* %word monotonic, align 4 ; CHECK: store atomic volatile i32 24, i32* %word monotonic, align 4 - store atomic volatile i32 25, i32* %word singlethread monotonic, align 4 - ; CHECK: store atomic volatile i32 25, i32* %word singlethread monotonic, align 4 + store atomic volatile i32 25, i32* %word syncscope("singlethread") monotonic, align 4 + ; CHECK: store atomic volatile i32 25, i32* %word syncscope("singlethread") monotonic, align 4 ret void } diff --git a/test/Bitcode/compatibility-3.7.ll b/test/Bitcode/compatibility-3.7.ll index ebdf4c30587..180dad258b6 100644 --- a/test/Bitcode/compatibility-3.7.ll +++ b/test/Bitcode/compatibility-3.7.ll @@ -596,8 +596,8 @@ define void @atomics(i32* %word) { ; CHECK: %cmpxchg.5 = cmpxchg weak i32* %word, i32 0, i32 9 seq_cst monotonic %cmpxchg.6 = cmpxchg volatile i32* %word, i32 0, i32 10 seq_cst monotonic ; CHECK: %cmpxchg.6 = cmpxchg volatile i32* %word, i32 0, i32 10 seq_cst monotonic - %cmpxchg.7 = cmpxchg weak volatile i32* %word, i32 0, i32 11 singlethread seq_cst monotonic - ; CHECK: %cmpxchg.7 = cmpxchg weak volatile i32* %word, i32 0, i32 11 singlethread seq_cst monotonic + %cmpxchg.7 = cmpxchg weak volatile i32* %word, i32 0, i32 11 syncscope("singlethread") seq_cst monotonic + ; CHECK: %cmpxchg.7 = cmpxchg weak volatile i32* %word, i32 0, i32 11 syncscope("singlethread") seq_cst monotonic %atomicrmw.xchg = atomicrmw xchg i32* %word, i32 12 monotonic ; CHECK: %atomicrmw.xchg = atomicrmw xchg i32* %word, i32 12 monotonic %atomicrmw.add = atomicrmw add i32* %word, i32 13 monotonic @@ -616,32 +616,32 @@ define void @atomics(i32* %word) { ; CHECK: %atomicrmw.max = atomicrmw max i32* %word, i32 19 monotonic %atomicrmw.min = atomicrmw volatile min i32* %word, i32 20 monotonic ; CHECK: %atomicrmw.min = atomicrmw volatile min i32* %word, i32 20 monotonic - %atomicrmw.umax = atomicrmw umax i32* %word, i32 21 singlethread monotonic - ; CHECK: %atomicrmw.umax = atomicrmw umax i32* %word, i32 21 singlethread monotonic - %atomicrmw.umin = atomicrmw volatile umin i32* %word, i32 22 singlethread monotonic - ; CHECK: %atomicrmw.umin = atomicrmw volatile umin i32* %word, i32 22 singlethread monotonic + %atomicrmw.umax = atomicrmw umax i32* %word, i32 21 syncscope("singlethread") monotonic + ; CHECK: %atomicrmw.umax = atomicrmw umax i32* %word, i32 21 syncscope("singlethread") monotonic + %atomicrmw.umin = atomicrmw volatile umin i32* %word, i32 22 syncscope("singlethread") monotonic + ; CHECK: %atomicrmw.umin = atomicrmw volatile umin i32* %word, i32 22 syncscope("singlethread") monotonic fence acquire ; CHECK: fence acquire fence release ; CHECK: fence release fence acq_rel ; CHECK: fence acq_rel - fence singlethread seq_cst - ; CHECK: fence singlethread seq_cst + fence syncscope("singlethread") seq_cst + ; CHECK: fence syncscope("singlethread") seq_cst %ld.1 = load atomic i32, i32* %word monotonic, align 4 ; CHECK: %ld.1 = load atomic i32, i32* %word monotonic, align 4 %ld.2 = load atomic volatile i32, i32* %word acquire, align 8 ; CHECK: %ld.2 = load atomic volatile i32, i32* %word acquire, align 8 - %ld.3 = load atomic volatile i32, i32* %word singlethread seq_cst, align 16 - ; CHECK: %ld.3 = load atomic volatile i32, i32* %word singlethread seq_cst, align 16 + %ld.3 = load atomic volatile i32, i32* %word syncscope("singlethread") seq_cst, align 16 + ; CHECK: %ld.3 = load atomic volatile i32, i32* %word syncscope("singlethread") seq_cst, align 16 store atomic i32 23, i32* %word monotonic, align 4 ; CHECK: store atomic i32 23, i32* %word monotonic, align 4 store atomic volatile i32 24, i32* %word monotonic, align 4 ; CHECK: store atomic volatile i32 24, i32* %word monotonic, align 4 - store atomic volatile i32 25, i32* %word singlethread monotonic, align 4 - ; CHECK: store atomic volatile i32 25, i32* %word singlethread monotonic, align 4 + store atomic volatile i32 25, i32* %word syncscope("singlethread") monotonic, align 4 + ; CHECK: store atomic volatile i32 25, i32* %word syncscope("singlethread") monotonic, align 4 ret void } diff --git a/test/Bitcode/compatibility-3.8.ll b/test/Bitcode/compatibility-3.8.ll index 57ea3e06837..370c7f51a2b 100644 --- a/test/Bitcode/compatibility-3.8.ll +++ b/test/Bitcode/compatibility-3.8.ll @@ -627,8 +627,8 @@ define void @atomics(i32* %word) { ; CHECK: %cmpxchg.5 = cmpxchg weak i32* %word, i32 0, i32 9 seq_cst monotonic %cmpxchg.6 = cmpxchg volatile i32* %word, i32 0, i32 10 seq_cst monotonic ; CHECK: %cmpxchg.6 = cmpxchg volatile i32* %word, i32 0, i32 10 seq_cst monotonic - %cmpxchg.7 = cmpxchg weak volatile i32* %word, i32 0, i32 11 singlethread seq_cst monotonic - ; CHECK: %cmpxchg.7 = cmpxchg weak volatile i32* %word, i32 0, i32 11 singlethread seq_cst monotonic + %cmpxchg.7 = cmpxchg weak volatile i32* %word, i32 0, i32 11 syncscope("singlethread") seq_cst monotonic + ; CHECK: %cmpxchg.7 = cmpxchg weak volatile i32* %word, i32 0, i32 11 syncscope("singlethread") seq_cst monotonic %atomicrmw.xchg = atomicrmw xchg i32* %word, i32 12 monotonic ; CHECK: %atomicrmw.xchg = atomicrmw xchg i32* %word, i32 12 monotonic %atomicrmw.add = atomicrmw add i32* %word, i32 13 monotonic @@ -647,32 +647,32 @@ define void @atomics(i32* %word) { ; CHECK: %atomicrmw.max = atomicrmw max i32* %word, i32 19 monotonic %atomicrmw.min = atomicrmw volatile min i32* %word, i32 20 monotonic ; CHECK: %atomicrmw.min = atomicrmw volatile min i32* %word, i32 20 monotonic - %atomicrmw.umax = atomicrmw umax i32* %word, i32 21 singlethread monotonic - ; CHECK: %atomicrmw.umax = atomicrmw umax i32* %word, i32 21 singlethread monotonic - %atomicrmw.umin = atomicrmw volatile umin i32* %word, i32 22 singlethread monotonic - ; CHECK: %atomicrmw.umin = atomicrmw volatile umin i32* %word, i32 22 singlethread monotonic + %atomicrmw.umax = atomicrmw umax i32* %word, i32 21 syncscope("singlethread") monotonic + ; CHECK: %atomicrmw.umax = atomicrmw umax i32* %word, i32 21 syncscope("singlethread") monotonic + %atomicrmw.umin = atomicrmw volatile umin i32* %word, i32 22 syncscope("singlethread") monotonic + ; CHECK: %atomicrmw.umin = atomicrmw volatile umin i32* %word, i32 22 syncscope("singlethread") monotonic fence acquire ; CHECK: fence acquire fence release ; CHECK: fence release fence acq_rel ; CHECK: fence acq_rel - fence singlethread seq_cst - ; CHECK: fence singlethread seq_cst + fence syncscope("singlethread") seq_cst + ; CHECK: fence syncscope("singlethread") seq_cst %ld.1 = load atomic i32, i32* %word monotonic, align 4 ; CHECK: %ld.1 = load atomic i32, i32* %word monotonic, align 4 %ld.2 = load atomic volatile i32, i32* %word acquire, align 8 ; CHECK: %ld.2 = load atomic volatile i32, i32* %word acquire, align 8 - %ld.3 = load atomic volatile i32, i32* %word singlethread seq_cst, align 16 - ; CHECK: %ld.3 = load atomic volatile i32, i32* %word singlethread seq_cst, align 16 + %ld.3 = load atomic volatile i32, i32* %word syncscope("singlethread") seq_cst, align 16 + ; CHECK: %ld.3 = load atomic volatile i32, i32* %word syncscope("singlethread") seq_cst, align 16 store atomic i32 23, i32* %word monotonic, align 4 ; CHECK: store atomic i32 23, i32* %word monotonic, align 4 store atomic volatile i32 24, i32* %word monotonic, align 4 ; CHECK: store atomic volatile i32 24, i32* %word monotonic, align 4 - store atomic volatile i32 25, i32* %word singlethread monotonic, align 4 - ; CHECK: store atomic volatile i32 25, i32* %word singlethread monotonic, align 4 + store atomic volatile i32 25, i32* %word syncscope("singlethread") monotonic, align 4 + ; CHECK: store atomic volatile i32 25, i32* %word syncscope("singlethread") monotonic, align 4 ret void } diff --git a/test/Bitcode/compatibility-3.9.ll b/test/Bitcode/compatibility-3.9.ll index 2a6cfe14cdb..4115cbd8fe6 100644 --- a/test/Bitcode/compatibility-3.9.ll +++ b/test/Bitcode/compatibility-3.9.ll @@ -698,8 +698,8 @@ define void @atomics(i32* %word) { ; CHECK: %cmpxchg.5 = cmpxchg weak i32* %word, i32 0, i32 9 seq_cst monotonic %cmpxchg.6 = cmpxchg volatile i32* %word, i32 0, i32 10 seq_cst monotonic ; CHECK: %cmpxchg.6 = cmpxchg volatile i32* %word, i32 0, i32 10 seq_cst monotonic - %cmpxchg.7 = cmpxchg weak volatile i32* %word, i32 0, i32 11 singlethread seq_cst monotonic - ; CHECK: %cmpxchg.7 = cmpxchg weak volatile i32* %word, i32 0, i32 11 singlethread seq_cst monotonic + %cmpxchg.7 = cmpxchg weak volatile i32* %word, i32 0, i32 11 syncscope("singlethread") seq_cst monotonic + ; CHECK: %cmpxchg.7 = cmpxchg weak volatile i32* %word, i32 0, i32 11 syncscope("singlethread") seq_cst monotonic %atomicrmw.xchg = atomicrmw xchg i32* %word, i32 12 monotonic ; CHECK: %atomicrmw.xchg = atomicrmw xchg i32* %word, i32 12 monotonic %atomicrmw.add = atomicrmw add i32* %word, i32 13 monotonic @@ -718,32 +718,32 @@ define void @atomics(i32* %word) { ; CHECK: %atomicrmw.max = atomicrmw max i32* %word, i32 19 monotonic %atomicrmw.min = atomicrmw volatile min i32* %word, i32 20 monotonic ; CHECK: %atomicrmw.min = atomicrmw volatile min i32* %word, i32 20 monotonic - %atomicrmw.umax = atomicrmw umax i32* %word, i32 21 singlethread monotonic - ; CHECK: %atomicrmw.umax = atomicrmw umax i32* %word, i32 21 singlethread monotonic - %atomicrmw.umin = atomicrmw volatile umin i32* %word, i32 22 singlethread monotonic - ; CHECK: %atomicrmw.umin = atomicrmw volatile umin i32* %word, i32 22 singlethread monotonic + %atomicrmw.umax = atomicrmw umax i32* %word, i32 21 syncscope("singlethread") monotonic + ; CHECK: %atomicrmw.umax = atomicrmw umax i32* %word, i32 21 syncscope("singlethread") monotonic + %atomicrmw.umin = atomicrmw volatile umin i32* %word, i32 22 syncscope("singlethread") monotonic + ; CHECK: %atomicrmw.umin = atomicrmw volatile umin i32* %word, i32 22 syncscope("singlethread") monotonic fence acquire ; CHECK: fence acquire fence release ; CHECK: fence release fence acq_rel ; CHECK: fence acq_rel - fence singlethread seq_cst - ; CHECK: fence singlethread seq_cst + fence syncscope("singlethread") seq_cst + ; CHECK: fence syncscope("singlethread") seq_cst %ld.1 = load atomic i32, i32* %word monotonic, align 4 ; CHECK: %ld.1 = load atomic i32, i32* %word monotonic, align 4 %ld.2 = load atomic volatile i32, i32* %word acquire, align 8 ; CHECK: %ld.2 = load atomic volatile i32, i32* %word acquire, align 8 - %ld.3 = load atomic volatile i32, i32* %word singlethread seq_cst, align 16 - ; CHECK: %ld.3 = load atomic volatile i32, i32* %word singlethread seq_cst, align 16 + %ld.3 = load atomic volatile i32, i32* %word syncscope("singlethread") seq_cst, align 16 + ; CHECK: %ld.3 = load atomic volatile i32, i32* %word syncscope("singlethread") seq_cst, align 16 store atomic i32 23, i32* %word monotonic, align 4 ; CHECK: store atomic i32 23, i32* %word monotonic, align 4 store atomic volatile i32 24, i32* %word monotonic, align 4 ; CHECK: store atomic volatile i32 24, i32* %word monotonic, align 4 - store atomic volatile i32 25, i32* %word singlethread monotonic, align 4 - ; CHECK: store atomic volatile i32 25, i32* %word singlethread monotonic, align 4 + store atomic volatile i32 25, i32* %word syncscope("singlethread") monotonic, align 4 + ; CHECK: store atomic volatile i32 25, i32* %word syncscope("singlethread") monotonic, align 4 ret void } diff --git a/test/Bitcode/compatibility-4.0.ll b/test/Bitcode/compatibility-4.0.ll index c83c107a292..eef925564ec 100644 --- a/test/Bitcode/compatibility-4.0.ll +++ b/test/Bitcode/compatibility-4.0.ll @@ -698,8 +698,8 @@ define void @atomics(i32* %word) { ; CHECK: %cmpxchg.5 = cmpxchg weak i32* %word, i32 0, i32 9 seq_cst monotonic %cmpxchg.6 = cmpxchg volatile i32* %word, i32 0, i32 10 seq_cst monotonic ; CHECK: %cmpxchg.6 = cmpxchg volatile i32* %word, i32 0, i32 10 seq_cst monotonic - %cmpxchg.7 = cmpxchg weak volatile i32* %word, i32 0, i32 11 singlethread seq_cst monotonic - ; CHECK: %cmpxchg.7 = cmpxchg weak volatile i32* %word, i32 0, i32 11 singlethread seq_cst monotonic + %cmpxchg.7 = cmpxchg weak volatile i32* %word, i32 0, i32 11 syncscope("singlethread") seq_cst monotonic + ; CHECK: %cmpxchg.7 = cmpxchg weak volatile i32* %word, i32 0, i32 11 syncscope("singlethread") seq_cst monotonic %atomicrmw.xchg = atomicrmw xchg i32* %word, i32 12 monotonic ; CHECK: %atomicrmw.xchg = atomicrmw xchg i32* %word, i32 12 monotonic %atomicrmw.add = atomicrmw add i32* %word, i32 13 monotonic @@ -718,32 +718,32 @@ define void @atomics(i32* %word) { ; CHECK: %atomicrmw.max = atomicrmw max i32* %word, i32 19 monotonic %atomicrmw.min = atomicrmw volatile min i32* %word, i32 20 monotonic ; CHECK: %atomicrmw.min = atomicrmw volatile min i32* %word, i32 20 monotonic - %atomicrmw.umax = atomicrmw umax i32* %word, i32 21 singlethread monotonic - ; CHECK: %atomicrmw.umax = atomicrmw umax i32* %word, i32 21 singlethread monotonic - %atomicrmw.umin = atomicrmw volatile umin i32* %word, i32 22 singlethread monotonic - ; CHECK: %atomicrmw.umin = atomicrmw volatile umin i32* %word, i32 22 singlethread monotonic + %atomicrmw.umax = atomicrmw umax i32* %word, i32 21 syncscope("singlethread") monotonic + ; CHECK: %atomicrmw.umax = atomicrmw umax i32* %word, i32 21 syncscope("singlethread") monotonic + %atomicrmw.umin = atomicrmw volatile umin i32* %word, i32 22 syncscope("singlethread") monotonic + ; CHECK: %atomicrmw.umin = atomicrmw volatile umin i32* %word, i32 22 syncscope("singlethread") monotonic fence acquire ; CHECK: fence acquire fence release ; CHECK: fence release fence acq_rel ; CHECK: fence acq_rel - fence singlethread seq_cst - ; CHECK: fence singlethread seq_cst + fence syncscope("singlethread") seq_cst + ; CHECK: fence syncscope("singlethread") seq_cst %ld.1 = load atomic i32, i32* %word monotonic, align 4 ; CHECK: %ld.1 = load atomic i32, i32* %word monotonic, align 4 %ld.2 = load atomic volatile i32, i32* %word acquire, align 8 ; CHECK: %ld.2 = load atomic volatile i32, i32* %word acquire, align 8 - %ld.3 = load atomic volatile i32, i32* %word singlethread seq_cst, align 16 - ; CHECK: %ld.3 = load atomic volatile i32, i32* %word singlethread seq_cst, align 16 + %ld.3 = load atomic volatile i32, i32* %word syncscope("singlethread") seq_cst, align 16 + ; CHECK: %ld.3 = load atomic volatile i32, i32* %word syncscope("singlethread") seq_cst, align 16 store atomic i32 23, i32* %word monotonic, align 4 ; CHECK: store atomic i32 23, i32* %word monotonic, align 4 store atomic volatile i32 24, i32* %word monotonic, align 4 ; CHECK: store atomic volatile i32 24, i32* %word monotonic, align 4 - store atomic volatile i32 25, i32* %word singlethread monotonic, align 4 - ; CHECK: store atomic volatile i32 25, i32* %word singlethread monotonic, align 4 + store atomic volatile i32 25, i32* %word syncscope("singlethread") monotonic, align 4 + ; CHECK: store atomic volatile i32 25, i32* %word syncscope("singlethread") monotonic, align 4 ret void } diff --git a/test/Bitcode/compatibility.ll b/test/Bitcode/compatibility.ll index ec69344947c..ebd727ba9ae 100644 --- a/test/Bitcode/compatibility.ll +++ b/test/Bitcode/compatibility.ll @@ -705,8 +705,8 @@ define void @atomics(i32* %word) { ; CHECK: %cmpxchg.5 = cmpxchg weak i32* %word, i32 0, i32 9 seq_cst monotonic %cmpxchg.6 = cmpxchg volatile i32* %word, i32 0, i32 10 seq_cst monotonic ; CHECK: %cmpxchg.6 = cmpxchg volatile i32* %word, i32 0, i32 10 seq_cst monotonic - %cmpxchg.7 = cmpxchg weak volatile i32* %word, i32 0, i32 11 singlethread seq_cst monotonic - ; CHECK: %cmpxchg.7 = cmpxchg weak volatile i32* %word, i32 0, i32 11 singlethread seq_cst monotonic + %cmpxchg.7 = cmpxchg weak volatile i32* %word, i32 0, i32 11 syncscope("singlethread") seq_cst monotonic + ; CHECK: %cmpxchg.7 = cmpxchg weak volatile i32* %word, i32 0, i32 11 syncscope("singlethread") seq_cst monotonic %atomicrmw.xchg = atomicrmw xchg i32* %word, i32 12 monotonic ; CHECK: %atomicrmw.xchg = atomicrmw xchg i32* %word, i32 12 monotonic %atomicrmw.add = atomicrmw add i32* %word, i32 13 monotonic @@ -725,32 +725,32 @@ define void @atomics(i32* %word) { ; CHECK: %atomicrmw.max = atomicrmw max i32* %word, i32 19 monotonic %atomicrmw.min = atomicrmw volatile min i32* %word, i32 20 monotonic ; CHECK: %atomicrmw.min = atomicrmw volatile min i32* %word, i32 20 monotonic - %atomicrmw.umax = atomicrmw umax i32* %word, i32 21 singlethread monotonic - ; CHECK: %atomicrmw.umax = atomicrmw umax i32* %word, i32 21 singlethread monotonic - %atomicrmw.umin = atomicrmw volatile umin i32* %word, i32 22 singlethread monotonic - ; CHECK: %atomicrmw.umin = atomicrmw volatile umin i32* %word, i32 22 singlethread monotonic + %atomicrmw.umax = atomicrmw umax i32* %word, i32 21 syncscope("singlethread") monotonic + ; CHECK: %atomicrmw.umax = atomicrmw umax i32* %word, i32 21 syncscope("singlethread") monotonic + %atomicrmw.umin = atomicrmw volatile umin i32* %word, i32 22 syncscope("singlethread") monotonic + ; CHECK: %atomicrmw.umin = atomicrmw volatile umin i32* %word, i32 22 syncscope("singlethread") monotonic fence acquire ; CHECK: fence acquire fence release ; CHECK: fence release fence acq_rel ; CHECK: fence acq_rel - fence singlethread seq_cst - ; CHECK: fence singlethread seq_cst + fence syncscope("singlethread") seq_cst + ; CHECK: fence syncscope("singlethread") seq_cst %ld.1 = load atomic i32, i32* %word monotonic, align 4 ; CHECK: %ld.1 = load atomic i32, i32* %word monotonic, align 4 %ld.2 = load atomic volatile i32, i32* %word acquire, align 8 ; CHECK: %ld.2 = load atomic volatile i32, i32* %word acquire, align 8 - %ld.3 = load atomic volatile i32, i32* %word singlethread seq_cst, align 16 - ; CHECK: %ld.3 = load atomic volatile i32, i32* %word singlethread seq_cst, align 16 + %ld.3 = load atomic volatile i32, i32* %word syncscope("singlethread") seq_cst, align 16 + ; CHECK: %ld.3 = load atomic volatile i32, i32* %word syncscope("singlethread") seq_cst, align 16 store atomic i32 23, i32* %word monotonic, align 4 ; CHECK: store atomic i32 23, i32* %word monotonic, align 4 store atomic volatile i32 24, i32* %word monotonic, align 4 ; CHECK: store atomic volatile i32 24, i32* %word monotonic, align 4 - store atomic volatile i32 25, i32* %word singlethread monotonic, align 4 - ; CHECK: store atomic volatile i32 25, i32* %word singlethread monotonic, align 4 + store atomic volatile i32 25, i32* %word syncscope("singlethread") monotonic, align 4 + ; CHECK: store atomic volatile i32 25, i32* %word syncscope("singlethread") monotonic, align 4 ret void } diff --git a/test/Bitcode/memInstructions.3.2.ll b/test/Bitcode/memInstructions.3.2.ll index 1ab05b6d1b4..c530b6d2cb1 100644 --- a/test/Bitcode/memInstructions.3.2.ll +++ b/test/Bitcode/memInstructions.3.2.ll @@ -107,29 +107,29 @@ entry: ; CHECK-NEXT: %res8 = load atomic volatile i8, i8* %ptr1 seq_cst, align 1 %res8 = load atomic volatile i8, i8* %ptr1 seq_cst, align 1 -; CHECK-NEXT: %res9 = load atomic i8, i8* %ptr1 singlethread unordered, align 1 - %res9 = load atomic i8, i8* %ptr1 singlethread unordered, align 1 +; CHECK-NEXT: %res9 = load atomic i8, i8* %ptr1 syncscope("singlethread") unordered, align 1 + %res9 = load atomic i8, i8* %ptr1 syncscope("singlethread") unordered, align 1 -; CHECK-NEXT: %res10 = load atomic i8, i8* %ptr1 singlethread monotonic, align 1 - %res10 = load atomic i8, i8* %ptr1 singlethread monotonic, align 1 +; CHECK-NEXT: %res10 = load atomic i8, i8* %ptr1 syncscope("singlethread") monotonic, align 1 + %res10 = load atomic i8, i8* %ptr1 syncscope("singlethread") monotonic, align 1 -; CHECK-NEXT: %res11 = load atomic i8, i8* %ptr1 singlethread acquire, align 1 - %res11 = load atomic i8, i8* %ptr1 singlethread acquire, align 1 +; CHECK-NEXT: %res11 = load atomic i8, i8* %ptr1 syncscope("singlethread") acquire, align 1 + %res11 = load atomic i8, i8* %ptr1 syncscope("singlethread") acquire, align 1 -; CHECK-NEXT: %res12 = load atomic i8, i8* %ptr1 singlethread seq_cst, align 1 - %res12 = load atomic i8, i8* %ptr1 singlethread seq_cst, align 1 +; CHECK-NEXT: %res12 = load atomic i8, i8* %ptr1 syncscope("singlethread") seq_cst, align 1 + %res12 = load atomic i8, i8* %ptr1 syncscope("singlethread") seq_cst, align 1 -; CHECK-NEXT: %res13 = load atomic volatile i8, i8* %ptr1 singlethread unordered, align 1 - %res13 = load atomic volatile i8, i8* %ptr1 singlethread unordered, align 1 +; CHECK-NEXT: %res13 = load atomic volatile i8, i8* %ptr1 syncscope("singlethread") unordered, align 1 + %res13 = load atomic volatile i8, i8* %ptr1 syncscope("singlethread") unordered, align 1 -; CHECK-NEXT: %res14 = load atomic volatile i8, i8* %ptr1 singlethread monotonic, align 1 - %res14 = load atomic volatile i8, i8* %ptr1 singlethread monotonic, align 1 +; CHECK-NEXT: %res14 = load atomic volatile i8, i8* %ptr1 syncscope("singlethread") monotonic, align 1 + %res14 = load atomic volatile i8, i8* %ptr1 syncscope("singlethread") monotonic, align 1 -; CHECK-NEXT: %res15 = load atomic volatile i8, i8* %ptr1 singlethread acquire, align 1 - %res15 = load atomic volatile i8, i8* %ptr1 singlethread acquire, align 1 +; CHECK-NEXT: %res15 = load atomic volatile i8, i8* %ptr1 syncscope("singlethread") acquire, align 1 + %res15 = load atomic volatile i8, i8* %ptr1 syncscope("singlethread") acquire, align 1 -; CHECK-NEXT: %res16 = load atomic volatile i8, i8* %ptr1 singlethread seq_cst, align 1 - %res16 = load atomic volatile i8, i8* %ptr1 singlethread seq_cst, align 1 +; CHECK-NEXT: %res16 = load atomic volatile i8, i8* %ptr1 syncscope("singlethread") seq_cst, align 1 + %res16 = load atomic volatile i8, i8* %ptr1 syncscope("singlethread") seq_cst, align 1 ret void } @@ -193,29 +193,29 @@ entry: ; CHECK-NEXT: store atomic volatile i8 2, i8* %ptr1 seq_cst, align 1 store atomic volatile i8 2, i8* %ptr1 seq_cst, align 1 -; CHECK-NEXT: store atomic i8 2, i8* %ptr1 singlethread unordered, align 1 - store atomic i8 2, i8* %ptr1 singlethread unordered, align 1 +; CHECK-NEXT: store atomic i8 2, i8* %ptr1 syncscope("singlethread") unordered, align 1 + store atomic i8 2, i8* %ptr1 syncscope("singlethread") unordered, align 1 -; CHECK-NEXT: store atomic i8 2, i8* %ptr1 singlethread monotonic, align 1 - store atomic i8 2, i8* %ptr1 singlethread monotonic, align 1 +; CHECK-NEXT: store atomic i8 2, i8* %ptr1 syncscope("singlethread") monotonic, align 1 + store atomic i8 2, i8* %ptr1 syncscope("singlethread") monotonic, align 1 -; CHECK-NEXT: store atomic i8 2, i8* %ptr1 singlethread release, align 1 - store atomic i8 2, i8* %ptr1 singlethread release, align 1 +; CHECK-NEXT: store atomic i8 2, i8* %ptr1 syncscope("singlethread") release, align 1 + store atomic i8 2, i8* %ptr1 syncscope("singlethread") release, align 1 -; CHECK-NEXT: store atomic i8 2, i8* %ptr1 singlethread seq_cst, align 1 - store atomic i8 2, i8* %ptr1 singlethread seq_cst, align 1 +; CHECK-NEXT: store atomic i8 2, i8* %ptr1 syncscope("singlethread") seq_cst, align 1 + store atomic i8 2, i8* %ptr1 syncscope("singlethread") seq_cst, align 1 -; CHECK-NEXT: store atomic volatile i8 2, i8* %ptr1 singlethread unordered, align 1 - store atomic volatile i8 2, i8* %ptr1 singlethread unordered, align 1 +; CHECK-NEXT: store atomic volatile i8 2, i8* %ptr1 syncscope("singlethread") unordered, align 1 + store atomic volatile i8 2, i8* %ptr1 syncscope("singlethread") unordered, align 1 -; CHECK-NEXT: store atomic volatile i8 2, i8* %ptr1 singlethread monotonic, align 1 - store atomic volatile i8 2, i8* %ptr1 singlethread monotonic, align 1 +; CHECK-NEXT: store atomic volatile i8 2, i8* %ptr1 syncscope("singlethread") monotonic, align 1 + store atomic volatile i8 2, i8* %ptr1 syncscope("singlethread") monotonic, align 1 -; CHECK-NEXT: store atomic volatile i8 2, i8* %ptr1 singlethread release, align 1 - store atomic volatile i8 2, i8* %ptr1 singlethread release, align 1 +; CHECK-NEXT: store atomic volatile i8 2, i8* %ptr1 syncscope("singlethread") release, align 1 + store atomic volatile i8 2, i8* %ptr1 syncscope("singlethread") release, align 1 -; CHECK-NEXT: store atomic volatile i8 2, i8* %ptr1 singlethread seq_cst, align 1 - store atomic volatile i8 2, i8* %ptr1 singlethread seq_cst, align 1 +; CHECK-NEXT: store atomic volatile i8 2, i8* %ptr1 syncscope("singlethread") seq_cst, align 1 + store atomic volatile i8 2, i8* %ptr1 syncscope("singlethread") seq_cst, align 1 ret void } @@ -232,13 +232,13 @@ entry: ; CHECK-NEXT: %res2 = extractvalue { i32, i1 } [[TMP]], 0 %res2 = cmpxchg volatile i32* %ptr, i32 %cmp, i32 %new monotonic monotonic -; CHECK-NEXT: [[TMP:%[a-z0-9]+]] = cmpxchg i32* %ptr, i32 %cmp, i32 %new singlethread monotonic monotonic +; CHECK-NEXT: [[TMP:%[a-z0-9]+]] = cmpxchg i32* %ptr, i32 %cmp, i32 %new syncscope("singlethread") monotonic monotonic ; CHECK-NEXT: %res3 = extractvalue { i32, i1 } [[TMP]], 0 - %res3 = cmpxchg i32* %ptr, i32 %cmp, i32 %new singlethread monotonic monotonic + %res3 = cmpxchg i32* %ptr, i32 %cmp, i32 %new syncscope("singlethread") monotonic monotonic -; CHECK-NEXT: [[TMP:%[a-z0-9]+]] = cmpxchg volatile i32* %ptr, i32 %cmp, i32 %new singlethread monotonic monotonic +; CHECK-NEXT: [[TMP:%[a-z0-9]+]] = cmpxchg volatile i32* %ptr, i32 %cmp, i32 %new syncscope("singlethread") monotonic monotonic ; CHECK-NEXT: %res4 = extractvalue { i32, i1 } [[TMP]], 0 - %res4 = cmpxchg volatile i32* %ptr, i32 %cmp, i32 %new singlethread monotonic monotonic + %res4 = cmpxchg volatile i32* %ptr, i32 %cmp, i32 %new syncscope("singlethread") monotonic monotonic ; CHECK-NEXT: [[TMP:%[a-z0-9]+]] = cmpxchg i32* %ptr, i32 %cmp, i32 %new acquire acquire @@ -249,13 +249,13 @@ entry: ; CHECK-NEXT: %res6 = extractvalue { i32, i1 } [[TMP]], 0 %res6 = cmpxchg volatile i32* %ptr, i32 %cmp, i32 %new acquire acquire -; CHECK-NEXT: [[TMP:%[a-z0-9]+]] = cmpxchg i32* %ptr, i32 %cmp, i32 %new singlethread acquire acquire +; CHECK-NEXT: [[TMP:%[a-z0-9]+]] = cmpxchg i32* %ptr, i32 %cmp, i32 %new syncscope("singlethread") acquire acquire ; CHECK-NEXT: %res7 = extractvalue { i32, i1 } [[TMP]], 0 - %res7 = cmpxchg i32* %ptr, i32 %cmp, i32 %new singlethread acquire acquire + %res7 = cmpxchg i32* %ptr, i32 %cmp, i32 %new syncscope("singlethread") acquire acquire -; CHECK-NEXT: [[TMP:%[a-z0-9]+]] = cmpxchg volatile i32* %ptr, i32 %cmp, i32 %new singlethread acquire acquire +; CHECK-NEXT: [[TMP:%[a-z0-9]+]] = cmpxchg volatile i32* %ptr, i32 %cmp, i32 %new syncscope("singlethread") acquire acquire ; CHECK-NEXT: %res8 = extractvalue { i32, i1 } [[TMP]], 0 - %res8 = cmpxchg volatile i32* %ptr, i32 %cmp, i32 %new singlethread acquire acquire + %res8 = cmpxchg volatile i32* %ptr, i32 %cmp, i32 %new syncscope("singlethread") acquire acquire ; CHECK-NEXT: [[TMP:%[a-z0-9]+]] = cmpxchg i32* %ptr, i32 %cmp, i32 %new release monotonic @@ -266,13 +266,13 @@ entry: ; CHECK-NEXT: %res10 = extractvalue { i32, i1 } [[TMP]], 0 %res10 = cmpxchg volatile i32* %ptr, i32 %cmp, i32 %new release monotonic -; CHECK-NEXT: [[TMP:%[a-z0-9]+]] = cmpxchg i32* %ptr, i32 %cmp, i32 %new singlethread release monotonic +; CHECK-NEXT: [[TMP:%[a-z0-9]+]] = cmpxchg i32* %ptr, i32 %cmp, i32 %new syncscope("singlethread") release monotonic ; CHECK-NEXT: %res11 = extractvalue { i32, i1 } [[TMP]], 0 - %res11 = cmpxchg i32* %ptr, i32 %cmp, i32 %new singlethread release monotonic + %res11 = cmpxchg i32* %ptr, i32 %cmp, i32 %new syncscope("singlethread") release monotonic -; CHECK-NEXT: [[TMP:%[a-z0-9]+]] = cmpxchg volatile i32* %ptr, i32 %cmp, i32 %new singlethread release monotonic +; CHECK-NEXT: [[TMP:%[a-z0-9]+]] = cmpxchg volatile i32* %ptr, i32 %cmp, i32 %new syncscope("singlethread") release monotonic ; CHECK-NEXT: %res12 = extractvalue { i32, i1 } [[TMP]], 0 - %res12 = cmpxchg volatile i32* %ptr, i32 %cmp, i32 %new singlethread release monotonic + %res12 = cmpxchg volatile i32* %ptr, i32 %cmp, i32 %new syncscope("singlethread") release monotonic ; CHECK-NEXT: [[TMP:%[a-z0-9]+]] = cmpxchg i32* %ptr, i32 %cmp, i32 %new acq_rel acquire @@ -283,13 +283,13 @@ entry: ; CHECK-NEXT: %res14 = extractvalue { i32, i1 } [[TMP]], 0 %res14 = cmpxchg volatile i32* %ptr, i32 %cmp, i32 %new acq_rel acquire -; CHECK-NEXT: [[TMP:%[a-z0-9]+]] = cmpxchg i32* %ptr, i32 %cmp, i32 %new singlethread acq_rel acquire +; CHECK-NEXT: [[TMP:%[a-z0-9]+]] = cmpxchg i32* %ptr, i32 %cmp, i32 %new syncscope("singlethread") acq_rel acquire ; CHECK-NEXT: %res15 = extractvalue { i32, i1 } [[TMP]], 0 - %res15 = cmpxchg i32* %ptr, i32 %cmp, i32 %new singlethread acq_rel acquire + %res15 = cmpxchg i32* %ptr, i32 %cmp, i32 %new syncscope("singlethread") acq_rel acquire -; CHECK-NEXT: [[TMP:%[a-z0-9]+]] = cmpxchg volatile i32* %ptr, i32 %cmp, i32 %new singlethread acq_rel acquire +; CHECK-NEXT: [[TMP:%[a-z0-9]+]] = cmpxchg volatile i32* %ptr, i32 %cmp, i32 %new syncscope("singlethread") acq_rel acquire ; CHECK-NEXT: %res16 = extractvalue { i32, i1 } [[TMP]], 0 - %res16 = cmpxchg volatile i32* %ptr, i32 %cmp, i32 %new singlethread acq_rel acquire + %res16 = cmpxchg volatile i32* %ptr, i32 %cmp, i32 %new syncscope("singlethread") acq_rel acquire ; CHECK-NEXT: [[TMP:%[a-z0-9]+]] = cmpxchg i32* %ptr, i32 %cmp, i32 %new seq_cst seq_cst @@ -300,13 +300,13 @@ entry: ; CHECK-NEXT: %res18 = extractvalue { i32, i1 } [[TMP]], 0 %res18 = cmpxchg volatile i32* %ptr, i32 %cmp, i32 %new seq_cst seq_cst -; CHECK-NEXT: [[TMP:%[a-z0-9]+]] = cmpxchg i32* %ptr, i32 %cmp, i32 %new singlethread seq_cst seq_cst +; CHECK-NEXT: [[TMP:%[a-z0-9]+]] = cmpxchg i32* %ptr, i32 %cmp, i32 %new syncscope("singlethread") seq_cst seq_cst ; CHECK-NEXT: %res19 = extractvalue { i32, i1 } [[TMP]], 0 - %res19 = cmpxchg i32* %ptr, i32 %cmp, i32 %new singlethread seq_cst seq_cst + %res19 = cmpxchg i32* %ptr, i32 %cmp, i32 %new syncscope("singlethread") seq_cst seq_cst -; CHECK-NEXT: [[TMP:%[a-z0-9]+]] = cmpxchg volatile i32* %ptr, i32 %cmp, i32 %new singlethread seq_cst seq_cst +; CHECK-NEXT: [[TMP:%[a-z0-9]+]] = cmpxchg volatile i32* %ptr, i32 %cmp, i32 %new syncscope("singlethread") seq_cst seq_cst ; CHECK-NEXT: %res20 = extractvalue { i32, i1 } [[TMP]], 0 - %res20 = cmpxchg volatile i32* %ptr, i32 %cmp, i32 %new singlethread seq_cst seq_cst + %res20 = cmpxchg volatile i32* %ptr, i32 %cmp, i32 %new syncscope("singlethread") seq_cst seq_cst ret void } diff --git a/test/CodeGen/AArch64/GlobalISel/arm64-irtranslator.ll b/test/CodeGen/AArch64/GlobalISel/arm64-irtranslator.ll index 50ad83feed8..10ce87c2a18 100644 --- a/test/CodeGen/AArch64/GlobalISel/arm64-irtranslator.ll +++ b/test/CodeGen/AArch64/GlobalISel/arm64-irtranslator.ll @@ -1328,16 +1328,16 @@ define void @test_load_store_atomics(i8* %addr) { ; CHECK: G_STORE [[V0]](s8), [[ADDR]](p0) :: (store monotonic 1 into %ir.addr) ; CHECK: [[V1:%[0-9]+]](s8) = G_LOAD [[ADDR]](p0) :: (load acquire 1 from %ir.addr) ; CHECK: G_STORE [[V1]](s8), [[ADDR]](p0) :: (store release 1 into %ir.addr) -; CHECK: [[V2:%[0-9]+]](s8) = G_LOAD [[ADDR]](p0) :: (load singlethread seq_cst 1 from %ir.addr) -; CHECK: G_STORE [[V2]](s8), [[ADDR]](p0) :: (store singlethread monotonic 1 into %ir.addr) +; CHECK: [[V2:%[0-9]+]](s8) = G_LOAD [[ADDR]](p0) :: (load syncscope("singlethread") seq_cst 1 from %ir.addr) +; CHECK: G_STORE [[V2]](s8), [[ADDR]](p0) :: (store syncscope("singlethread") monotonic 1 into %ir.addr) %v0 = load atomic i8, i8* %addr unordered, align 1 store atomic i8 %v0, i8* %addr monotonic, align 1 %v1 = load atomic i8, i8* %addr acquire, align 1 store atomic i8 %v1, i8* %addr release, align 1 - %v2 = load atomic i8, i8* %addr singlethread seq_cst, align 1 - store atomic i8 %v2, i8* %addr singlethread monotonic, align 1 + %v2 = load atomic i8, i8* %addr syncscope("singlethread") seq_cst, align 1 + store atomic i8 %v2, i8* %addr syncscope("singlethread") monotonic, align 1 ret void } diff --git a/test/CodeGen/AArch64/fence-singlethread.ll b/test/CodeGen/AArch64/fence-singlethread.ll index 2ed74427738..0af0e58a91d 100644 --- a/test/CodeGen/AArch64/fence-singlethread.ll +++ b/test/CodeGen/AArch64/fence-singlethread.ll @@ -16,6 +16,6 @@ define void @fence_singlethread() { ; IOS: ; COMPILER BARRIER ; IOS-NOT: dmb - fence singlethread seq_cst + fence syncscope("singlethread") seq_cst ret void } diff --git a/test/CodeGen/AMDGPU/syncscopes.ll b/test/CodeGen/AMDGPU/syncscopes.ll new file mode 100644 index 00000000000..3741ce78899 --- /dev/null +++ b/test/CodeGen/AMDGPU/syncscopes.ll @@ -0,0 +1,19 @@ +; RUN: llc -mtriple=amdgcn-amd-amdhsa -mcpu=gfx803 -stop-before=si-debugger-insert-nops < %s | FileCheck --check-prefix=GCN %s + +; GCN-LABEL: name: syncscopes +; GCN: FLAT_STORE_DWORD killed %vgpr1_vgpr2, killed %vgpr0, 0, -1, 0, implicit %exec, implicit %flat_scr :: (volatile store syncscope("agent") seq_cst 4 into %ir.agent_out) +; GCN: FLAT_STORE_DWORD killed %vgpr4_vgpr5, killed %vgpr3, 0, -1, 0, implicit %exec, implicit %flat_scr :: (volatile store syncscope("workgroup") seq_cst 4 into %ir.workgroup_out) +; GCN: FLAT_STORE_DWORD killed %vgpr7_vgpr8, killed %vgpr6, 0, -1, 0, implicit %exec, implicit %flat_scr :: (volatile store syncscope("wavefront") seq_cst 4 into %ir.wavefront_out) +define void @syncscopes( + i32 %agent, + i32 addrspace(4)* %agent_out, + i32 %workgroup, + i32 addrspace(4)* %workgroup_out, + i32 %wavefront, + i32 addrspace(4)* %wavefront_out) { +entry: + store atomic i32 %agent, i32 addrspace(4)* %agent_out syncscope("agent") seq_cst, align 4 + store atomic i32 %workgroup, i32 addrspace(4)* %workgroup_out syncscope("workgroup") seq_cst, align 4 + store atomic i32 %wavefront, i32 addrspace(4)* %wavefront_out syncscope("wavefront") seq_cst, align 4 + ret void +} diff --git a/test/CodeGen/ARM/fence-singlethread.ll b/test/CodeGen/ARM/fence-singlethread.ll index ec032ccac42..536b6cc7c9d 100644 --- a/test/CodeGen/ARM/fence-singlethread.ll +++ b/test/CodeGen/ARM/fence-singlethread.ll @@ -11,6 +11,6 @@ define void @fence_singlethread() { ; CHECK: @ COMPILER BARRIER ; CHECK-NOT: dmb - fence singlethread seq_cst + fence syncscope("singlethread") seq_cst ret void } diff --git a/test/CodeGen/MIR/AArch64/atomic-memoperands.mir b/test/CodeGen/MIR/AArch64/atomic-memoperands.mir index 1fe42a73148..1c81f580bee 100644 --- a/test/CodeGen/MIR/AArch64/atomic-memoperands.mir +++ b/test/CodeGen/MIR/AArch64/atomic-memoperands.mir @@ -14,7 +14,7 @@ # CHECK: %3(s16) = G_LOAD %0(p0) :: (load acquire 2) # CHECK: G_STORE %3(s16), %0(p0) :: (store release 2) # CHECK: G_STORE %2(s32), %0(p0) :: (store acq_rel 4) -# CHECK: G_STORE %1(s64), %0(p0) :: (store singlethread seq_cst 8) +# CHECK: G_STORE %1(s64), %0(p0) :: (store syncscope("singlethread") seq_cst 8) name: atomic_memoperands body: | bb.0: @@ -25,6 +25,6 @@ body: | %3:_(s16) = G_LOAD %0(p0) :: (load acquire 2) G_STORE %3(s16), %0(p0) :: (store release 2) G_STORE %2(s32), %0(p0) :: (store acq_rel 4) - G_STORE %1(s64), %0(p0) :: (store singlethread seq_cst 8) + G_STORE %1(s64), %0(p0) :: (store syncscope("singlethread") seq_cst 8) RET_ReallyLR ... diff --git a/test/CodeGen/MIR/AMDGPU/syncscopes.mir b/test/CodeGen/MIR/AMDGPU/syncscopes.mir new file mode 100644 index 00000000000..83506257d8b --- /dev/null +++ b/test/CodeGen/MIR/AMDGPU/syncscopes.mir @@ -0,0 +1,98 @@ +# RUN: llc -mtriple=amdgcn-amd-amdhsa -mcpu=gfx803 -run-pass=none %s -o - | FileCheck --check-prefix=GCN %s + +--- | + ; ModuleID = '<stdin>' + source_filename = "<stdin>" + target datalayout = "e-p:32:32-p1:64:64-p2:64:64-p3:32:32-p4:64:64-p5:32:32-i64:64-v16:16-v24:32-v32:32-v48:64-v96:128-v192:256-v256:256-v512:512-v1024:1024-v2048:2048-n32:64" + target triple = "amdgcn-amd-amdhsa" + + define void @syncscopes(i32 %agent, i32 addrspace(4)* %agent_out, i32 %workgroup, i32 addrspace(4)* %workgroup_out, i32 %wavefront, i32 addrspace(4)* %wavefront_out) #0 { + entry: + store atomic i32 %agent, i32 addrspace(4)* %agent_out syncscope("agent") seq_cst, align 4 + store atomic i32 %workgroup, i32 addrspace(4)* %workgroup_out syncscope("workgroup") seq_cst, align 4 + store atomic i32 %wavefront, i32 addrspace(4)* %wavefront_out syncscope("wavefront") seq_cst, align 4 + ret void + } + + ; Function Attrs: convergent nounwind + declare { i1, i64 } @llvm.amdgcn.if(i1) #1 + + ; Function Attrs: convergent nounwind + declare { i1, i64 } @llvm.amdgcn.else(i64) #1 + + ; Function Attrs: convergent nounwind readnone + declare i64 @llvm.amdgcn.break(i64) #2 + + ; Function Attrs: convergent nounwind readnone + declare i64 @llvm.amdgcn.if.break(i1, i64) #2 + + ; Function Attrs: convergent nounwind readnone + declare i64 @llvm.amdgcn.else.break(i64, i64) #2 + + ; Function Attrs: convergent nounwind + declare i1 @llvm.amdgcn.loop(i64) #1 + + ; Function Attrs: convergent nounwind + declare void @llvm.amdgcn.end.cf(i64) #1 + + attributes #0 = { "target-cpu"="gfx803" } + attributes #1 = { convergent nounwind } + attributes #2 = { convergent nounwind readnone } + +# GCN-LABEL: name: syncscopes +# GCN: FLAT_STORE_DWORD killed %vgpr0_vgpr1, killed %vgpr2, 0, -1, 0, implicit %exec, implicit %flat_scr :: (volatile store syncscope("agent") seq_cst 4 into %ir.agent_out) +# GCN: FLAT_STORE_DWORD killed %vgpr0_vgpr1, killed %vgpr2, 0, -1, 0, implicit %exec, implicit %flat_scr :: (volatile store syncscope("workgroup") seq_cst 4 into %ir.workgroup_out) +# GCN: FLAT_STORE_DWORD killed %vgpr0_vgpr1, killed %vgpr2, 0, -1, 0, implicit %exec, implicit %flat_scr :: (volatile store syncscope("wavefront") seq_cst 4 into %ir.wavefront_out) +... +--- +name: syncscopes +alignment: 0 +exposesReturnsTwice: false +legalized: false +regBankSelected: false +selected: false +tracksRegLiveness: true +liveins: + - { reg: '%sgpr4_sgpr5' } +frameInfo: + isFrameAddressTaken: false + isReturnAddressTaken: false + hasStackMap: false + hasPatchPoint: false + stackSize: 0 + offsetAdjustment: 0 + maxAlignment: 0 + adjustsStack: false + hasCalls: false + hasOpaqueSPAdjustment: false + hasVAStart: false + hasMustTailInVarArgFunc: false +body: | + bb.0.entry: + liveins: %sgpr4_sgpr5 + + S_WAITCNT 0 + %sgpr0_sgpr1 = S_LOAD_DWORDX2_IMM %sgpr4_sgpr5, 8, 0 :: (non-temporal dereferenceable invariant load 8 from `i64 addrspace(2)* undef`) + %sgpr6 = S_LOAD_DWORD_IMM %sgpr4_sgpr5, 0, 0 :: (non-temporal dereferenceable invariant load 4 from `i32 addrspace(2)* undef`) + %sgpr2_sgpr3 = S_LOAD_DWORDX2_IMM %sgpr4_sgpr5, 24, 0 :: (non-temporal dereferenceable invariant load 8 from `i64 addrspace(2)* undef`) + %sgpr7 = S_LOAD_DWORD_IMM %sgpr4_sgpr5, 16, 0 :: (non-temporal dereferenceable invariant load 4 from `i32 addrspace(2)* undef`) + %sgpr8 = S_LOAD_DWORD_IMM %sgpr4_sgpr5, 32, 0 :: (non-temporal dereferenceable invariant load 4 from `i32 addrspace(2)* undef`) + S_WAITCNT 127 + %vgpr0 = V_MOV_B32_e32 %sgpr0, implicit %exec, implicit-def %vgpr0_vgpr1, implicit %sgpr0_sgpr1 + %sgpr4_sgpr5 = S_LOAD_DWORDX2_IMM killed %sgpr4_sgpr5, 40, 0 :: (non-temporal dereferenceable invariant load 8 from `i64 addrspace(2)* undef`) + %vgpr1 = V_MOV_B32_e32 killed %sgpr1, implicit %exec, implicit killed %sgpr0_sgpr1, implicit %sgpr0_sgpr1, implicit %exec + %vgpr2 = V_MOV_B32_e32 killed %sgpr6, implicit %exec, implicit %exec + FLAT_STORE_DWORD killed %vgpr0_vgpr1, killed %vgpr2, 0, -1, 0, implicit %exec, implicit %flat_scr :: (volatile store syncscope("agent") seq_cst 4 into %ir.agent_out) + S_WAITCNT 112 + %vgpr0 = V_MOV_B32_e32 %sgpr2, implicit %exec, implicit-def %vgpr0_vgpr1, implicit %sgpr2_sgpr3 + %vgpr1 = V_MOV_B32_e32 killed %sgpr3, implicit %exec, implicit killed %sgpr2_sgpr3, implicit %sgpr2_sgpr3, implicit %exec + %vgpr2 = V_MOV_B32_e32 killed %sgpr7, implicit %exec, implicit %exec + FLAT_STORE_DWORD killed %vgpr0_vgpr1, killed %vgpr2, 0, -1, 0, implicit %exec, implicit %flat_scr :: (volatile store syncscope("workgroup") seq_cst 4 into %ir.workgroup_out) + S_WAITCNT 112 + %vgpr0 = V_MOV_B32_e32 %sgpr4, implicit %exec, implicit-def %vgpr0_vgpr1, implicit %sgpr4_sgpr5 + %vgpr1 = V_MOV_B32_e32 killed %sgpr5, implicit %exec, implicit killed %sgpr4_sgpr5, implicit %sgpr4_sgpr5, implicit %exec + %vgpr2 = V_MOV_B32_e32 killed %sgpr8, implicit %exec, implicit %exec + FLAT_STORE_DWORD killed %vgpr0_vgpr1, killed %vgpr2, 0, -1, 0, implicit %exec, implicit %flat_scr :: (volatile store syncscope("wavefront") seq_cst 4 into %ir.wavefront_out) + S_ENDPGM + +... diff --git a/test/CodeGen/PowerPC/atomics-regression.ll b/test/CodeGen/PowerPC/atomics-regression.ll index d57b3a20379..0c7a31d16b1 100644 --- a/test/CodeGen/PowerPC/atomics-regression.ll +++ b/test/CodeGen/PowerPC/atomics-regression.ll @@ -370,7 +370,7 @@ define void @test36() { ; PPC64LE: # BB#0: ; PPC64LE-NEXT: lwsync ; PPC64LE-NEXT: blr - fence singlethread acquire + fence syncscope("singlethread") acquire ret void } @@ -379,7 +379,7 @@ define void @test37() { ; PPC64LE: # BB#0: ; PPC64LE-NEXT: lwsync ; PPC64LE-NEXT: blr - fence singlethread release + fence syncscope("singlethread") release ret void } @@ -388,7 +388,7 @@ define void @test38() { ; PPC64LE: # BB#0: ; PPC64LE-NEXT: lwsync ; PPC64LE-NEXT: blr - fence singlethread acq_rel + fence syncscope("singlethread") acq_rel ret void } @@ -397,7 +397,7 @@ define void @test39() { ; PPC64LE: # BB#0: ; PPC64LE-NEXT: sync ; PPC64LE-NEXT: blr - fence singlethread seq_cst + fence syncscope("singlethread") seq_cst ret void } @@ -1273,7 +1273,7 @@ define void @test80(i8* %ptr, i8 %cmp, i8 %val) { ; PPC64LE-NEXT: # BB#3: ; PPC64LE-NEXT: stbcx. 6, 0, 3 ; PPC64LE-NEXT: blr - %res = cmpxchg i8* %ptr, i8 %cmp, i8 %val singlethread monotonic monotonic + %res = cmpxchg i8* %ptr, i8 %cmp, i8 %val syncscope("singlethread") monotonic monotonic ret void } @@ -1294,7 +1294,7 @@ define void @test81(i8* %ptr, i8 %cmp, i8 %val) { ; PPC64LE-NEXT: stbcx. 6, 0, 3 ; PPC64LE-NEXT: lwsync ; PPC64LE-NEXT: blr - %res = cmpxchg i8* %ptr, i8 %cmp, i8 %val singlethread acquire monotonic + %res = cmpxchg i8* %ptr, i8 %cmp, i8 %val syncscope("singlethread") acquire monotonic ret void } @@ -1315,7 +1315,7 @@ define void @test82(i8* %ptr, i8 %cmp, i8 %val) { ; PPC64LE-NEXT: stbcx. 6, 0, 3 ; PPC64LE-NEXT: lwsync ; PPC64LE-NEXT: blr - %res = cmpxchg i8* %ptr, i8 %cmp, i8 %val singlethread acquire acquire + %res = cmpxchg i8* %ptr, i8 %cmp, i8 %val syncscope("singlethread") acquire acquire ret void } @@ -1336,7 +1336,7 @@ define void @test83(i8* %ptr, i8 %cmp, i8 %val) { ; PPC64LE-NEXT: # BB#3: ; PPC64LE-NEXT: stbcx. 6, 0, 3 ; PPC64LE-NEXT: blr - %res = cmpxchg i8* %ptr, i8 %cmp, i8 %val singlethread release monotonic + %res = cmpxchg i8* %ptr, i8 %cmp, i8 %val syncscope("singlethread") release monotonic ret void } @@ -1357,7 +1357,7 @@ define void @test84(i8* %ptr, i8 %cmp, i8 %val) { ; PPC64LE-NEXT: # BB#3: ; PPC64LE-NEXT: stbcx. 6, 0, 3 ; PPC64LE-NEXT: blr - %res = cmpxchg i8* %ptr, i8 %cmp, i8 %val singlethread release acquire + %res = cmpxchg i8* %ptr, i8 %cmp, i8 %val syncscope("singlethread") release acquire ret void } @@ -1379,7 +1379,7 @@ define void @test85(i8* %ptr, i8 %cmp, i8 %val) { ; PPC64LE-NEXT: stbcx. 6, 0, 3 ; PPC64LE-NEXT: lwsync ; PPC64LE-NEXT: blr - %res = cmpxchg i8* %ptr, i8 %cmp, i8 %val singlethread acq_rel monotonic + %res = cmpxchg i8* %ptr, i8 %cmp, i8 %val syncscope("singlethread") acq_rel monotonic ret void } @@ -1401,7 +1401,7 @@ define void @test86(i8* %ptr, i8 %cmp, i8 %val) { ; PPC64LE-NEXT: stbcx. 6, 0, 3 ; PPC64LE-NEXT: lwsync ; PPC64LE-NEXT: blr - %res = cmpxchg i8* %ptr, i8 %cmp, i8 %val singlethread acq_rel acquire + %res = cmpxchg i8* %ptr, i8 %cmp, i8 %val syncscope("singlethread") acq_rel acquire ret void } @@ -1423,7 +1423,7 @@ define void @test87(i8* %ptr, i8 %cmp, i8 %val) { ; PPC64LE-NEXT: stbcx. 6, 0, 3 ; PPC64LE-NEXT: lwsync ; PPC64LE-NEXT: blr - %res = cmpxchg i8* %ptr, i8 %cmp, i8 %val singlethread seq_cst monotonic + %res = cmpxchg i8* %ptr, i8 %cmp, i8 %val syncscope("singlethread") seq_cst monotonic ret void } @@ -1445,7 +1445,7 @@ define void @test88(i8* %ptr, i8 %cmp, i8 %val) { ; PPC64LE-NEXT: stbcx. 6, 0, 3 ; PPC64LE-NEXT: lwsync ; PPC64LE-NEXT: blr - %res = cmpxchg i8* %ptr, i8 %cmp, i8 %val singlethread seq_cst acquire + %res = cmpxchg i8* %ptr, i8 %cmp, i8 %val syncscope("singlethread") seq_cst acquire ret void } @@ -1467,7 +1467,7 @@ define void @test89(i8* %ptr, i8 %cmp, i8 %val) { ; PPC64LE-NEXT: stbcx. 6, 0, 3 ; PPC64LE-NEXT: lwsync ; PPC64LE-NEXT: blr - %res = cmpxchg i8* %ptr, i8 %cmp, i8 %val singlethread seq_cst seq_cst + %res = cmpxchg i8* %ptr, i8 %cmp, i8 %val syncscope("singlethread") seq_cst seq_cst ret void } @@ -1487,7 +1487,7 @@ define void @test90(i16* %ptr, i16 %cmp, i16 %val) { ; PPC64LE-NEXT: # BB#3: ; PPC64LE-NEXT: sthcx. 6, 0, 3 ; PPC64LE-NEXT: blr - %res = cmpxchg i16* %ptr, i16 %cmp, i16 %val singlethread monotonic monotonic + %res = cmpxchg i16* %ptr, i16 %cmp, i16 %val syncscope("singlethread") monotonic monotonic ret void } @@ -1508,7 +1508,7 @@ define void @test91(i16* %ptr, i16 %cmp, i16 %val) { ; PPC64LE-NEXT: sthcx. 6, 0, 3 ; PPC64LE-NEXT: lwsync ; PPC64LE-NEXT: blr - %res = cmpxchg i16* %ptr, i16 %cmp, i16 %val singlethread acquire monotonic + %res = cmpxchg i16* %ptr, i16 %cmp, i16 %val syncscope("singlethread") acquire monotonic ret void } @@ -1529,7 +1529,7 @@ define void @test92(i16* %ptr, i16 %cmp, i16 %val) { ; PPC64LE-NEXT: sthcx. 6, 0, 3 ; PPC64LE-NEXT: lwsync ; PPC64LE-NEXT: blr - %res = cmpxchg i16* %ptr, i16 %cmp, i16 %val singlethread acquire acquire + %res = cmpxchg i16* %ptr, i16 %cmp, i16 %val syncscope("singlethread") acquire acquire ret void } @@ -1550,7 +1550,7 @@ define void @test93(i16* %ptr, i16 %cmp, i16 %val) { ; PPC64LE-NEXT: # BB#3: ; PPC64LE-NEXT: sthcx. 6, 0, 3 ; PPC64LE-NEXT: blr - %res = cmpxchg i16* %ptr, i16 %cmp, i16 %val singlethread release monotonic + %res = cmpxchg i16* %ptr, i16 %cmp, i16 %val syncscope("singlethread") release monotonic ret void } @@ -1571,7 +1571,7 @@ define void @test94(i16* %ptr, i16 %cmp, i16 %val) { ; PPC64LE-NEXT: # BB#3: ; PPC64LE-NEXT: sthcx. 6, 0, 3 ; PPC64LE-NEXT: blr - %res = cmpxchg i16* %ptr, i16 %cmp, i16 %val singlethread release acquire + %res = cmpxchg i16* %ptr, i16 %cmp, i16 %val syncscope("singlethread") release acquire ret void } @@ -1593,7 +1593,7 @@ define void @test95(i16* %ptr, i16 %cmp, i16 %val) { ; PPC64LE-NEXT: sthcx. 6, 0, 3 ; PPC64LE-NEXT: lwsync ; PPC64LE-NEXT: blr - %res = cmpxchg i16* %ptr, i16 %cmp, i16 %val singlethread acq_rel monotonic + %res = cmpxchg i16* %ptr, i16 %cmp, i16 %val syncscope("singlethread") acq_rel monotonic ret void } @@ -1615,7 +1615,7 @@ define void @test96(i16* %ptr, i16 %cmp, i16 %val) { ; PPC64LE-NEXT: sthcx. 6, 0, 3 ; PPC64LE-NEXT: lwsync ; PPC64LE-NEXT: blr - %res = cmpxchg i16* %ptr, i16 %cmp, i16 %val singlethread acq_rel acquire + %res = cmpxchg i16* %ptr, i16 %cmp, i16 %val syncscope("singlethread") acq_rel acquire ret void } @@ -1637,7 +1637,7 @@ define void @test97(i16* %ptr, i16 %cmp, i16 %val) { ; PPC64LE-NEXT: sthcx. 6, 0, 3 ; PPC64LE-NEXT: lwsync ; PPC64LE-NEXT: blr - %res = cmpxchg i16* %ptr, i16 %cmp, i16 %val singlethread seq_cst monotonic + %res = cmpxchg i16* %ptr, i16 %cmp, i16 %val syncscope("singlethread") seq_cst monotonic ret void } @@ -1659,7 +1659,7 @@ define void @test98(i16* %ptr, i16 %cmp, i16 %val) { ; PPC64LE-NEXT: sthcx. 6, 0, 3 ; PPC64LE-NEXT: lwsync ; PPC64LE-NEXT: blr - %res = cmpxchg i16* %ptr, i16 %cmp, i16 %val singlethread seq_cst acquire + %res = cmpxchg i16* %ptr, i16 %cmp, i16 %val syncscope("singlethread") seq_cst acquire ret void } @@ -1681,7 +1681,7 @@ define void @test99(i16* %ptr, i16 %cmp, i16 %val) { ; PPC64LE-NEXT: sthcx. 6, 0, 3 ; PPC64LE-NEXT: lwsync ; PPC64LE-NEXT: blr - %res = cmpxchg i16* %ptr, i16 %cmp, i16 %val singlethread seq_cst seq_cst + %res = cmpxchg i16* %ptr, i16 %cmp, i16 %val syncscope("singlethread") seq_cst seq_cst ret void } @@ -1701,7 +1701,7 @@ define void @test100(i32* %ptr, i32 %cmp, i32 %val) { ; PPC64LE-NEXT: # BB#3: ; PPC64LE-NEXT: stwcx. 6, 0, 3 ; PPC64LE-NEXT: blr - %res = cmpxchg i32* %ptr, i32 %cmp, i32 %val singlethread monotonic monotonic + %res = cmpxchg i32* %ptr, i32 %cmp, i32 %val syncscope("singlethread") monotonic monotonic ret void } @@ -1722,7 +1722,7 @@ define void @test101(i32* %ptr, i32 %cmp, i32 %val) { ; PPC64LE-NEXT: stwcx. 6, 0, 3 ; PPC64LE-NEXT: lwsync ; PPC64LE-NEXT: blr - %res = cmpxchg i32* %ptr, i32 %cmp, i32 %val singlethread acquire monotonic + %res = cmpxchg i32* %ptr, i32 %cmp, i32 %val syncscope("singlethread") acquire monotonic ret void } @@ -1743,7 +1743,7 @@ define void @test102(i32* %ptr, i32 %cmp, i32 %val) { ; PPC64LE-NEXT: stwcx. 6, 0, 3 ; PPC64LE-NEXT: lwsync ; PPC64LE-NEXT: blr - %res = cmpxchg i32* %ptr, i32 %cmp, i32 %val singlethread acquire acquire + %res = cmpxchg i32* %ptr, i32 %cmp, i32 %val syncscope("singlethread") acquire acquire ret void } @@ -1764,7 +1764,7 @@ define void @test103(i32* %ptr, i32 %cmp, i32 %val) { ; PPC64LE-NEXT: # BB#3: ; PPC64LE-NEXT: stwcx. 6, 0, 3 ; PPC64LE-NEXT: blr - %res = cmpxchg i32* %ptr, i32 %cmp, i32 %val singlethread release monotonic + %res = cmpxchg i32* %ptr, i32 %cmp, i32 %val syncscope("singlethread") release monotonic ret void } @@ -1785,7 +1785,7 @@ define void @test104(i32* %ptr, i32 %cmp, i32 %val) { ; PPC64LE-NEXT: # BB#3: ; PPC64LE-NEXT: stwcx. 6, 0, 3 ; PPC64LE-NEXT: blr - %res = cmpxchg i32* %ptr, i32 %cmp, i32 %val singlethread release acquire + %res = cmpxchg i32* %ptr, i32 %cmp, i32 %val syncscope("singlethread") release acquire ret void } @@ -1807,7 +1807,7 @@ define void @test105(i32* %ptr, i32 %cmp, i32 %val) { ; PPC64LE-NEXT: stwcx. 6, 0, 3 ; PPC64LE-NEXT: lwsync ; PPC64LE-NEXT: blr - %res = cmpxchg i32* %ptr, i32 %cmp, i32 %val singlethread acq_rel monotonic + %res = cmpxchg i32* %ptr, i32 %cmp, i32 %val syncscope("singlethread") acq_rel monotonic ret void } @@ -1829,7 +1829,7 @@ define void @test106(i32* %ptr, i32 %cmp, i32 %val) { ; PPC64LE-NEXT: stwcx. 6, 0, 3 ; PPC64LE-NEXT: lwsync ; PPC64LE-NEXT: blr - %res = cmpxchg i32* %ptr, i32 %cmp, i32 %val singlethread acq_rel acquire + %res = cmpxchg i32* %ptr, i32 %cmp, i32 %val syncscope("singlethread") acq_rel acquire ret void } @@ -1851,7 +1851,7 @@ define void @test107(i32* %ptr, i32 %cmp, i32 %val) { ; PPC64LE-NEXT: stwcx. 6, 0, 3 ; PPC64LE-NEXT: lwsync ; PPC64LE-NEXT: blr - %res = cmpxchg i32* %ptr, i32 %cmp, i32 %val singlethread seq_cst monotonic + %res = cmpxchg i32* %ptr, i32 %cmp, i32 %val syncscope("singlethread") seq_cst monotonic ret void } @@ -1873,7 +1873,7 @@ define void @test108(i32* %ptr, i32 %cmp, i32 %val) { ; PPC64LE-NEXT: stwcx. 6, 0, 3 ; PPC64LE-NEXT: lwsync ; PPC64LE-NEXT: blr - %res = cmpxchg i32* %ptr, i32 %cmp, i32 %val singlethread seq_cst acquire + %res = cmpxchg i32* %ptr, i32 %cmp, i32 %val syncscope("singlethread") seq_cst acquire ret void } @@ -1895,7 +1895,7 @@ define void @test109(i32* %ptr, i32 %cmp, i32 %val) { ; PPC64LE-NEXT: stwcx. 6, 0, 3 ; PPC64LE-NEXT: lwsync ; PPC64LE-NEXT: blr - %res = cmpxchg i32* %ptr, i32 %cmp, i32 %val singlethread seq_cst seq_cst + %res = cmpxchg i32* %ptr, i32 %cmp, i32 %val syncscope("singlethread") seq_cst seq_cst ret void } @@ -1915,7 +1915,7 @@ define void @test110(i64* %ptr, i64 %cmp, i64 %val) { ; PPC64LE-NEXT: # BB#3: ; PPC64LE-NEXT: stdcx. 6, 0, 3 ; PPC64LE-NEXT: blr - %res = cmpxchg i64* %ptr, i64 %cmp, i64 %val singlethread monotonic monotonic + %res = cmpxchg i64* %ptr, i64 %cmp, i64 %val syncscope("singlethread") monotonic monotonic ret void } @@ -1936,7 +1936,7 @@ define void @test111(i64* %ptr, i64 %cmp, i64 %val) { ; PPC64LE-NEXT: stdcx. 6, 0, 3 ; PPC64LE-NEXT: lwsync ; PPC64LE-NEXT: blr - %res = cmpxchg i64* %ptr, i64 %cmp, i64 %val singlethread acquire monotonic + %res = cmpxchg i64* %ptr, i64 %cmp, i64 %val syncscope("singlethread") acquire monotonic ret void } @@ -1957,7 +1957,7 @@ define void @test112(i64* %ptr, i64 %cmp, i64 %val) { ; PPC64LE-NEXT: stdcx. 6, 0, 3 ; PPC64LE-NEXT: lwsync ; PPC64LE-NEXT: blr - %res = cmpxchg i64* %ptr, i64 %cmp, i64 %val singlethread acquire acquire + %res = cmpxchg i64* %ptr, i64 %cmp, i64 %val syncscope("singlethread") acquire acquire ret void } @@ -1978,7 +1978,7 @@ define void @test113(i64* %ptr, i64 %cmp, i64 %val) { ; PPC64LE-NEXT: # BB#3: ; PPC64LE-NEXT: stdcx. 6, 0, 3 ; PPC64LE-NEXT: blr - %res = cmpxchg i64* %ptr, i64 %cmp, i64 %val singlethread release monotonic + %res = cmpxchg i64* %ptr, i64 %cmp, i64 %val syncscope("singlethread") release monotonic ret void } @@ -1999,7 +1999,7 @@ define void @test114(i64* %ptr, i64 %cmp, i64 %val) { ; PPC64LE-NEXT: # BB#3: ; PPC64LE-NEXT: stdcx. 6, 0, 3 ; PPC64LE-NEXT: blr - %res = cmpxchg i64* %ptr, i64 %cmp, i64 %val singlethread release acquire + %res = cmpxchg i64* %ptr, i64 %cmp, i64 %val syncscope("singlethread") release acquire ret void } @@ -2021,7 +2021,7 @@ define void @test115(i64* %ptr, i64 %cmp, i64 %val) { ; PPC64LE-NEXT: stdcx. 6, 0, 3 ; PPC64LE-NEXT: lwsync ; PPC64LE-NEXT: blr - %res = cmpxchg i64* %ptr, i64 %cmp, i64 %val singlethread acq_rel monotonic + %res = cmpxchg i64* %ptr, i64 %cmp, i64 %val syncscope("singlethread") acq_rel monotonic ret void } @@ -2043,7 +2043,7 @@ define void @test116(i64* %ptr, i64 %cmp, i64 %val) { ; PPC64LE-NEXT: stdcx. 6, 0, 3 ; PPC64LE-NEXT: lwsync ; PPC64LE-NEXT: blr - %res = cmpxchg i64* %ptr, i64 %cmp, i64 %val singlethread acq_rel acquire + %res = cmpxchg i64* %ptr, i64 %cmp, i64 %val syncscope("singlethread") acq_rel acquire ret void } @@ -2065,7 +2065,7 @@ define void @test117(i64* %ptr, i64 %cmp, i64 %val) { ; PPC64LE-NEXT: stdcx. 6, 0, 3 ; PPC64LE-NEXT: lwsync ; PPC64LE-NEXT: blr - %res = cmpxchg i64* %ptr, i64 %cmp, i64 %val singlethread seq_cst monotonic + %res = cmpxchg i64* %ptr, i64 %cmp, i64 %val syncscope("singlethread") seq_cst monotonic ret void } @@ -2087,7 +2087,7 @@ define void @test118(i64* %ptr, i64 %cmp, i64 %val) { ; PPC64LE-NEXT: stdcx. 6, 0, 3 ; PPC64LE-NEXT: lwsync ; PPC64LE-NEXT: blr - %res = cmpxchg i64* %ptr, i64 %cmp, i64 %val singlethread seq_cst acquire + %res = cmpxchg i64* %ptr, i64 %cmp, i64 %val syncscope("singlethread") seq_cst acquire ret void } @@ -2109,7 +2109,7 @@ define void @test119(i64* %ptr, i64 %cmp, i64 %val) { ; PPC64LE-NEXT: stdcx. 6, 0, 3 ; PPC64LE-NEXT: lwsync ; PPC64LE-NEXT: blr - %res = cmpxchg i64* %ptr, i64 %cmp, i64 %val singlethread seq_cst seq_cst + %res = cmpxchg i64* %ptr, i64 %cmp, i64 %val syncscope("singlethread") seq_cst seq_cst ret void } @@ -5847,7 +5847,7 @@ define i8 @test340(i8* %ptr, i8 %val) { ; PPC64LE-NEXT: # BB#2: ; PPC64LE-NEXT: mr 3, 5 ; PPC64LE-NEXT: blr - %ret = atomicrmw xchg i8* %ptr, i8 %val singlethread monotonic + %ret = atomicrmw xchg i8* %ptr, i8 %val syncscope("singlethread") monotonic ret i8 %ret } @@ -5862,7 +5862,7 @@ define i8 @test341(i8* %ptr, i8 %val) { ; PPC64LE-NEXT: # BB#2: ; PPC64LE-NEXT: lwsync ; PPC64LE-NEXT: blr - %ret = atomicrmw xchg i8* %ptr, i8 %val singlethread acquire + %ret = atomicrmw xchg i8* %ptr, i8 %val syncscope("singlethread") acquire ret i8 %ret } @@ -5877,7 +5877,7 @@ define i8 @test342(i8* %ptr, i8 %val) { ; PPC64LE-NEXT: # BB#2: ; PPC64LE-NEXT: mr 3, 5 ; PPC64LE-NEXT: blr - %ret = atomicrmw xchg i8* %ptr, i8 %val singlethread release + %ret = atomicrmw xchg i8* %ptr, i8 %val syncscope("singlethread") release ret i8 %ret } @@ -5893,7 +5893,7 @@ define i8 @test343(i8* %ptr, i8 %val) { ; PPC64LE-NEXT: mr 3, 5 ; PPC64LE-NEXT: lwsync ; PPC64LE-NEXT: blr - %ret = atomicrmw xchg i8* %ptr, i8 %val singlethread acq_rel + %ret = atomicrmw xchg i8* %ptr, i8 %val syncscope("singlethread") acq_rel ret i8 %ret } @@ -5909,7 +5909,7 @@ define i8 @test344(i8* %ptr, i8 %val) { ; PPC64LE-NEXT: mr 3, 5 ; PPC64LE-NEXT: lwsync ; PPC64LE-NEXT: blr - %ret = atomicrmw xchg i8* %ptr, i8 %val singlethread seq_cst + %ret = atomicrmw xchg i8* %ptr, i8 %val syncscope("singlethread") seq_cst ret i8 %ret } @@ -5923,7 +5923,7 @@ define i16 @test345(i16* %ptr, i16 %val) { ; PPC64LE-NEXT: # BB#2: ; PPC64LE-NEXT: mr 3, 5 ; PPC64LE-NEXT: blr - %ret = atomicrmw xchg i16* %ptr, i16 %val singlethread monotonic + %ret = atomicrmw xchg i16* %ptr, i16 %val syncscope("singlethread") monotonic ret i16 %ret } @@ -5938,7 +5938,7 @@ define i16 @test346(i16* %ptr, i16 %val) { ; PPC64LE-NEXT: # BB#2: ; PPC64LE-NEXT: lwsync ; PPC64LE-NEXT: blr - %ret = atomicrmw xchg i16* %ptr, i16 %val singlethread acquire + %ret = atomicrmw xchg i16* %ptr, i16 %val syncscope("singlethread") acquire ret i16 %ret } @@ -5953,7 +5953,7 @@ define i16 @test347(i16* %ptr, i16 %val) { ; PPC64LE-NEXT: # BB#2: ; PPC64LE-NEXT: mr 3, 5 ; PPC64LE-NEXT: blr - %ret = atomicrmw xchg i16* %ptr, i16 %val singlethread release + %ret = atomicrmw xchg i16* %ptr, i16 %val syncscope("singlethread") release ret i16 %ret } @@ -5969,7 +5969,7 @@ define i16 @test348(i16* %ptr, i16 %val) { ; PPC64LE-NEXT: mr 3, 5 ; PPC64LE-NEXT: lwsync ; PPC64LE-NEXT: blr - %ret = atomicrmw xchg i16* %ptr, i16 %val singlethread acq_rel + %ret = atomicrmw xchg i16* %ptr, i16 %val syncscope("singlethread") acq_rel ret i16 %ret } @@ -5985,7 +5985,7 @@ define i16 @test349(i16* %ptr, i16 %val) { ; PPC64LE-NEXT: mr 3, 5 ; PPC64LE-NEXT: lwsync ; PPC64LE-NEXT: blr - %ret = atomicrmw xchg i16* %ptr, i16 %val singlethread seq_cst + %ret = atomicrmw xchg i16* %ptr, i16 %val syncscope("singlethread") seq_cst ret i16 %ret } @@ -5999,7 +5999,7 @@ define i32 @test350(i32* %ptr, i32 %val) { ; PPC64LE-NEXT: # BB#2: ; PPC64LE-NEXT: mr 3, 5 ; PPC64LE-NEXT: blr - %ret = atomicrmw xchg i32* %ptr, i32 %val singlethread monotonic + %ret = atomicrmw xchg i32* %ptr, i32 %val syncscope("singlethread") monotonic ret i32 %ret } @@ -6014,7 +6014,7 @@ define i32 @test351(i32* %ptr, i32 %val) { ; PPC64LE-NEXT: # BB#2: ; PPC64LE-NEXT: lwsync ; PPC64LE-NEXT: blr - %ret = atomicrmw xchg i32* %ptr, i32 %val singlethread acquire + %ret = atomicrmw xchg i32* %ptr, i32 %val syncscope("singlethread") acquire ret i32 %ret } @@ -6029,7 +6029,7 @@ define i32 @test352(i32* %ptr, i32 %val) { ; PPC64LE-NEXT: # BB#2: ; PPC64LE-NEXT: mr 3, 5 ; PPC64LE-NEXT: blr - %ret = atomicrmw xchg i32* %ptr, i32 %val singlethread release + %ret = atomicrmw xchg i32* %ptr, i32 %val syncscope("singlethread") release ret i32 %ret } @@ -6045,7 +6045,7 @@ define i32 @test353(i32* %ptr, i32 %val) { ; PPC64LE-NEXT: mr 3, 5 ; PPC64LE-NEXT: lwsync ; PPC64LE-NEXT: blr - %ret = atomicrmw xchg i32* %ptr, i32 %val singlethread acq_rel + %ret = atomicrmw xchg i32* %ptr, i32 %val syncscope("singlethread") acq_rel ret i32 %ret } @@ -6061,7 +6061,7 @@ define i32 @test354(i32* %ptr, i32 %val) { ; PPC64LE-NEXT: mr 3, 5 ; PPC64LE-NEXT: lwsync ; PPC64LE-NEXT: blr - %ret = atomicrmw xchg i32* %ptr, i32 %val singlethread seq_cst + %ret = atomicrmw xchg i32* %ptr, i32 %val syncscope("singlethread") seq_cst ret i32 %ret } @@ -6075,7 +6075,7 @@ define i64 @test355(i64* %ptr, i64 %val) { ; PPC64LE-NEXT: # BB#2: ; PPC64LE-NEXT: mr 3, 5 ; PPC64LE-NEXT: blr - %ret = atomicrmw xchg i64* %ptr, i64 %val singlethread monotonic + %ret = atomicrmw xchg i64* %ptr, i64 %val syncscope("singlethread") monotonic ret i64 %ret } @@ -6090,7 +6090,7 @@ define i64 @test356(i64* %ptr, i64 %val) { ; PPC64LE-NEXT: # BB#2: ; PPC64LE-NEXT: lwsync ; PPC64LE-NEXT: blr - %ret = atomicrmw xchg i64* %ptr, i64 %val singlethread acquire + %ret = atomicrmw xchg i64* %ptr, i64 %val syncscope("singlethread") acquire ret i64 %ret } @@ -6105,7 +6105,7 @@ define i64 @test357(i64* %ptr, i64 %val) { ; PPC64LE-NEXT: # BB#2: ; PPC64LE-NEXT: mr 3, 5 ; PPC64LE-NEXT: blr - %ret = atomicrmw xchg i64* %ptr, i64 %val singlethread release + %ret = atomicrmw xchg i64* %ptr, i64 %val syncscope("singlethread") release ret i64 %ret } @@ -6121,7 +6121,7 @@ define i64 @test358(i64* %ptr, i64 %val) { ; PPC64LE-NEXT: mr 3, 5 ; PPC64LE-NEXT: lwsync ; PPC64LE-NEXT: blr - %ret = atomicrmw xchg i64* %ptr, i64 %val singlethread acq_rel + %ret = atomicrmw xchg i64* %ptr, i64 %val syncscope("singlethread") acq_rel ret i64 %ret } @@ -6137,7 +6137,7 @@ define i64 @test359(i64* %ptr, i64 %val) { ; PPC64LE-NEXT: mr 3, 5 ; PPC64LE-NEXT: lwsync ; PPC64LE-NEXT: blr - %ret = atomicrmw xchg i64* %ptr, i64 %val singlethread seq_cst + %ret = atomicrmw xchg i64* %ptr, i64 %val syncscope("singlethread") seq_cst ret i64 %ret } @@ -6152,7 +6152,7 @@ define i8 @test360(i8* %ptr, i8 %val) { ; PPC64LE-NEXT: # BB#2: ; PPC64LE-NEXT: mr 3, 5 ; PPC64LE-NEXT: blr - %ret = atomicrmw add i8* %ptr, i8 %val singlethread monotonic + %ret = atomicrmw add i8* %ptr, i8 %val syncscope("singlethread") monotonic ret i8 %ret } @@ -6168,7 +6168,7 @@ define i8 @test361(i8* %ptr, i8 %val) { ; PPC64LE-NEXT: # BB#2: ; PPC64LE-NEXT: lwsync ; PPC64LE-NEXT: blr - %ret = atomicrmw add i8* %ptr, i8 %val singlethread acquire + %ret = atomicrmw add i8* %ptr, i8 %val syncscope("singlethread") acquire ret i8 %ret } @@ -6184,7 +6184,7 @@ define i8 @test362(i8* %ptr, i8 %val) { ; PPC64LE-NEXT: # BB#2: ; PPC64LE-NEXT: mr 3, 5 ; PPC64LE-NEXT: blr - %ret = atomicrmw add i8* %ptr, i8 %val singlethread release + %ret = atomicrmw add i8* %ptr, i8 %val syncscope("singlethread") release ret i8 %ret } @@ -6201,7 +6201,7 @@ define i8 @test363(i8* %ptr, i8 %val) { ; PPC64LE-NEXT: mr 3, 5 ; PPC64LE-NEXT: lwsync ; PPC64LE-NEXT: blr - %ret = atomicrmw add i8* %ptr, i8 %val singlethread acq_rel + %ret = atomicrmw add i8* %ptr, i8 %val syncscope("singlethread") acq_rel ret i8 %ret } @@ -6218,7 +6218,7 @@ define i8 @test364(i8* %ptr, i8 %val) { ; PPC64LE-NEXT: mr 3, 5 ; PPC64LE-NEXT: lwsync ; PPC64LE-NEXT: blr - %ret = atomicrmw add i8* %ptr, i8 %val singlethread seq_cst + %ret = atomicrmw add i8* %ptr, i8 %val syncscope("singlethread") seq_cst ret i8 %ret } @@ -6233,7 +6233,7 @@ define i16 @test365(i16* %ptr, i16 %val) { ; PPC64LE-NEXT: # BB#2: ; PPC64LE-NEXT: mr 3, 5 ; PPC64LE-NEXT: blr - %ret = atomicrmw add i16* %ptr, i16 %val singlethread monotonic + %ret = atomicrmw add i16* %ptr, i16 %val syncscope("singlethread") monotonic ret i16 %ret } @@ -6249,7 +6249,7 @@ define i16 @test366(i16* %ptr, i16 %val) { ; PPC64LE-NEXT: # BB#2: ; PPC64LE-NEXT: lwsync ; PPC64LE-NEXT: blr - %ret = atomicrmw add i16* %ptr, i16 %val singlethread acquire + %ret = atomicrmw add i16* %ptr, i16 %val syncscope("singlethread") acquire ret i16 %ret } @@ -6265,7 +6265,7 @@ define i16 @test367(i16* %ptr, i16 %val) { ; PPC64LE-NEXT: # BB#2: ; PPC64LE-NEXT: mr 3, 5 ; PPC64LE-NEXT: blr - %ret = atomicrmw add i16* %ptr, i16 %val singlethread release + %ret = atomicrmw add i16* %ptr, i16 %val syncscope("singlethread") release ret i16 %ret } @@ -6282,7 +6282,7 @@ define i16 @test368(i16* %ptr, i16 %val) { ; PPC64LE-NEXT: mr 3, 5 ; PPC64LE-NEXT: lwsync ; PPC64LE-NEXT: blr - %ret = atomicrmw add i16* %ptr, i16 %val singlethread acq_rel + %ret = atomicrmw add i16* %ptr, i16 %val syncscope("singlethread") acq_rel ret i16 %ret } @@ -6299,7 +6299,7 @@ define i16 @test369(i16* %ptr, i16 %val) { ; PPC64LE-NEXT: mr 3, 5 ; PPC64LE-NEXT: lwsync ; PPC64LE-NEXT: blr - %ret = atomicrmw add i16* %ptr, i16 %val singlethread seq_cst + %ret = atomicrmw add i16* %ptr, i16 %val syncscope("singlethread") seq_cst ret i16 %ret } @@ -6314,7 +6314,7 @@ define i32 @test370(i32* %ptr, i32 %val) { ; PPC64LE-NEXT: # BB#2: ; PPC64LE-NEXT: mr 3, 5 ; PPC64LE-NEXT: blr - %ret = atomicrmw add i32* %ptr, i32 %val singlethread monotonic + %ret = atomicrmw add i32* %ptr, i32 %val syncscope("singlethread") monotonic ret i32 %ret } @@ -6330,7 +6330,7 @@ define i32 @test371(i32* %ptr, i32 %val) { ; PPC64LE-NEXT: # BB#2: ; PPC64LE-NEXT: lwsync ; PPC64LE-NEXT: blr - %ret = atomicrmw add i32* %ptr, i32 %val singlethread acquire + %ret = atomicrmw add i32* %ptr, i32 %val syncscope("singlethread") acquire ret i32 %ret } @@ -6346,7 +6346,7 @@ define i32 @test372(i32* %ptr, i32 %val) { ; PPC64LE-NEXT: # BB#2: ; PPC64LE-NEXT: mr 3, 5 ; PPC64LE-NEXT: blr - %ret = atomicrmw add i32* %ptr, i32 %val singlethread release + %ret = atomicrmw add i32* %ptr, i32 %val syncscope("singlethread") release ret i32 %ret } @@ -6363,7 +6363,7 @@ define i32 @test373(i32* %ptr, i32 %val) { ; PPC64LE-NEXT: mr 3, 5 ; PPC64LE-NEXT: lwsync ; PPC64LE-NEXT: blr - %ret = atomicrmw add i32* %ptr, i32 %val singlethread acq_rel + %ret = atomicrmw add i32* %ptr, i32 %val syncscope("singlethread") acq_rel ret i32 %ret } @@ -6380,7 +6380,7 @@ define i32 @test374(i32* %ptr, i32 %val) { ; PPC64LE-NEXT: mr 3, 5 ; PPC64LE-NEXT: lwsync ; PPC64LE-NEXT: blr - %ret = atomicrmw add i32* %ptr, i32 %val singlethread seq_cst + %ret = atomicrmw add i32* %ptr, i32 %val syncscope("singlethread") seq_cst ret i32 %ret } @@ -6395,7 +6395,7 @@ define i64 @test375(i64* %ptr, i64 %val) { ; PPC64LE-NEXT: # BB#2: ; PPC64LE-NEXT: mr 3, 5 ; PPC64LE-NEXT: blr - %ret = atomicrmw add i64* %ptr, i64 %val singlethread monotonic + %ret = atomicrmw add i64* %ptr, i64 %val syncscope("singlethread") monotonic ret i64 %ret } @@ -6411,7 +6411,7 @@ define i64 @test376(i64* %ptr, i64 %val) { ; PPC64LE-NEXT: # BB#2: ; PPC64LE-NEXT: lwsync ; PPC64LE-NEXT: blr - %ret = atomicrmw add i64* %ptr, i64 %val singlethread acquire + %ret = atomicrmw add i64* %ptr, i64 %val syncscope("singlethread") acquire ret i64 %ret } @@ -6427,7 +6427,7 @@ define i64 @test377(i64* %ptr, i64 %val) { ; PPC64LE-NEXT: # BB#2: ; PPC64LE-NEXT: mr 3, 5 ; PPC64LE-NEXT: blr - %ret = atomicrmw add i64* %ptr, i64 %val singlethread release + %ret = atomicrmw add i64* %ptr, i64 %val syncscope("singlethread") release ret i64 %ret } @@ -6444,7 +6444,7 @@ define i64 @test378(i64* %ptr, i64 %val) { ; PPC64LE-NEXT: mr 3, 5 ; PPC64LE-NEXT: lwsync ; PPC64LE-NEXT: blr - %ret = atomicrmw add i64* %ptr, i64 %val singlethread acq_rel + %ret = atomicrmw add i64* %ptr, i64 %val syncscope("singlethread") acq_rel ret i64 %ret } @@ -6461,7 +6461,7 @@ define i64 @test379(i64* %ptr, i64 %val) { ; PPC64LE-NEXT: mr 3, 5 ; PPC64LE-NEXT: lwsync ; PPC64LE-NEXT: blr - %ret = atomicrmw add i64* %ptr, i64 %val singlethread seq_cst + %ret = atomicrmw add i64* %ptr, i64 %val syncscope("singlethread") seq_cst ret i64 %ret } @@ -6476,7 +6476,7 @@ define i8 @test380(i8* %ptr, i8 %val) { ; PPC64LE-NEXT: # BB#2: ; PPC64LE-NEXT: mr 3, 5 ; PPC64LE-NEXT: blr - %ret = atomicrmw sub i8* %ptr, i8 %val singlethread monotonic + %ret = atomicrmw sub i8* %ptr, i8 %val syncscope("singlethread") monotonic ret i8 %ret } @@ -6492,7 +6492,7 @@ define i8 @test381(i8* %ptr, i8 %val) { ; PPC64LE-NEXT: # BB#2: ; PPC64LE-NEXT: lwsync ; PPC64LE-NEXT: blr - %ret = atomicrmw sub i8* %ptr, i8 %val singlethread acquire + %ret = atomicrmw sub i8* %ptr, i8 %val syncscope("singlethread") acquire ret i8 %ret } @@ -6508,7 +6508,7 @@ define i8 @test382(i8* %ptr, i8 %val) { ; PPC64LE-NEXT: # BB#2: ; PPC64LE-NEXT: mr 3, 5 ; PPC64LE-NEXT: blr - %ret = atomicrmw sub i8* %ptr, i8 %val singlethread release + %ret = atomicrmw sub i8* %ptr, i8 %val syncscope("singlethread") release ret i8 %ret } @@ -6525,7 +6525,7 @@ define i8 @test383(i8* %ptr, i8 %val) { ; PPC64LE-NEXT: mr 3, 5 ; PPC64LE-NEXT: lwsync ; PPC64LE-NEXT: blr - %ret = atomicrmw sub i8* %ptr, i8 %val singlethread acq_rel + %ret = atomicrmw sub i8* %ptr, i8 %val syncscope("singlethread") acq_rel ret i8 %ret } @@ -6542,7 +6542,7 @@ define i8 @test384(i8* %ptr, i8 %val) { ; PPC64LE-NEXT: mr 3, 5 ; PPC64LE-NEXT: lwsync ; PPC64LE-NEXT: blr - %ret = atomicrmw sub i8* %ptr, i8 %val singlethread seq_cst + %ret = atomicrmw sub i8* %ptr, i8 %val syncscope("singlethread") seq_cst ret i8 %ret } @@ -6557,7 +6557,7 @@ define i16 @test385(i16* %ptr, i16 %val) { ; PPC64LE-NEXT: # BB#2: ; PPC64LE-NEXT: mr 3, 5 ; PPC64LE-NEXT: blr - %ret = atomicrmw sub i16* %ptr, i16 %val singlethread monotonic + %ret = atomicrmw sub i16* %ptr, i16 %val syncscope("singlethread") monotonic ret i16 %ret } @@ -6573,7 +6573,7 @@ define i16 @test386(i16* %ptr, i16 %val) { ; PPC64LE-NEXT: # BB#2: ; PPC64LE-NEXT: lwsync ; PPC64LE-NEXT: blr - %ret = atomicrmw sub i16* %ptr, i16 %val singlethread acquire + %ret = atomicrmw sub i16* %ptr, i16 %val syncscope("singlethread") acquire ret i16 %ret } @@ -6589,7 +6589,7 @@ define i16 @test387(i16* %ptr, i16 %val) { ; PPC64LE-NEXT: # BB#2: ; PPC64LE-NEXT: mr 3, 5 ; PPC64LE-NEXT: blr - %ret = atomicrmw sub i16* %ptr, i16 %val singlethread release + %ret = atomicrmw sub i16* %ptr, i16 %val syncscope("singlethread") release ret i16 %ret } @@ -6606,7 +6606,7 @@ define i16 @test388(i16* %ptr, i16 %val) { ; PPC64LE-NEXT: mr 3, 5 ; PPC64LE-NEXT: lwsync ; PPC64LE-NEXT: blr - %ret = atomicrmw sub i16* %ptr, i16 %val singlethread acq_rel + %ret = atomicrmw sub i16* %ptr, i16 %val syncscope("singlethread") acq_rel ret i16 %ret } @@ -6623,7 +6623,7 @@ define i16 @test389(i16* %ptr, i16 %val) { ; PPC64LE-NEXT: mr 3, 5 ; PPC64LE-NEXT: lwsync ; PPC64LE-NEXT: blr - %ret = atomicrmw sub i16* %ptr, i16 %val singlethread seq_cst + %ret = atomicrmw sub i16* %ptr, i16 %val syncscope("singlethread") seq_cst ret i16 %ret } @@ -6638,7 +6638,7 @@ define i32 @test390(i32* %ptr, i32 %val) { ; PPC64LE-NEXT: # BB#2: ; PPC64LE-NEXT: mr 3, 5 ; PPC64LE-NEXT: blr - %ret = atomicrmw sub i32* %ptr, i32 %val singlethread monotonic + %ret = atomicrmw sub i32* %ptr, i32 %val syncscope("singlethread") monotonic ret i32 %ret } @@ -6654,7 +6654,7 @@ define i32 @test391(i32* %ptr, i32 %val) { ; PPC64LE-NEXT: # BB#2: ; PPC64LE-NEXT: lwsync ; PPC64LE-NEXT: blr - %ret = atomicrmw sub i32* %ptr, i32 %val singlethread acquire + %ret = atomicrmw sub i32* %ptr, i32 %val syncscope("singlethread") acquire ret i32 %ret } @@ -6670,7 +6670,7 @@ define i32 @test392(i32* %ptr, i32 %val) { ; PPC64LE-NEXT: # BB#2: ; PPC64LE-NEXT: mr 3, 5 ; PPC64LE-NEXT: blr - %ret = atomicrmw sub i32* %ptr, i32 %val singlethread release + %ret = atomicrmw sub i32* %ptr, i32 %val syncscope("singlethread") release ret i32 %ret } @@ -6687,7 +6687,7 @@ define i32 @test393(i32* %ptr, i32 %val) { ; PPC64LE-NEXT: mr 3, 5 ; PPC64LE-NEXT: lwsync ; PPC64LE-NEXT: blr - %ret = atomicrmw sub i32* %ptr, i32 %val singlethread acq_rel + %ret = atomicrmw sub i32* %ptr, i32 %val syncscope("singlethread") acq_rel ret i32 %ret } @@ -6704,7 +6704,7 @@ define i32 @test394(i32* %ptr, i32 %val) { ; PPC64LE-NEXT: mr 3, 5 ; PPC64LE-NEXT: lwsync ; PPC64LE-NEXT: blr - %ret = atomicrmw sub i32* %ptr, i32 %val singlethread seq_cst + %ret = atomicrmw sub i32* %ptr, i32 %val syncscope("singlethread") seq_cst ret i32 %ret } @@ -6719,7 +6719,7 @@ define i64 @test395(i64* %ptr, i64 %val) { ; PPC64LE-NEXT: # BB#2: ; PPC64LE-NEXT: mr 3, 5 ; PPC64LE-NEXT: blr - %ret = atomicrmw sub i64* %ptr, i64 %val singlethread monotonic + %ret = atomicrmw sub i64* %ptr, i64 %val syncscope("singlethread") monotonic ret i64 %ret } @@ -6735,7 +6735,7 @@ define i64 @test396(i64* %ptr, i64 %val) { ; PPC64LE-NEXT: # BB#2: ; PPC64LE-NEXT: lwsync ; PPC64LE-NEXT: blr - %ret = atomicrmw sub i64* %ptr, i64 %val singlethread acquire + %ret = atomicrmw sub i64* %ptr, i64 %val syncscope("singlethread") acquire ret i64 %ret } @@ -6751,7 +6751,7 @@ define i64 @test397(i64* %ptr, i64 %val) { ; PPC64LE-NEXT: # BB#2: ; PPC64LE-NEXT: mr 3, 5 ; PPC64LE-NEXT: blr - %ret = atomicrmw sub i64* %ptr, i64 %val singlethread release + %ret = atomicrmw sub i64* %ptr, i64 %val syncscope("singlethread") release ret i64 %ret } @@ -6768,7 +6768,7 @@ define i64 @test398(i64* %ptr, i64 %val) { ; PPC64LE-NEXT: mr 3, 5 ; PPC64LE-NEXT: lwsync ; PPC64LE-NEXT: blr - %ret = atomicrmw sub i64* %ptr, i64 %val singlethread acq_rel + %ret = atomicrmw sub i64* %ptr, i64 %val syncscope("singlethread") acq_rel ret i64 %ret } @@ -6785,7 +6785,7 @@ define i64 @test399(i64* %ptr, i64 %val) { ; PPC64LE-NEXT: mr 3, 5 ; PPC64LE-NEXT: lwsync ; PPC64LE-NEXT: blr - %ret = atomicrmw sub i64* %ptr, i64 %val singlethread seq_cst + %ret = atomicrmw sub i64* %ptr, i64 %val syncscope("singlethread") seq_cst ret i64 %ret } @@ -6800,7 +6800,7 @@ define i8 @test400(i8* %ptr, i8 %val) { ; PPC64LE-NEXT: # BB#2: ; PPC64LE-NEXT: mr 3, 5 ; PPC64LE-NEXT: blr - %ret = atomicrmw and i8* %ptr, i8 %val singlethread monotonic + %ret = atomicrmw and i8* %ptr, i8 %val syncscope("singlethread") monotonic ret i8 %ret } @@ -6816,7 +6816,7 @@ define i8 @test401(i8* %ptr, i8 %val) { ; PPC64LE-NEXT: # BB#2: ; PPC64LE-NEXT: lwsync ; PPC64LE-NEXT: blr - %ret = atomicrmw and i8* %ptr, i8 %val singlethread acquire + %ret = atomicrmw and i8* %ptr, i8 %val syncscope("singlethread") acquire ret i8 %ret } @@ -6832,7 +6832,7 @@ define i8 @test402(i8* %ptr, i8 %val) { ; PPC64LE-NEXT: # BB#2: ; PPC64LE-NEXT: mr 3, 5 ; PPC64LE-NEXT: blr - %ret = atomicrmw and i8* %ptr, i8 %val singlethread release + %ret = atomicrmw and i8* %ptr, i8 %val syncscope("singlethread") release ret i8 %ret } @@ -6849,7 +6849,7 @@ define i8 @test403(i8* %ptr, i8 %val) { ; PPC64LE-NEXT: mr 3, 5 ; PPC64LE-NEXT: lwsync ; PPC64LE-NEXT: blr - %ret = atomicrmw and i8* %ptr, i8 %val singlethread acq_rel + %ret = atomicrmw and i8* %ptr, i8 %val syncscope("singlethread") acq_rel ret i8 %ret } @@ -6866,7 +6866,7 @@ define i8 @test404(i8* %ptr, i8 %val) { ; PPC64LE-NEXT: mr 3, 5 ; PPC64LE-NEXT: lwsync ; PPC64LE-NEXT: blr - %ret = atomicrmw and i8* %ptr, i8 %val singlethread seq_cst + %ret = atomicrmw and i8* %ptr, i8 %val syncscope("singlethread") seq_cst ret i8 %ret } @@ -6881,7 +6881,7 @@ define i16 @test405(i16* %ptr, i16 %val) { ; PPC64LE-NEXT: # BB#2: ; PPC64LE-NEXT: mr 3, 5 ; PPC64LE-NEXT: blr - %ret = atomicrmw and i16* %ptr, i16 %val singlethread monotonic + %ret = atomicrmw and i16* %ptr, i16 %val syncscope("singlethread") monotonic ret i16 %ret } @@ -6897,7 +6897,7 @@ define i16 @test406(i16* %ptr, i16 %val) { ; PPC64LE-NEXT: # BB#2: ; PPC64LE-NEXT: lwsync ; PPC64LE-NEXT: blr - %ret = atomicrmw and i16* %ptr, i16 %val singlethread acquire + %ret = atomicrmw and i16* %ptr, i16 %val syncscope("singlethread") acquire ret i16 %ret } @@ -6913,7 +6913,7 @@ define i16 @test407(i16* %ptr, i16 %val) { ; PPC64LE-NEXT: # BB#2: ; PPC64LE-NEXT: mr 3, 5 ; PPC64LE-NEXT: blr - %ret = atomicrmw and i16* %ptr, i16 %val singlethread release + %ret = atomicrmw and i16* %ptr, i16 %val syncscope("singlethread") release ret i16 %ret } @@ -6930,7 +6930,7 @@ define i16 @test408(i16* %ptr, i16 %val) { ; PPC64LE-NEXT: mr 3, 5 ; PPC64LE-NEXT: lwsync ; PPC64LE-NEXT: blr - %ret = atomicrmw and i16* %ptr, i16 %val singlethread acq_rel + %ret = atomicrmw and i16* %ptr, i16 %val syncscope("singlethread") acq_rel ret i16 %ret } @@ -6947,7 +6947,7 @@ define i16 @test409(i16* %ptr, i16 %val) { ; PPC64LE-NEXT: mr 3, 5 ; PPC64LE-NEXT: lwsync ; PPC64LE-NEXT: blr - %ret = atomicrmw and i16* %ptr, i16 %val singlethread seq_cst + %ret = atomicrmw and i16* %ptr, i16 %val syncscope("singlethread") seq_cst ret i16 %ret } @@ -6962,7 +6962,7 @@ define i32 @test410(i32* %ptr, i32 %val) { ; PPC64LE-NEXT: # BB#2: ; PPC64LE-NEXT: mr 3, 5 ; PPC64LE-NEXT: blr - %ret = atomicrmw and i32* %ptr, i32 %val singlethread monotonic + %ret = atomicrmw and i32* %ptr, i32 %val syncscope("singlethread") monotonic ret i32 %ret } @@ -6978,7 +6978,7 @@ define i32 @test411(i32* %ptr, i32 %val) { ; PPC64LE-NEXT: # BB#2: ; PPC64LE-NEXT: lwsync ; PPC64LE-NEXT: blr - %ret = atomicrmw and i32* %ptr, i32 %val singlethread acquire + %ret = atomicrmw and i32* %ptr, i32 %val syncscope("singlethread") acquire ret i32 %ret } @@ -6994,7 +6994,7 @@ define i32 @test412(i32* %ptr, i32 %val) { ; PPC64LE-NEXT: # BB#2: ; PPC64LE-NEXT: mr 3, 5 ; PPC64LE-NEXT: blr - %ret = atomicrmw and i32* %ptr, i32 %val singlethread release + %ret = atomicrmw and i32* %ptr, i32 %val syncscope("singlethread") release ret i32 %ret } @@ -7011,7 +7011,7 @@ define i32 @test413(i32* %ptr, i32 %val) { ; PPC64LE-NEXT: mr 3, 5 ; PPC64LE-NEXT: lwsync ; PPC64LE-NEXT: blr - %ret = atomicrmw and i32* %ptr, i32 %val singlethread acq_rel + %ret = atomicrmw and i32* %ptr, i32 %val syncscope("singlethread") acq_rel ret i32 %ret } @@ -7028,7 +7028,7 @@ define i32 @test414(i32* %ptr, i32 %val) { ; PPC64LE-NEXT: mr 3, 5 ; PPC64LE-NEXT: lwsync ; PPC64LE-NEXT: blr - %ret = atomicrmw and i32* %ptr, i32 %val singlethread seq_cst + %ret = atomicrmw and i32* %ptr, i32 %val syncscope("singlethread") seq_cst ret i32 %ret } @@ -7043,7 +7043,7 @@ define i64 @test415(i64* %ptr, i64 %val) { ; PPC64LE-NEXT: # BB#2: ; PPC64LE-NEXT: mr 3, 5 ; PPC64LE-NEXT: blr - %ret = atomicrmw and i64* %ptr, i64 %val singlethread monotonic + %ret = atomicrmw and i64* %ptr, i64 %val syncscope("singlethread") monotonic ret i64 %ret } @@ -7059,7 +7059,7 @@ define i64 @test416(i64* %ptr, i64 %val) { ; PPC64LE-NEXT: # BB#2: ; PPC64LE-NEXT: lwsync ; PPC64LE-NEXT: blr - %ret = atomicrmw and i64* %ptr, i64 %val singlethread acquire + %ret = atomicrmw and i64* %ptr, i64 %val syncscope("singlethread") acquire ret i64 %ret } @@ -7075,7 +7075,7 @@ define i64 @test417(i64* %ptr, i64 %val) { ; PPC64LE-NEXT: # BB#2: ; PPC64LE-NEXT: mr 3, 5 ; PPC64LE-NEXT: blr - %ret = atomicrmw and i64* %ptr, i64 %val singlethread release + %ret = atomicrmw and i64* %ptr, i64 %val syncscope("singlethread") release ret i64 %ret } @@ -7092,7 +7092,7 @@ define i64 @test418(i64* %ptr, i64 %val) { ; PPC64LE-NEXT: mr 3, 5 ; PPC64LE-NEXT: lwsync ; PPC64LE-NEXT: blr - %ret = atomicrmw and i64* %ptr, i64 %val singlethread acq_rel + %ret = atomicrmw and i64* %ptr, i64 %val syncscope("singlethread") acq_rel ret i64 %ret } @@ -7109,7 +7109,7 @@ define i64 @test419(i64* %ptr, i64 %val) { ; PPC64LE-NEXT: mr 3, 5 ; PPC64LE-NEXT: lwsync ; PPC64LE-NEXT: blr - %ret = atomicrmw and i64* %ptr, i64 %val singlethread seq_cst + %ret = atomicrmw and i64* %ptr, i64 %val syncscope("singlethread") seq_cst ret i64 %ret } @@ -7124,7 +7124,7 @@ define i8 @test420(i8* %ptr, i8 %val) { ; PPC64LE-NEXT: # BB#2: ; PPC64LE-NEXT: mr 3, 5 ; PPC64LE-NEXT: blr - %ret = atomicrmw nand i8* %ptr, i8 %val singlethread monotonic + %ret = atomicrmw nand i8* %ptr, i8 %val syncscope("singlethread") monotonic ret i8 %ret } @@ -7140,7 +7140,7 @@ define i8 @test421(i8* %ptr, i8 %val) { ; PPC64LE-NEXT: # BB#2: ; PPC64LE-NEXT: lwsync ; PPC64LE-NEXT: blr - %ret = atomicrmw nand i8* %ptr, i8 %val singlethread acquire + %ret = atomicrmw nand i8* %ptr, i8 %val syncscope("singlethread") acquire ret i8 %ret } @@ -7156,7 +7156,7 @@ define i8 @test422(i8* %ptr, i8 %val) { ; PPC64LE-NEXT: # BB#2: ; PPC64LE-NEXT: mr 3, 5 ; PPC64LE-NEXT: blr - %ret = atomicrmw nand i8* %ptr, i8 %val singlethread release + %ret = atomicrmw nand i8* %ptr, i8 %val syncscope("singlethread") release ret i8 %ret } @@ -7173,7 +7173,7 @@ define i8 @test423(i8* %ptr, i8 %val) { ; PPC64LE-NEXT: mr 3, 5 ; PPC64LE-NEXT: lwsync ; PPC64LE-NEXT: blr - %ret = atomicrmw nand i8* %ptr, i8 %val singlethread acq_rel + %ret = atomicrmw nand i8* %ptr, i8 %val syncscope("singlethread") acq_rel ret i8 %ret } @@ -7190,7 +7190,7 @@ define i8 @test424(i8* %ptr, i8 %val) { ; PPC64LE-NEXT: mr 3, 5 ; PPC64LE-NEXT: lwsync ; PPC64LE-NEXT: blr - %ret = atomicrmw nand i8* %ptr, i8 %val singlethread seq_cst + %ret = atomicrmw nand i8* %ptr, i8 %val syncscope("singlethread") seq_cst ret i8 %ret } @@ -7205,7 +7205,7 @@ define i16 @test425(i16* %ptr, i16 %val) { ; PPC64LE-NEXT: # BB#2: ; PPC64LE-NEXT: mr 3, 5 ; PPC64LE-NEXT: blr - %ret = atomicrmw nand i16* %ptr, i16 %val singlethread monotonic + %ret = atomicrmw nand i16* %ptr, i16 %val syncscope("singlethread") monotonic ret i16 %ret } @@ -7221,7 +7221,7 @@ define i16 @test426(i16* %ptr, i16 %val) { ; PPC64LE-NEXT: # BB#2: ; PPC64LE-NEXT: lwsync ; PPC64LE-NEXT: blr - %ret = atomicrmw nand i16* %ptr, i16 %val singlethread acquire + %ret = atomicrmw nand i16* %ptr, i16 %val syncscope("singlethread") acquire ret i16 %ret } @@ -7237,7 +7237,7 @@ define i16 @test427(i16* %ptr, i16 %val) { ; PPC64LE-NEXT: # BB#2: ; PPC64LE-NEXT: mr 3, 5 ; PPC64LE-NEXT: blr - %ret = atomicrmw nand i16* %ptr, i16 %val singlethread release + %ret = atomicrmw nand i16* %ptr, i16 %val syncscope("singlethread") release ret i16 %ret } @@ -7254,7 +7254,7 @@ define i16 @test428(i16* %ptr, i16 %val) { ; PPC64LE-NEXT: mr 3, 5 ; PPC64LE-NEXT: lwsync ; PPC64LE-NEXT: blr - %ret = atomicrmw nand i16* %ptr, i16 %val singlethread acq_rel + %ret = atomicrmw nand i16* %ptr, i16 %val syncscope("singlethread") acq_rel ret i16 %ret } @@ -7271,7 +7271,7 @@ define i16 @test429(i16* %ptr, i16 %val) { ; PPC64LE-NEXT: mr 3, 5 ; PPC64LE-NEXT: lwsync ; PPC64LE-NEXT: blr - %ret = atomicrmw nand i16* %ptr, i16 %val singlethread seq_cst + %ret = atomicrmw nand i16* %ptr, i16 %val syncscope("singlethread") seq_cst ret i16 %ret } @@ -7286,7 +7286,7 @@ define i32 @test430(i32* %ptr, i32 %val) { ; PPC64LE-NEXT: # BB#2: ; PPC64LE-NEXT: mr 3, 5 ; PPC64LE-NEXT: blr - %ret = atomicrmw nand i32* %ptr, i32 %val singlethread monotonic + %ret = atomicrmw nand i32* %ptr, i32 %val syncscope("singlethread") monotonic ret i32 %ret } @@ -7302,7 +7302,7 @@ define i32 @test431(i32* %ptr, i32 %val) { ; PPC64LE-NEXT: # BB#2: ; PPC64LE-NEXT: lwsync ; PPC64LE-NEXT: blr - %ret = atomicrmw nand i32* %ptr, i32 %val singlethread acquire + %ret = atomicrmw nand i32* %ptr, i32 %val syncscope("singlethread") acquire ret i32 %ret } @@ -7318,7 +7318,7 @@ define i32 @test432(i32* %ptr, i32 %val) { ; PPC64LE-NEXT: # BB#2: ; PPC64LE-NEXT: mr 3, 5 ; PPC64LE-NEXT: blr - %ret = atomicrmw nand i32* %ptr, i32 %val singlethread release + %ret = atomicrmw nand i32* %ptr, i32 %val syncscope("singlethread") release ret i32 %ret } @@ -7335,7 +7335,7 @@ define i32 @test433(i32* %ptr, i32 %val) { ; PPC64LE-NEXT: mr 3, 5 ; PPC64LE-NEXT: lwsync ; PPC64LE-NEXT: blr - %ret = atomicrmw nand i32* %ptr, i32 %val singlethread acq_rel + %ret = atomicrmw nand i32* %ptr, i32 %val syncscope("singlethread") acq_rel ret i32 %ret } @@ -7352,7 +7352,7 @@ define i32 @test434(i32* %ptr, i32 %val) { ; PPC64LE-NEXT: mr 3, 5 ; PPC64LE-NEXT: lwsync ; PPC64LE-NEXT: blr - %ret = atomicrmw nand i32* %ptr, i32 %val singlethread seq_cst + %ret = atomicrmw nand i32* %ptr, i32 %val syncscope("singlethread") seq_cst ret i32 %ret } @@ -7367,7 +7367,7 @@ define i64 @test435(i64* %ptr, i64 %val) { ; PPC64LE-NEXT: # BB#2: ; PPC64LE-NEXT: mr 3, 5 ; PPC64LE-NEXT: blr - %ret = atomicrmw nand i64* %ptr, i64 %val singlethread monotonic + %ret = atomicrmw nand i64* %ptr, i64 %val syncscope("singlethread") monotonic ret i64 %ret } @@ -7383,7 +7383,7 @@ define i64 @test436(i64* %ptr, i64 %val) { ; PPC64LE-NEXT: # BB#2: ; PPC64LE-NEXT: lwsync ; PPC64LE-NEXT: blr - %ret = atomicrmw nand i64* %ptr, i64 %val singlethread acquire + %ret = atomicrmw nand i64* %ptr, i64 %val syncscope("singlethread") acquire ret i64 %ret } @@ -7399,7 +7399,7 @@ define i64 @test437(i64* %ptr, i64 %val) { ; PPC64LE-NEXT: # BB#2: ; PPC64LE-NEXT: mr 3, 5 ; PPC64LE-NEXT: blr - %ret = atomicrmw nand i64* %ptr, i64 %val singlethread release + %ret = atomicrmw nand i64* %ptr, i64 %val syncscope("singlethread") release ret i64 %ret } @@ -7416,7 +7416,7 @@ define i64 @test438(i64* %ptr, i64 %val) { ; PPC64LE-NEXT: mr 3, 5 ; PPC64LE-NEXT: lwsync ; PPC64LE-NEXT: blr - %ret = atomicrmw nand i64* %ptr, i64 %val singlethread acq_rel + %ret = atomicrmw nand i64* %ptr, i64 %val syncscope("singlethread") acq_rel ret i64 %ret } @@ -7433,7 +7433,7 @@ define i64 @test439(i64* %ptr, i64 %val) { ; PPC64LE-NEXT: mr 3, 5 ; PPC64LE-NEXT: lwsync ; PPC64LE-NEXT: blr - %ret = atomicrmw nand i64* %ptr, i64 %val singlethread seq_cst + %ret = atomicrmw nand i64* %ptr, i64 %val syncscope("singlethread") seq_cst ret i64 %ret } @@ -7448,7 +7448,7 @@ define i8 @test440(i8* %ptr, i8 %val) { ; PPC64LE-NEXT: # BB#2: ; PPC64LE-NEXT: mr 3, 5 ; PPC64LE-NEXT: blr - %ret = atomicrmw or i8* %ptr, i8 %val singlethread monotonic + %ret = atomicrmw or i8* %ptr, i8 %val syncscope("singlethread") monotonic ret i8 %ret } @@ -7464,7 +7464,7 @@ define i8 @test441(i8* %ptr, i8 %val) { ; PPC64LE-NEXT: # BB#2: ; PPC64LE-NEXT: lwsync ; PPC64LE-NEXT: blr - %ret = atomicrmw or i8* %ptr, i8 %val singlethread acquire + %ret = atomicrmw or i8* %ptr, i8 %val syncscope("singlethread") acquire ret i8 %ret } @@ -7480,7 +7480,7 @@ define i8 @test442(i8* %ptr, i8 %val) { ; PPC64LE-NEXT: # BB#2: ; PPC64LE-NEXT: mr 3, 5 ; PPC64LE-NEXT: blr - %ret = atomicrmw or i8* %ptr, i8 %val singlethread release + %ret = atomicrmw or i8* %ptr, i8 %val syncscope("singlethread") release ret i8 %ret } @@ -7497,7 +7497,7 @@ define i8 @test443(i8* %ptr, i8 %val) { ; PPC64LE-NEXT: mr 3, 5 ; PPC64LE-NEXT: lwsync ; PPC64LE-NEXT: blr - %ret = atomicrmw or i8* %ptr, i8 %val singlethread acq_rel + %ret = atomicrmw or i8* %ptr, i8 %val syncscope("singlethread") acq_rel ret i8 %ret } @@ -7514,7 +7514,7 @@ define i8 @test444(i8* %ptr, i8 %val) { ; PPC64LE-NEXT: mr 3, 5 ; PPC64LE-NEXT: lwsync ; PPC64LE-NEXT: blr - %ret = atomicrmw or i8* %ptr, i8 %val singlethread seq_cst + %ret = atomicrmw or i8* %ptr, i8 %val syncscope("singlethread") seq_cst ret i8 %ret } @@ -7529,7 +7529,7 @@ define i16 @test445(i16* %ptr, i16 %val) { ; PPC64LE-NEXT: # BB#2: ; PPC64LE-NEXT: mr 3, 5 ; PPC64LE-NEXT: blr - %ret = atomicrmw or i16* %ptr, i16 %val singlethread monotonic + %ret = atomicrmw or i16* %ptr, i16 %val syncscope("singlethread") monotonic ret i16 %ret } @@ -7545,7 +7545,7 @@ define i16 @test446(i16* %ptr, i16 %val) { ; PPC64LE-NEXT: # BB#2: ; PPC64LE-NEXT: lwsync ; PPC64LE-NEXT: blr - %ret = atomicrmw or i16* %ptr, i16 %val singlethread acquire + %ret = atomicrmw or i16* %ptr, i16 %val syncscope("singlethread") acquire ret i16 %ret } @@ -7561,7 +7561,7 @@ define i16 @test447(i16* %ptr, i16 %val) { ; PPC64LE-NEXT: # BB#2: ; PPC64LE-NEXT: mr 3, 5 ; PPC64LE-NEXT: blr - %ret = atomicrmw or i16* %ptr, i16 %val singlethread release + %ret = atomicrmw or i16* %ptr, i16 %val syncscope("singlethread") release ret i16 %ret } @@ -7578,7 +7578,7 @@ define i16 @test448(i16* %ptr, i16 %val) { ; PPC64LE-NEXT: mr 3, 5 ; PPC64LE-NEXT: lwsync ; PPC64LE-NEXT: blr - %ret = atomicrmw or i16* %ptr, i16 %val singlethread acq_rel + %ret = atomicrmw or i16* %ptr, i16 %val syncscope("singlethread") acq_rel ret i16 %ret } @@ -7595,7 +7595,7 @@ define i16 @test449(i16* %ptr, i16 %val) { ; PPC64LE-NEXT: mr 3, 5 ; PPC64LE-NEXT: lwsync ; PPC64LE-NEXT: blr - %ret = atomicrmw or i16* %ptr, i16 %val singlethread seq_cst + %ret = atomicrmw or i16* %ptr, i16 %val syncscope("singlethread") seq_cst ret i16 %ret } @@ -7610,7 +7610,7 @@ define i32 @test450(i32* %ptr, i32 %val) { ; PPC64LE-NEXT: # BB#2: ; PPC64LE-NEXT: mr 3, 5 ; PPC64LE-NEXT: blr - %ret = atomicrmw or i32* %ptr, i32 %val singlethread monotonic + %ret = atomicrmw or i32* %ptr, i32 %val syncscope("singlethread") monotonic ret i32 %ret } @@ -7626,7 +7626,7 @@ define i32 @test451(i32* %ptr, i32 %val) { ; PPC64LE-NEXT: # BB#2: ; PPC64LE-NEXT: lwsync ; PPC64LE-NEXT: blr - %ret = atomicrmw or i32* %ptr, i32 %val singlethread acquire + %ret = atomicrmw or i32* %ptr, i32 %val syncscope("singlethread") acquire ret i32 %ret } @@ -7642,7 +7642,7 @@ define i32 @test452(i32* %ptr, i32 %val) { ; PPC64LE-NEXT: # BB#2: ; PPC64LE-NEXT: mr 3, 5 ; PPC64LE-NEXT: blr - %ret = atomicrmw or i32* %ptr, i32 %val singlethread release + %ret = atomicrmw or i32* %ptr, i32 %val syncscope("singlethread") release ret i32 %ret } @@ -7659,7 +7659,7 @@ define i32 @test453(i32* %ptr, i32 %val) { ; PPC64LE-NEXT: mr 3, 5 ; PPC64LE-NEXT: lwsync ; PPC64LE-NEXT: blr - %ret = atomicrmw or i32* %ptr, i32 %val singlethread acq_rel + %ret = atomicrmw or i32* %ptr, i32 %val syncscope("singlethread") acq_rel ret i32 %ret } @@ -7676,7 +7676,7 @@ define i32 @test454(i32* %ptr, i32 %val) { ; PPC64LE-NEXT: mr 3, 5 ; PPC64LE-NEXT: lwsync ; PPC64LE-NEXT: blr - %ret = atomicrmw or i32* %ptr, i32 %val singlethread seq_cst + %ret = atomicrmw or i32* %ptr, i32 %val syncscope("singlethread") seq_cst ret i32 %ret } @@ -7691,7 +7691,7 @@ define i64 @test455(i64* %ptr, i64 %val) { ; PPC64LE-NEXT: # BB#2: ; PPC64LE-NEXT: mr 3, 5 ; PPC64LE-NEXT: blr - %ret = atomicrmw or i64* %ptr, i64 %val singlethread monotonic + %ret = atomicrmw or i64* %ptr, i64 %val syncscope("singlethread") monotonic ret i64 %ret } @@ -7707,7 +7707,7 @@ define i64 @test456(i64* %ptr, i64 %val) { ; PPC64LE-NEXT: # BB#2: ; PPC64LE-NEXT: lwsync ; PPC64LE-NEXT: blr - %ret = atomicrmw or i64* %ptr, i64 %val singlethread acquire + %ret = atomicrmw or i64* %ptr, i64 %val syncscope("singlethread") acquire ret i64 %ret } @@ -7723,7 +7723,7 @@ define i64 @test457(i64* %ptr, i64 %val) { ; PPC64LE-NEXT: # BB#2: ; PPC64LE-NEXT: mr 3, 5 ; PPC64LE-NEXT: blr - %ret = atomicrmw or i64* %ptr, i64 %val singlethread release + %ret = atomicrmw or i64* %ptr, i64 %val syncscope("singlethread") release ret i64 %ret } @@ -7740,7 +7740,7 @@ define i64 @test458(i64* %ptr, i64 %val) { ; PPC64LE-NEXT: mr 3, 5 ; PPC64LE-NEXT: lwsync ; PPC64LE-NEXT: blr - %ret = atomicrmw or i64* %ptr, i64 %val singlethread acq_rel + %ret = atomicrmw or i64* %ptr, i64 %val syncscope("singlethread") acq_rel ret i64 %ret } @@ -7757,7 +7757,7 @@ define i64 @test459(i64* %ptr, i64 %val) { ; PPC64LE-NEXT: mr 3, 5 ; PPC64LE-NEXT: lwsync ; PPC64LE-NEXT: blr - %ret = atomicrmw or i64* %ptr, i64 %val singlethread seq_cst + %ret = atomicrmw or i64* %ptr, i64 %val syncscope("singlethread") seq_cst ret i64 %ret } @@ -7772,7 +7772,7 @@ define i8 @test460(i8* %ptr, i8 %val) { ; PPC64LE-NEXT: # BB#2: ; PPC64LE-NEXT: mr 3, 5 ; PPC64LE-NEXT: blr - %ret = atomicrmw xor i8* %ptr, i8 %val singlethread monotonic + %ret = atomicrmw xor i8* %ptr, i8 %val syncscope("singlethread") monotonic ret i8 %ret } @@ -7788,7 +7788,7 @@ define i8 @test461(i8* %ptr, i8 %val) { ; PPC64LE-NEXT: # BB#2: ; PPC64LE-NEXT: lwsync ; PPC64LE-NEXT: blr - %ret = atomicrmw xor i8* %ptr, i8 %val singlethread acquire + %ret = atomicrmw xor i8* %ptr, i8 %val syncscope("singlethread") acquire ret i8 %ret } @@ -7804,7 +7804,7 @@ define i8 @test462(i8* %ptr, i8 %val) { ; PPC64LE-NEXT: # BB#2: ; PPC64LE-NEXT: mr 3, 5 ; PPC64LE-NEXT: blr - %ret = atomicrmw xor i8* %ptr, i8 %val singlethread release + %ret = atomicrmw xor i8* %ptr, i8 %val syncscope("singlethread") release ret i8 %ret } @@ -7821,7 +7821,7 @@ define i8 @test463(i8* %ptr, i8 %val) { ; PPC64LE-NEXT: mr 3, 5 ; PPC64LE-NEXT: lwsync ; PPC64LE-NEXT: blr - %ret = atomicrmw xor i8* %ptr, i8 %val singlethread acq_rel + %ret = atomicrmw xor i8* %ptr, i8 %val syncscope("singlethread") acq_rel ret i8 %ret } @@ -7838,7 +7838,7 @@ define i8 @test464(i8* %ptr, i8 %val) { ; PPC64LE-NEXT: mr 3, 5 ; PPC64LE-NEXT: lwsync ; PPC64LE-NEXT: blr - %ret = atomicrmw xor i8* %ptr, i8 %val singlethread seq_cst + %ret = atomicrmw xor i8* %ptr, i8 %val syncscope("singlethread") seq_cst ret i8 %ret } @@ -7853,7 +7853,7 @@ define i16 @test465(i16* %ptr, i16 %val) { ; PPC64LE-NEXT: # BB#2: ; PPC64LE-NEXT: mr 3, 5 ; PPC64LE-NEXT: blr - %ret = atomicrmw xor i16* %ptr, i16 %val singlethread monotonic + %ret = atomicrmw xor i16* %ptr, i16 %val syncscope("singlethread") monotonic ret i16 %ret } @@ -7869,7 +7869,7 @@ define i16 @test466(i16* %ptr, i16 %val) { ; PPC64LE-NEXT: # BB#2: ; PPC64LE-NEXT: lwsync ; PPC64LE-NEXT: blr - %ret = atomicrmw xor i16* %ptr, i16 %val singlethread acquire + %ret = atomicrmw xor i16* %ptr, i16 %val syncscope("singlethread") acquire ret i16 %ret } @@ -7885,7 +7885,7 @@ define i16 @test467(i16* %ptr, i16 %val) { ; PPC64LE-NEXT: # BB#2: ; PPC64LE-NEXT: mr 3, 5 ; PPC64LE-NEXT: blr - %ret = atomicrmw xor i16* %ptr, i16 %val singlethread release + %ret = atomicrmw xor i16* %ptr, i16 %val syncscope("singlethread") release ret i16 %ret } @@ -7902,7 +7902,7 @@ define i16 @test468(i16* %ptr, i16 %val) { ; PPC64LE-NEXT: mr 3, 5 ; PPC64LE-NEXT: lwsync ; PPC64LE-NEXT: blr - %ret = atomicrmw xor i16* %ptr, i16 %val singlethread acq_rel + %ret = atomicrmw xor i16* %ptr, i16 %val syncscope("singlethread") acq_rel ret i16 %ret } @@ -7919,7 +7919,7 @@ define i16 @test469(i16* %ptr, i16 %val) { ; PPC64LE-NEXT: mr 3, 5 ; PPC64LE-NEXT: lwsync ; PPC64LE-NEXT: blr - %ret = atomicrmw xor i16* %ptr, i16 %val singlethread seq_cst + %ret = atomicrmw xor i16* %ptr, i16 %val syncscope("singlethread") seq_cst ret i16 %ret } @@ -7934,7 +7934,7 @@ define i32 @test470(i32* %ptr, i32 %val) { ; PPC64LE-NEXT: # BB#2: ; PPC64LE-NEXT: mr 3, 5 ; PPC64LE-NEXT: blr - %ret = atomicrmw xor i32* %ptr, i32 %val singlethread monotonic + %ret = atomicrmw xor i32* %ptr, i32 %val syncscope("singlethread") monotonic ret i32 %ret } @@ -7950,7 +7950,7 @@ define i32 @test471(i32* %ptr, i32 %val) { ; PPC64LE-NEXT: # BB#2: ; PPC64LE-NEXT: lwsync ; PPC64LE-NEXT: blr - %ret = atomicrmw xor i32* %ptr, i32 %val singlethread acquire + %ret = atomicrmw xor i32* %ptr, i32 %val syncscope("singlethread") acquire ret i32 %ret } @@ -7966,7 +7966,7 @@ define i32 @test472(i32* %ptr, i32 %val) { ; PPC64LE-NEXT: # BB#2: ; PPC64LE-NEXT: mr 3, 5 ; PPC64LE-NEXT: blr - %ret = atomicrmw xor i32* %ptr, i32 %val singlethread release + %ret = atomicrmw xor i32* %ptr, i32 %val syncscope("singlethread") release ret i32 %ret } @@ -7983,7 +7983,7 @@ define i32 @test473(i32* %ptr, i32 %val) { ; PPC64LE-NEXT: mr 3, 5 ; PPC64LE-NEXT: lwsync ; PPC64LE-NEXT: blr - %ret = atomicrmw xor i32* %ptr, i32 %val singlethread acq_rel + %ret = atomicrmw xor i32* %ptr, i32 %val syncscope("singlethread") acq_rel ret i32 %ret } @@ -8000,7 +8000,7 @@ define i32 @test474(i32* %ptr, i32 %val) { ; PPC64LE-NEXT: mr 3, 5 ; PPC64LE-NEXT: lwsync ; PPC64LE-NEXT: blr - %ret = atomicrmw xor i32* %ptr, i32 %val singlethread seq_cst + %ret = atomicrmw xor i32* %ptr, i32 %val syncscope("singlethread") seq_cst ret i32 %ret } @@ -8015,7 +8015,7 @@ define i64 @test475(i64* %ptr, i64 %val) { ; PPC64LE-NEXT: # BB#2: ; PPC64LE-NEXT: mr 3, 5 ; PPC64LE-NEXT: blr - %ret = atomicrmw xor i64* %ptr, i64 %val singlethread monotonic + %ret = atomicrmw xor i64* %ptr, i64 %val syncscope("singlethread") monotonic ret i64 %ret } @@ -8031,7 +8031,7 @@ define i64 @test476(i64* %ptr, i64 %val) { ; PPC64LE-NEXT: # BB#2: ; PPC64LE-NEXT: lwsync ; PPC64LE-NEXT: blr - %ret = atomicrmw xor i64* %ptr, i64 %val singlethread acquire + %ret = atomicrmw xor i64* %ptr, i64 %val syncscope("singlethread") acquire ret i64 %ret } @@ -8047,7 +8047,7 @@ define i64 @test477(i64* %ptr, i64 %val) { ; PPC64LE-NEXT: # BB#2: ; PPC64LE-NEXT: mr 3, 5 ; PPC64LE-NEXT: blr - %ret = atomicrmw xor i64* %ptr, i64 %val singlethread release + %ret = atomicrmw xor i64* %ptr, i64 %val syncscope("singlethread") release ret i64 %ret } @@ -8064,7 +8064,7 @@ define i64 @test478(i64* %ptr, i64 %val) { ; PPC64LE-NEXT: mr 3, 5 ; PPC64LE-NEXT: lwsync ; PPC64LE-NEXT: blr - %ret = atomicrmw xor i64* %ptr, i64 %val singlethread acq_rel + %ret = atomicrmw xor i64* %ptr, i64 %val syncscope("singlethread") acq_rel ret i64 %ret } @@ -8081,7 +8081,7 @@ define i64 @test479(i64* %ptr, i64 %val) { ; PPC64LE-NEXT: mr 3, 5 ; PPC64LE-NEXT: lwsync ; PPC64LE-NEXT: blr - %ret = atomicrmw xor i64* %ptr, i64 %val singlethread seq_cst + %ret = atomicrmw xor i64* %ptr, i64 %val syncscope("singlethread") seq_cst ret i64 %ret } @@ -8099,7 +8099,7 @@ define i8 @test480(i8* %ptr, i8 %val) { ; PPC64LE-NEXT: .LBB480_3: ; PPC64LE-NEXT: mr 3, 5 ; PPC64LE-NEXT: blr - %ret = atomicrmw max i8* %ptr, i8 %val singlethread monotonic + %ret = atomicrmw max i8* %ptr, i8 %val syncscope("singlethread") monotonic ret i8 %ret } @@ -8118,7 +8118,7 @@ define i8 @test481(i8* %ptr, i8 %val) { ; PPC64LE-NEXT: .LBB481_3: ; PPC64LE-NEXT: lwsync ; PPC64LE-NEXT: blr - %ret = atomicrmw max i8* %ptr, i8 %val singlethread acquire + %ret = atomicrmw max i8* %ptr, i8 %val syncscope("singlethread") acquire ret i8 %ret } @@ -8137,7 +8137,7 @@ define i8 @test482(i8* %ptr, i8 %val) { ; PPC64LE-NEXT: .LBB482_3: ; PPC64LE-NEXT: mr 3, 5 ; PPC64LE-NEXT: blr - %ret = atomicrmw max i8* %ptr, i8 %val singlethread release + %ret = atomicrmw max i8* %ptr, i8 %val syncscope("singlethread") release ret i8 %ret } @@ -8157,7 +8157,7 @@ define i8 @test483(i8* %ptr, i8 %val) { ; PPC64LE-NEXT: mr 3, 5 ; PPC64LE-NEXT: lwsync ; PPC64LE-NEXT: blr - %ret = atomicrmw max i8* %ptr, i8 %val singlethread acq_rel + %ret = atomicrmw max i8* %ptr, i8 %val syncscope("singlethread") acq_rel ret i8 %ret } @@ -8177,7 +8177,7 @@ define i8 @test484(i8* %ptr, i8 %val) { ; PPC64LE-NEXT: mr 3, 5 ; PPC64LE-NEXT: lwsync ; PPC64LE-NEXT: blr - %ret = atomicrmw max i8* %ptr, i8 %val singlethread seq_cst + %ret = atomicrmw max i8* %ptr, i8 %val syncscope("singlethread") seq_cst ret i8 %ret } @@ -8195,7 +8195,7 @@ define i16 @test485(i16* %ptr, i16 %val) { ; PPC64LE-NEXT: .LBB485_3: ; PPC64LE-NEXT: mr 3, 5 ; PPC64LE-NEXT: blr - %ret = atomicrmw max i16* %ptr, i16 %val singlethread monotonic + %ret = atomicrmw max i16* %ptr, i16 %val syncscope("singlethread") monotonic ret i16 %ret } @@ -8214,7 +8214,7 @@ define i16 @test486(i16* %ptr, i16 %val) { ; PPC64LE-NEXT: .LBB486_3: ; PPC64LE-NEXT: lwsync ; PPC64LE-NEXT: blr - %ret = atomicrmw max i16* %ptr, i16 %val singlethread acquire + %ret = atomicrmw max i16* %ptr, i16 %val syncscope("singlethread") acquire ret i16 %ret } @@ -8233,7 +8233,7 @@ define i16 @test487(i16* %ptr, i16 %val) { ; PPC64LE-NEXT: .LBB487_3: ; PPC64LE-NEXT: mr 3, 5 ; PPC64LE-NEXT: blr - %ret = atomicrmw max i16* %ptr, i16 %val singlethread release + %ret = atomicrmw max i16* %ptr, i16 %val syncscope("singlethread") release ret i16 %ret } @@ -8253,7 +8253,7 @@ define i16 @test488(i16* %ptr, i16 %val) { ; PPC64LE-NEXT: mr 3, 5 ; PPC64LE-NEXT: lwsync ; PPC64LE-NEXT: blr - %ret = atomicrmw max i16* %ptr, i16 %val singlethread acq_rel + %ret = atomicrmw max i16* %ptr, i16 %val syncscope("singlethread") acq_rel ret i16 %ret } @@ -8273,7 +8273,7 @@ define i16 @test489(i16* %ptr, i16 %val) { ; PPC64LE-NEXT: mr 3, 5 ; PPC64LE-NEXT: lwsync ; PPC64LE-NEXT: blr - %ret = atomicrmw max i16* %ptr, i16 %val singlethread seq_cst + %ret = atomicrmw max i16* %ptr, i16 %val syncscope("singlethread") seq_cst ret i16 %ret } @@ -8290,7 +8290,7 @@ define i32 @test490(i32* %ptr, i32 %val) { ; PPC64LE-NEXT: .LBB490_3: ; PPC64LE-NEXT: mr 3, 5 ; PPC64LE-NEXT: blr - %ret = atomicrmw max i32* %ptr, i32 %val singlethread monotonic + %ret = atomicrmw max i32* %ptr, i32 %val syncscope("singlethread") monotonic ret i32 %ret } @@ -8308,7 +8308,7 @@ define i32 @test491(i32* %ptr, i32 %val) { ; PPC64LE-NEXT: .LBB491_3: ; PPC64LE-NEXT: lwsync ; PPC64LE-NEXT: blr - %ret = atomicrmw max i32* %ptr, i32 %val singlethread acquire + %ret = atomicrmw max i32* %ptr, i32 %val syncscope("singlethread") acquire ret i32 %ret } @@ -8326,7 +8326,7 @@ define i32 @test492(i32* %ptr, i32 %val) { ; PPC64LE-NEXT: .LBB492_3: ; PPC64LE-NEXT: mr 3, 5 ; PPC64LE-NEXT: blr - %ret = atomicrmw max i32* %ptr, i32 %val singlethread release + %ret = atomicrmw max i32* %ptr, i32 %val syncscope("singlethread") release ret i32 %ret } @@ -8345,7 +8345,7 @@ define i32 @test493(i32* %ptr, i32 %val) { ; PPC64LE-NEXT: mr 3, 5 ; PPC64LE-NEXT: lwsync ; PPC64LE-NEXT: blr - %ret = atomicrmw max i32* %ptr, i32 %val singlethread acq_rel + %ret = atomicrmw max i32* %ptr, i32 %val syncscope("singlethread") acq_rel ret i32 %ret } @@ -8364,7 +8364,7 @@ define i32 @test494(i32* %ptr, i32 %val) { ; PPC64LE-NEXT: mr 3, 5 ; PPC64LE-NEXT: lwsync ; PPC64LE-NEXT: blr - %ret = atomicrmw max i32* %ptr, i32 %val singlethread seq_cst + %ret = atomicrmw max i32* %ptr, i32 %val syncscope("singlethread") seq_cst ret i32 %ret } @@ -8381,7 +8381,7 @@ define i64 @test495(i64* %ptr, i64 %val) { ; PPC64LE-NEXT: .LBB495_3: ; PPC64LE-NEXT: mr 3, 5 ; PPC64LE-NEXT: blr - %ret = atomicrmw max i64* %ptr, i64 %val singlethread monotonic + %ret = atomicrmw max i64* %ptr, i64 %val syncscope("singlethread") monotonic ret i64 %ret } @@ -8399,7 +8399,7 @@ define i64 @test496(i64* %ptr, i64 %val) { ; PPC64LE-NEXT: .LBB496_3: ; PPC64LE-NEXT: lwsync ; PPC64LE-NEXT: blr - %ret = atomicrmw max i64* %ptr, i64 %val singlethread acquire + %ret = atomicrmw max i64* %ptr, i64 %val syncscope("singlethread") acquire ret i64 %ret } @@ -8417,7 +8417,7 @@ define i64 @test497(i64* %ptr, i64 %val) { ; PPC64LE-NEXT: .LBB497_3: ; PPC64LE-NEXT: mr 3, 5 ; PPC64LE-NEXT: blr - %ret = atomicrmw max i64* %ptr, i64 %val singlethread release + %ret = atomicrmw max i64* %ptr, i64 %val syncscope("singlethread") release ret i64 %ret } @@ -8436,7 +8436,7 @@ define i64 @test498(i64* %ptr, i64 %val) { ; PPC64LE-NEXT: mr 3, 5 ; PPC64LE-NEXT: lwsync ; PPC64LE-NEXT: blr - %ret = atomicrmw max i64* %ptr, i64 %val singlethread acq_rel + %ret = atomicrmw max i64* %ptr, i64 %val syncscope("singlethread") acq_rel ret i64 %ret } @@ -8455,7 +8455,7 @@ define i64 @test499(i64* %ptr, i64 %val) { ; PPC64LE-NEXT: mr 3, 5 ; PPC64LE-NEXT: lwsync ; PPC64LE-NEXT: blr - %ret = atomicrmw max i64* %ptr, i64 %val singlethread seq_cst + %ret = atomicrmw max i64* %ptr, i64 %val syncscope("singlethread") seq_cst ret i64 %ret } @@ -8473,7 +8473,7 @@ define i8 @test500(i8* %ptr, i8 %val) { ; PPC64LE-NEXT: .LBB500_3: ; PPC64LE-NEXT: mr 3, 5 ; PPC64LE-NEXT: blr - %ret = atomicrmw min i8* %ptr, i8 %val singlethread monotonic + %ret = atomicrmw min i8* %ptr, i8 %val syncscope("singlethread") monotonic ret i8 %ret } @@ -8492,7 +8492,7 @@ define i8 @test501(i8* %ptr, i8 %val) { ; PPC64LE-NEXT: .LBB501_3: ; PPC64LE-NEXT: lwsync ; PPC64LE-NEXT: blr - %ret = atomicrmw min i8* %ptr, i8 %val singlethread acquire + %ret = atomicrmw min i8* %ptr, i8 %val syncscope("singlethread") acquire ret i8 %ret } @@ -8511,7 +8511,7 @@ define i8 @test502(i8* %ptr, i8 %val) { ; PPC64LE-NEXT: .LBB502_3: ; PPC64LE-NEXT: mr 3, 5 ; PPC64LE-NEXT: blr - %ret = atomicrmw min i8* %ptr, i8 %val singlethread release + %ret = atomicrmw min i8* %ptr, i8 %val syncscope("singlethread") release ret i8 %ret } @@ -8531,7 +8531,7 @@ define i8 @test503(i8* %ptr, i8 %val) { ; PPC64LE-NEXT: mr 3, 5 ; PPC64LE-NEXT: lwsync ; PPC64LE-NEXT: blr - %ret = atomicrmw min i8* %ptr, i8 %val singlethread acq_rel + %ret = atomicrmw min i8* %ptr, i8 %val syncscope("singlethread") acq_rel ret i8 %ret } @@ -8551,7 +8551,7 @@ define i8 @test504(i8* %ptr, i8 %val) { ; PPC64LE-NEXT: mr 3, 5 ; PPC64LE-NEXT: lwsync ; PPC64LE-NEXT: blr - %ret = atomicrmw min i8* %ptr, i8 %val singlethread seq_cst + %ret = atomicrmw min i8* %ptr, i8 %val syncscope("singlethread") seq_cst ret i8 %ret } @@ -8569,7 +8569,7 @@ define i16 @test505(i16* %ptr, i16 %val) { ; PPC64LE-NEXT: .LBB505_3: ; PPC64LE-NEXT: mr 3, 5 ; PPC64LE-NEXT: blr - %ret = atomicrmw min i16* %ptr, i16 %val singlethread monotonic + %ret = atomicrmw min i16* %ptr, i16 %val syncscope("singlethread") monotonic ret i16 %ret } @@ -8588,7 +8588,7 @@ define i16 @test506(i16* %ptr, i16 %val) { ; PPC64LE-NEXT: .LBB506_3: ; PPC64LE-NEXT: lwsync ; PPC64LE-NEXT: blr - %ret = atomicrmw min i16* %ptr, i16 %val singlethread acquire + %ret = atomicrmw min i16* %ptr, i16 %val syncscope("singlethread") acquire ret i16 %ret } @@ -8607,7 +8607,7 @@ define i16 @test507(i16* %ptr, i16 %val) { ; PPC64LE-NEXT: .LBB507_3: ; PPC64LE-NEXT: mr 3, 5 ; PPC64LE-NEXT: blr - %ret = atomicrmw min i16* %ptr, i16 %val singlethread release + %ret = atomicrmw min i16* %ptr, i16 %val syncscope("singlethread") release ret i16 %ret } @@ -8627,7 +8627,7 @@ define i16 @test508(i16* %ptr, i16 %val) { ; PPC64LE-NEXT: mr 3, 5 ; PPC64LE-NEXT: lwsync ; PPC64LE-NEXT: blr - %ret = atomicrmw min i16* %ptr, i16 %val singlethread acq_rel + %ret = atomicrmw min i16* %ptr, i16 %val syncscope("singlethread") acq_rel ret i16 %ret } @@ -8647,7 +8647,7 @@ define i16 @test509(i16* %ptr, i16 %val) { ; PPC64LE-NEXT: mr 3, 5 ; PPC64LE-NEXT: lwsync ; PPC64LE-NEXT: blr - %ret = atomicrmw min i16* %ptr, i16 %val singlethread seq_cst + %ret = atomicrmw min i16* %ptr, i16 %val syncscope("singlethread") seq_cst ret i16 %ret } @@ -8664,7 +8664,7 @@ define i32 @test510(i32* %ptr, i32 %val) { ; PPC64LE-NEXT: .LBB510_3: ; PPC64LE-NEXT: mr 3, 5 ; PPC64LE-NEXT: blr - %ret = atomicrmw min i32* %ptr, i32 %val singlethread monotonic + %ret = atomicrmw min i32* %ptr, i32 %val syncscope("singlethread") monotonic ret i32 %ret } @@ -8682,7 +8682,7 @@ define i32 @test511(i32* %ptr, i32 %val) { ; PPC64LE-NEXT: .LBB511_3: ; PPC64LE-NEXT: lwsync ; PPC64LE-NEXT: blr - %ret = atomicrmw min i32* %ptr, i32 %val singlethread acquire + %ret = atomicrmw min i32* %ptr, i32 %val syncscope("singlethread") acquire ret i32 %ret } @@ -8700,7 +8700,7 @@ define i32 @test512(i32* %ptr, i32 %val) { ; PPC64LE-NEXT: .LBB512_3: ; PPC64LE-NEXT: mr 3, 5 ; PPC64LE-NEXT: blr - %ret = atomicrmw min i32* %ptr, i32 %val singlethread release + %ret = atomicrmw min i32* %ptr, i32 %val syncscope("singlethread") release ret i32 %ret } @@ -8719,7 +8719,7 @@ define i32 @test513(i32* %ptr, i32 %val) { ; PPC64LE-NEXT: mr 3, 5 ; PPC64LE-NEXT: lwsync ; PPC64LE-NEXT: blr - %ret = atomicrmw min i32* %ptr, i32 %val singlethread acq_rel + %ret = atomicrmw min i32* %ptr, i32 %val syncscope("singlethread") acq_rel ret i32 %ret } @@ -8738,7 +8738,7 @@ define i32 @test514(i32* %ptr, i32 %val) { ; PPC64LE-NEXT: mr 3, 5 ; PPC64LE-NEXT: lwsync ; PPC64LE-NEXT: blr - %ret = atomicrmw min i32* %ptr, i32 %val singlethread seq_cst + %ret = atomicrmw min i32* %ptr, i32 %val syncscope("singlethread") seq_cst ret i32 %ret } @@ -8755,7 +8755,7 @@ define i64 @test515(i64* %ptr, i64 %val) { ; PPC64LE-NEXT: .LBB515_3: ; PPC64LE-NEXT: mr 3, 5 ; PPC64LE-NEXT: blr - %ret = atomicrmw min i64* %ptr, i64 %val singlethread monotonic + %ret = atomicrmw min i64* %ptr, i64 %val syncscope("singlethread") monotonic ret i64 %ret } @@ -8773,7 +8773,7 @@ define i64 @test516(i64* %ptr, i64 %val) { ; PPC64LE-NEXT: .LBB516_3: ; PPC64LE-NEXT: lwsync ; PPC64LE-NEXT: blr - %ret = atomicrmw min i64* %ptr, i64 %val singlethread acquire + %ret = atomicrmw min i64* %ptr, i64 %val syncscope("singlethread") acquire ret i64 %ret } @@ -8791,7 +8791,7 @@ define i64 @test517(i64* %ptr, i64 %val) { ; PPC64LE-NEXT: .LBB517_3: ; PPC64LE-NEXT: mr 3, 5 ; PPC64LE-NEXT: blr - %ret = atomicrmw min i64* %ptr, i64 %val singlethread release + %ret = atomicrmw min i64* %ptr, i64 %val syncscope("singlethread") release ret i64 %ret } @@ -8810,7 +8810,7 @@ define i64 @test518(i64* %ptr, i64 %val) { ; PPC64LE-NEXT: mr 3, 5 ; PPC64LE-NEXT: lwsync ; PPC64LE-NEXT: blr - %ret = atomicrmw min i64* %ptr, i64 %val singlethread acq_rel + %ret = atomicrmw min i64* %ptr, i64 %val syncscope("singlethread") acq_rel ret i64 %ret } @@ -8829,7 +8829,7 @@ define i64 @test519(i64* %ptr, i64 %val) { ; PPC64LE-NEXT: mr 3, 5 ; PPC64LE-NEXT: lwsync ; PPC64LE-NEXT: blr - %ret = atomicrmw min i64* %ptr, i64 %val singlethread seq_cst + %ret = atomicrmw min i64* %ptr, i64 %val syncscope("singlethread") seq_cst ret i64 %ret } @@ -8846,7 +8846,7 @@ define i8 @test520(i8* %ptr, i8 %val) { ; PPC64LE-NEXT: .LBB520_3: ; PPC64LE-NEXT: mr 3, 5 ; PPC64LE-NEXT: blr - %ret = atomicrmw umax i8* %ptr, i8 %val singlethread monotonic + %ret = atomicrmw umax i8* %ptr, i8 %val syncscope("singlethread") monotonic ret i8 %ret } @@ -8864,7 +8864,7 @@ define i8 @test521(i8* %ptr, i8 %val) { ; PPC64LE-NEXT: .LBB521_3: ; PPC64LE-NEXT: lwsync ; PPC64LE-NEXT: blr - %ret = atomicrmw umax i8* %ptr, i8 %val singlethread acquire + %ret = atomicrmw umax i8* %ptr, i8 %val syncscope("singlethread") acquire ret i8 %ret } @@ -8882,7 +8882,7 @@ define i8 @test522(i8* %ptr, i8 %val) { ; PPC64LE-NEXT: .LBB522_3: ; PPC64LE-NEXT: mr 3, 5 ; PPC64LE-NEXT: blr - %ret = atomicrmw umax i8* %ptr, i8 %val singlethread release + %ret = atomicrmw umax i8* %ptr, i8 %val syncscope("singlethread") release ret i8 %ret } @@ -8901,7 +8901,7 @@ define i8 @test523(i8* %ptr, i8 %val) { ; PPC64LE-NEXT: mr 3, 5 ; PPC64LE-NEXT: lwsync ; PPC64LE-NEXT: blr - %ret = atomicrmw umax i8* %ptr, i8 %val singlethread acq_rel + %ret = atomicrmw umax i8* %ptr, i8 %val syncscope("singlethread") acq_rel ret i8 %ret } @@ -8920,7 +8920,7 @@ define i8 @test524(i8* %ptr, i8 %val) { ; PPC64LE-NEXT: mr 3, 5 ; PPC64LE-NEXT: lwsync ; PPC64LE-NEXT: blr - %ret = atomicrmw umax i8* %ptr, i8 %val singlethread seq_cst + %ret = atomicrmw umax i8* %ptr, i8 %val syncscope("singlethread") seq_cst ret i8 %ret } @@ -8937,7 +8937,7 @@ define i16 @test525(i16* %ptr, i16 %val) { ; PPC64LE-NEXT: .LBB525_3: ; PPC64LE-NEXT: mr 3, 5 ; PPC64LE-NEXT: blr - %ret = atomicrmw umax i16* %ptr, i16 %val singlethread monotonic + %ret = atomicrmw umax i16* %ptr, i16 %val syncscope("singlethread") monotonic ret i16 %ret } @@ -8955,7 +8955,7 @@ define i16 @test526(i16* %ptr, i16 %val) { ; PPC64LE-NEXT: .LBB526_3: ; PPC64LE-NEXT: lwsync ; PPC64LE-NEXT: blr - %ret = atomicrmw umax i16* %ptr, i16 %val singlethread acquire + %ret = atomicrmw umax i16* %ptr, i16 %val syncscope("singlethread") acquire ret i16 %ret } @@ -8973,7 +8973,7 @@ define i16 @test527(i16* %ptr, i16 %val) { ; PPC64LE-NEXT: .LBB527_3: ; PPC64LE-NEXT: mr 3, 5 ; PPC64LE-NEXT: blr - %ret = atomicrmw umax i16* %ptr, i16 %val singlethread release + %ret = atomicrmw umax i16* %ptr, i16 %val syncscope("singlethread") release ret i16 %ret } @@ -8992,7 +8992,7 @@ define i16 @test528(i16* %ptr, i16 %val) { ; PPC64LE-NEXT: mr 3, 5 ; PPC64LE-NEXT: lwsync ; PPC64LE-NEXT: blr - %ret = atomicrmw umax i16* %ptr, i16 %val singlethread acq_rel + %ret = atomicrmw umax i16* %ptr, i16 %val syncscope("singlethread") acq_rel ret i16 %ret } @@ -9011,7 +9011,7 @@ define i16 @test529(i16* %ptr, i16 %val) { ; PPC64LE-NEXT: mr 3, 5 ; PPC64LE-NEXT: lwsync ; PPC64LE-NEXT: blr - %ret = atomicrmw umax i16* %ptr, i16 %val singlethread seq_cst + %ret = atomicrmw umax i16* %ptr, i16 %val syncscope("singlethread") seq_cst ret i16 %ret } @@ -9028,7 +9028,7 @@ define i32 @test530(i32* %ptr, i32 %val) { ; PPC64LE-NEXT: .LBB530_3: ; PPC64LE-NEXT: mr 3, 5 ; PPC64LE-NEXT: blr - %ret = atomicrmw umax i32* %ptr, i32 %val singlethread monotonic + %ret = atomicrmw umax i32* %ptr, i32 %val syncscope("singlethread") monotonic ret i32 %ret } @@ -9046,7 +9046,7 @@ define i32 @test531(i32* %ptr, i32 %val) { ; PPC64LE-NEXT: .LBB531_3: ; PPC64LE-NEXT: lwsync ; PPC64LE-NEXT: blr - %ret = atomicrmw umax i32* %ptr, i32 %val singlethread acquire + %ret = atomicrmw umax i32* %ptr, i32 %val syncscope("singlethread") acquire ret i32 %ret } @@ -9064,7 +9064,7 @@ define i32 @test532(i32* %ptr, i32 %val) { ; PPC64LE-NEXT: .LBB532_3: ; PPC64LE-NEXT: mr 3, 5 ; PPC64LE-NEXT: blr - %ret = atomicrmw umax i32* %ptr, i32 %val singlethread release + %ret = atomicrmw umax i32* %ptr, i32 %val syncscope("singlethread") release ret i32 %ret } @@ -9083,7 +9083,7 @@ define i32 @test533(i32* %ptr, i32 %val) { ; PPC64LE-NEXT: mr 3, 5 ; PPC64LE-NEXT: lwsync ; PPC64LE-NEXT: blr - %ret = atomicrmw umax i32* %ptr, i32 %val singlethread acq_rel + %ret = atomicrmw umax i32* %ptr, i32 %val syncscope("singlethread") acq_rel ret i32 %ret } @@ -9102,7 +9102,7 @@ define i32 @test534(i32* %ptr, i32 %val) { ; PPC64LE-NEXT: mr 3, 5 ; PPC64LE-NEXT: lwsync ; PPC64LE-NEXT: blr - %ret = atomicrmw umax i32* %ptr, i32 %val singlethread seq_cst + %ret = atomicrmw umax i32* %ptr, i32 %val syncscope("singlethread") seq_cst ret i32 %ret } @@ -9119,7 +9119,7 @@ define i64 @test535(i64* %ptr, i64 %val) { ; PPC64LE-NEXT: .LBB535_3: ; PPC64LE-NEXT: mr 3, 5 ; PPC64LE-NEXT: blr - %ret = atomicrmw umax i64* %ptr, i64 %val singlethread monotonic + %ret = atomicrmw umax i64* %ptr, i64 %val syncscope("singlethread") monotonic ret i64 %ret } @@ -9137,7 +9137,7 @@ define i64 @test536(i64* %ptr, i64 %val) { ; PPC64LE-NEXT: .LBB536_3: ; PPC64LE-NEXT: lwsync ; PPC64LE-NEXT: blr - %ret = atomicrmw umax i64* %ptr, i64 %val singlethread acquire + %ret = atomicrmw umax i64* %ptr, i64 %val syncscope("singlethread") acquire ret i64 %ret } @@ -9155,7 +9155,7 @@ define i64 @test537(i64* %ptr, i64 %val) { ; PPC64LE-NEXT: .LBB537_3: ; PPC64LE-NEXT: mr 3, 5 ; PPC64LE-NEXT: blr - %ret = atomicrmw umax i64* %ptr, i64 %val singlethread release + %ret = atomicrmw umax i64* %ptr, i64 %val syncscope("singlethread") release ret i64 %ret } @@ -9174,7 +9174,7 @@ define i64 @test538(i64* %ptr, i64 %val) { ; PPC64LE-NEXT: mr 3, 5 ; PPC64LE-NEXT: lwsync ; PPC64LE-NEXT: blr - %ret = atomicrmw umax i64* %ptr, i64 %val singlethread acq_rel + %ret = atomicrmw umax i64* %ptr, i64 %val syncscope("singlethread") acq_rel ret i64 %ret } @@ -9193,7 +9193,7 @@ define i64 @test539(i64* %ptr, i64 %val) { ; PPC64LE-NEXT: mr 3, 5 ; PPC64LE-NEXT: lwsync ; PPC64LE-NEXT: blr - %ret = atomicrmw umax i64* %ptr, i64 %val singlethread seq_cst + %ret = atomicrmw umax i64* %ptr, i64 %val syncscope("singlethread") seq_cst ret i64 %ret } @@ -9210,7 +9210,7 @@ define i8 @test540(i8* %ptr, i8 %val) { ; PPC64LE-NEXT: .LBB540_3: ; PPC64LE-NEXT: mr 3, 5 ; PPC64LE-NEXT: blr - %ret = atomicrmw umin i8* %ptr, i8 %val singlethread monotonic + %ret = atomicrmw umin i8* %ptr, i8 %val syncscope("singlethread") monotonic ret i8 %ret } @@ -9228,7 +9228,7 @@ define i8 @test541(i8* %ptr, i8 %val) { ; PPC64LE-NEXT: .LBB541_3: ; PPC64LE-NEXT: lwsync ; PPC64LE-NEXT: blr - %ret = atomicrmw umin i8* %ptr, i8 %val singlethread acquire + %ret = atomicrmw umin i8* %ptr, i8 %val syncscope("singlethread") acquire ret i8 %ret } @@ -9246,7 +9246,7 @@ define i8 @test542(i8* %ptr, i8 %val) { ; PPC64LE-NEXT: .LBB542_3: ; PPC64LE-NEXT: mr 3, 5 ; PPC64LE-NEXT: blr - %ret = atomicrmw umin i8* %ptr, i8 %val singlethread release + %ret = atomicrmw umin i8* %ptr, i8 %val syncscope("singlethread") release ret i8 %ret } @@ -9265,7 +9265,7 @@ define i8 @test543(i8* %ptr, i8 %val) { ; PPC64LE-NEXT: mr 3, 5 ; PPC64LE-NEXT: lwsync ; PPC64LE-NEXT: blr - %ret = atomicrmw umin i8* %ptr, i8 %val singlethread acq_rel + %ret = atomicrmw umin i8* %ptr, i8 %val syncscope("singlethread") acq_rel ret i8 %ret } @@ -9284,7 +9284,7 @@ define i8 @test544(i8* %ptr, i8 %val) { ; PPC64LE-NEXT: mr 3, 5 ; PPC64LE-NEXT: lwsync ; PPC64LE-NEXT: blr - %ret = atomicrmw umin i8* %ptr, i8 %val singlethread seq_cst + %ret = atomicrmw umin i8* %ptr, i8 %val syncscope("singlethread") seq_cst ret i8 %ret } @@ -9301,7 +9301,7 @@ define i16 @test545(i16* %ptr, i16 %val) { ; PPC64LE-NEXT: .LBB545_3: ; PPC64LE-NEXT: mr 3, 5 ; PPC64LE-NEXT: blr - %ret = atomicrmw umin i16* %ptr, i16 %val singlethread monotonic + %ret = atomicrmw umin i16* %ptr, i16 %val syncscope("singlethread") monotonic ret i16 %ret } @@ -9319,7 +9319,7 @@ define i16 @test546(i16* %ptr, i16 %val) { ; PPC64LE-NEXT: .LBB546_3: ; PPC64LE-NEXT: lwsync ; PPC64LE-NEXT: blr - %ret = atomicrmw umin i16* %ptr, i16 %val singlethread acquire + %ret = atomicrmw umin i16* %ptr, i16 %val syncscope("singlethread") acquire ret i16 %ret } @@ -9337,7 +9337,7 @@ define i16 @test547(i16* %ptr, i16 %val) { ; PPC64LE-NEXT: .LBB547_3: ; PPC64LE-NEXT: mr 3, 5 ; PPC64LE-NEXT: blr - %ret = atomicrmw umin i16* %ptr, i16 %val singlethread release + %ret = atomicrmw umin i16* %ptr, i16 %val syncscope("singlethread") release ret i16 %ret } @@ -9356,7 +9356,7 @@ define i16 @test548(i16* %ptr, i16 %val) { ; PPC64LE-NEXT: mr 3, 5 ; PPC64LE-NEXT: lwsync ; PPC64LE-NEXT: blr - %ret = atomicrmw umin i16* %ptr, i16 %val singlethread acq_rel + %ret = atomicrmw umin i16* %ptr, i16 %val syncscope("singlethread") acq_rel ret i16 %ret } @@ -9375,7 +9375,7 @@ define i16 @test549(i16* %ptr, i16 %val) { ; PPC64LE-NEXT: mr 3, 5 ; PPC64LE-NEXT: lwsync ; PPC64LE-NEXT: blr - %ret = atomicrmw umin i16* %ptr, i16 %val singlethread seq_cst + %ret = atomicrmw umin i16* %ptr, i16 %val syncscope("singlethread") seq_cst ret i16 %ret } @@ -9392,7 +9392,7 @@ define i32 @test550(i32* %ptr, i32 %val) { ; PPC64LE-NEXT: .LBB550_3: ; PPC64LE-NEXT: mr 3, 5 ; PPC64LE-NEXT: blr - %ret = atomicrmw umin i32* %ptr, i32 %val singlethread monotonic + %ret = atomicrmw umin i32* %ptr, i32 %val syncscope("singlethread") monotonic ret i32 %ret } @@ -9410,7 +9410,7 @@ define i32 @test551(i32* %ptr, i32 %val) { ; PPC64LE-NEXT: .LBB551_3: ; PPC64LE-NEXT: lwsync ; PPC64LE-NEXT: blr - %ret = atomicrmw umin i32* %ptr, i32 %val singlethread acquire + %ret = atomicrmw umin i32* %ptr, i32 %val syncscope("singlethread") acquire ret i32 %ret } @@ -9428,7 +9428,7 @@ define i32 @test552(i32* %ptr, i32 %val) { ; PPC64LE-NEXT: .LBB552_3: ; PPC64LE-NEXT: mr 3, 5 ; PPC64LE-NEXT: blr - %ret = atomicrmw umin i32* %ptr, i32 %val singlethread release + %ret = atomicrmw umin i32* %ptr, i32 %val syncscope("singlethread") release ret i32 %ret } @@ -9447,7 +9447,7 @@ define i32 @test553(i32* %ptr, i32 %val) { ; PPC64LE-NEXT: mr 3, 5 ; PPC64LE-NEXT: lwsync ; PPC64LE-NEXT: blr - %ret = atomicrmw umin i32* %ptr, i32 %val singlethread acq_rel + %ret = atomicrmw umin i32* %ptr, i32 %val syncscope("singlethread") acq_rel ret i32 %ret } @@ -9466,7 +9466,7 @@ define i32 @test554(i32* %ptr, i32 %val) { ; PPC64LE-NEXT: mr 3, 5 ; PPC64LE-NEXT: lwsync ; PPC64LE-NEXT: blr - %ret = atomicrmw umin i32* %ptr, i32 %val singlethread seq_cst + %ret = atomicrmw umin i32* %ptr, i32 %val syncscope("singlethread") seq_cst ret i32 %ret } @@ -9483,7 +9483,7 @@ define i64 @test555(i64* %ptr, i64 %val) { ; PPC64LE-NEXT: .LBB555_3: ; PPC64LE-NEXT: mr 3, 5 ; PPC64LE-NEXT: blr - %ret = atomicrmw umin i64* %ptr, i64 %val singlethread monotonic + %ret = atomicrmw umin i64* %ptr, i64 %val syncscope("singlethread") monotonic ret i64 %ret } @@ -9501,7 +9501,7 @@ define i64 @test556(i64* %ptr, i64 %val) { ; PPC64LE-NEXT: .LBB556_3: ; PPC64LE-NEXT: lwsync ; PPC64LE-NEXT: blr - %ret = atomicrmw umin i64* %ptr, i64 %val singlethread acquire + %ret = atomicrmw umin i64* %ptr, i64 %val syncscope("singlethread") acquire ret i64 %ret } @@ -9519,7 +9519,7 @@ define i64 @test557(i64* %ptr, i64 %val) { ; PPC64LE-NEXT: .LBB557_3: ; PPC64LE-NEXT: mr 3, 5 ; PPC64LE-NEXT: blr - %ret = atomicrmw umin i64* %ptr, i64 %val singlethread release + %ret = atomicrmw umin i64* %ptr, i64 %val syncscope("singlethread") release ret i64 %ret } @@ -9538,7 +9538,7 @@ define i64 @test558(i64* %ptr, i64 %val) { ; PPC64LE-NEXT: mr 3, 5 ; PPC64LE-NEXT: lwsync ; PPC64LE-NEXT: blr - %ret = atomicrmw umin i64* %ptr, i64 %val singlethread acq_rel + %ret = atomicrmw umin i64* %ptr, i64 %val syncscope("singlethread") acq_rel ret i64 %ret } @@ -9557,7 +9557,7 @@ define i64 @test559(i64* %ptr, i64 %val) { ; PPC64LE-NEXT: mr 3, 5 ; PPC64LE-NEXT: lwsync ; PPC64LE-NEXT: blr - %ret = atomicrmw umin i64* %ptr, i64 %val singlethread seq_cst + %ret = atomicrmw umin i64* %ptr, i64 %val syncscope("singlethread") seq_cst ret i64 %ret } diff --git a/test/Instrumentation/ThreadSanitizer/atomic.ll b/test/Instrumentation/ThreadSanitizer/atomic.ll index 30c58fea4cb..3d83d9299e6 100644 --- a/test/Instrumentation/ThreadSanitizer/atomic.ll +++ b/test/Instrumentation/ThreadSanitizer/atomic.ll @@ -1959,7 +1959,7 @@ entry: define void @atomic_signal_fence_acquire() nounwind uwtable { entry: - fence singlethread acquire, !dbg !7 + fence syncscope("singlethread") acquire, !dbg !7 ret void, !dbg !7 } ; CHECK-LABEL: atomic_signal_fence_acquire @@ -1975,7 +1975,7 @@ entry: define void @atomic_signal_fence_release() nounwind uwtable { entry: - fence singlethread release, !dbg !7 + fence syncscope("singlethread") release, !dbg !7 ret void, !dbg !7 } ; CHECK-LABEL: atomic_signal_fence_release @@ -1991,7 +1991,7 @@ entry: define void @atomic_signal_fence_acq_rel() nounwind uwtable { entry: - fence singlethread acq_rel, !dbg !7 + fence syncscope("singlethread") acq_rel, !dbg !7 ret void, !dbg !7 } ; CHECK-LABEL: atomic_signal_fence_acq_rel @@ -2007,7 +2007,7 @@ entry: define void @atomic_signal_fence_seq_cst() nounwind uwtable { entry: - fence singlethread seq_cst, !dbg !7 + fence syncscope("singlethread") seq_cst, !dbg !7 ret void, !dbg !7 } ; CHECK-LABEL: atomic_signal_fence_seq_cst diff --git a/test/Linker/Inputs/syncscope-1.ll b/test/Linker/Inputs/syncscope-1.ll new file mode 100644 index 00000000000..90578e931dd --- /dev/null +++ b/test/Linker/Inputs/syncscope-1.ll @@ -0,0 +1,6 @@ +define void @syncscope_1() { + fence syncscope("agent") seq_cst + fence syncscope("workgroup") seq_cst + fence syncscope("wavefront") seq_cst + ret void +} diff --git a/test/Linker/Inputs/syncscope-2.ll b/test/Linker/Inputs/syncscope-2.ll new file mode 100644 index 00000000000..527c5bf93d0 --- /dev/null +++ b/test/Linker/Inputs/syncscope-2.ll @@ -0,0 +1,6 @@ +define void @syncscope_2() { + fence syncscope("image") seq_cst + fence syncscope("agent") seq_cst + fence syncscope("workgroup") seq_cst + ret void +} diff --git a/test/Linker/syncscopes.ll b/test/Linker/syncscopes.ll new file mode 100644 index 00000000000..a572c23cffb --- /dev/null +++ b/test/Linker/syncscopes.ll @@ -0,0 +1,11 @@ +; RUN: llvm-link %S/Inputs/syncscope-1.ll %S/Inputs/syncscope-2.ll -S | FileCheck %s + +; CHECK-LABEL: define void @syncscope_1 +; CHECK: fence syncscope("agent") seq_cst +; CHECK: fence syncscope("workgroup") seq_cst +; CHECK: fence syncscope("wavefront") seq_cst + +; CHECK-LABEL: define void @syncscope_2 +; CHECK: fence syncscope("image") seq_cst +; CHECK: fence syncscope("agent") seq_cst +; CHECK: fence syncscope("workgroup") seq_cst diff --git a/test/Transforms/GVN/PRE/atomic.ll b/test/Transforms/GVN/PRE/atomic.ll index 509acd613e9..3479bc9a0e3 100644 --- a/test/Transforms/GVN/PRE/atomic.ll +++ b/test/Transforms/GVN/PRE/atomic.ll @@ -208,14 +208,14 @@ define void @fence_seq_cst(i32* %P1, i32* %P2) { ret void } -; Can't DSE across a full singlethread fence +; Can't DSE across a full syncscope("singlethread") fence define void @fence_seq_cst_st(i32* %P1, i32* %P2) { ; CHECK-LABEL: @fence_seq_cst_st( ; CHECK: store -; CHECK: fence singlethread seq_cst +; CHECK: fence syncscope("singlethread") seq_cst ; CHECK: store store i32 0, i32* %P1, align 4 - fence singlethread seq_cst + fence syncscope("singlethread") seq_cst store i32 0, i32* %P1, align 4 ret void } diff --git a/test/Transforms/InstCombine/consecutive-fences.ll b/test/Transforms/InstCombine/consecutive-fences.ll index 6f1c4127738..8ecb399f39c 100644 --- a/test/Transforms/InstCombine/consecutive-fences.ll +++ b/test/Transforms/InstCombine/consecutive-fences.ll @@ -4,7 +4,7 @@ ; CHECK-LABEL: define void @tinkywinky ; CHECK-NEXT: fence seq_cst -; CHECK-NEXT: fence singlethread acquire +; CHECK-NEXT: fence syncscope("singlethread") acquire ; CHECK-NEXT: ret void ; CHECK-NEXT: } @@ -12,21 +12,21 @@ define void @tinkywinky() { fence seq_cst fence seq_cst fence seq_cst - fence singlethread acquire - fence singlethread acquire - fence singlethread acquire + fence syncscope("singlethread") acquire + fence syncscope("singlethread") acquire + fence syncscope("singlethread") acquire ret void } ; CHECK-LABEL: define void @dipsy ; CHECK-NEXT: fence seq_cst -; CHECK-NEXT: fence singlethread seq_cst +; CHECK-NEXT: fence syncscope("singlethread") seq_cst ; CHECK-NEXT: ret void ; CHECK-NEXT: } define void @dipsy() { fence seq_cst - fence singlethread seq_cst + fence syncscope("singlethread") seq_cst ret void } diff --git a/test/Transforms/Sink/fence.ll b/test/Transforms/Sink/fence.ll index aa237d8192b..09aa565d88f 100644 --- a/test/Transforms/Sink/fence.ll +++ b/test/Transforms/Sink/fence.ll @@ -5,9 +5,9 @@ target triple = "x86_64-unknown-linux-gnu" define void @test1(i32* ()*) { entry: %1 = call i32* %0() #0 - fence singlethread seq_cst + fence syncscope("singlethread") seq_cst %2 = load i32, i32* %1, align 4 - fence singlethread seq_cst + fence syncscope("singlethread") seq_cst %3 = icmp eq i32 %2, 0 br i1 %3, label %fail, label %pass @@ -20,9 +20,9 @@ pass: ; preds = %fail, %top ; CHECK-LABEL: @test1( ; CHECK: %[[call:.*]] = call i32* %0() -; CHECK: fence singlethread seq_cst +; CHECK: fence syncscope("singlethread") seq_cst ; CHECK: load i32, i32* %[[call]], align 4 -; CHECK: fence singlethread seq_cst +; CHECK: fence syncscope("singlethread") seq_cst attributes #0 = { nounwind readnone } diff --git a/unittests/Analysis/AliasAnalysisTest.cpp b/unittests/Analysis/AliasAnalysisTest.cpp index 84a04257bc2..9a864b77a9d 100644 --- a/unittests/Analysis/AliasAnalysisTest.cpp +++ b/unittests/Analysis/AliasAnalysisTest.cpp @@ -180,10 +180,11 @@ TEST_F(AliasAnalysisTest, getModRefInfo) { auto *VAArg1 = new VAArgInst(Addr, PtrType, "vaarg", BB); auto *CmpXChg1 = new AtomicCmpXchgInst( Addr, ConstantInt::get(IntType, 0), ConstantInt::get(IntType, 1), - AtomicOrdering::Monotonic, AtomicOrdering::Monotonic, CrossThread, BB); + AtomicOrdering::Monotonic, AtomicOrdering::Monotonic, + SyncScope::System, BB); auto *AtomicRMW = new AtomicRMWInst(AtomicRMWInst::Xchg, Addr, ConstantInt::get(IntType, 1), - AtomicOrdering::Monotonic, CrossThread, BB); + AtomicOrdering::Monotonic, SyncScope::System, BB); ReturnInst::Create(C, nullptr, BB); |