Home | History | Annotate | only in /src/common/lib/libc/arch/arm/atomic
History log of /src/common/lib/libc/arch/arm/atomic
RevisionDateAuthorComments
 1.29 07-Jun-2015  matt branches: 1.29.16;
Back out last change; fixed in the correct place.
 1.28 07-Jun-2015  joerg Require ARMv5TE to assemble.
 1.27 17-May-2015  justin Move arm sync_* changes to Makefile.inc
 1.26 14-Oct-2014  martin Provide C++ 2011 <atomic> support functions for hppa and arm.
 1.25 13-Oct-2014  martin Move the and_{16,8}_nv sources into the right (libc only) block.
 1.24 13-Oct-2014  martin Provide __sync_and_and_fetch_2 and __sync_and_and_fetch_1 for pre-ARMv6,
they are needed for the C++ 2011 <atomic> stuff.
 1.23 05-Jul-2014  joerg branches: 1.23.2;
Provide a basic implementation of __atomic_load_* and __atomic_store_*,
used by GCC and LLVM as backing for C11/C++11 atomics, if the hardware
is not known to have corresponding features. Include it on ARM as LLVM
and libc++ hit it when compiled for ARMv4.
 1.22 04-Mar-2014  matt branches: 1.22.2;
Don't export __sync* if _KERNEL || _STANDALONE are defined.
(except if _RUMPKERNEL is defined)
 1.21 04-Mar-2014  matt fix typo.
 1.20 04-Mar-2014  matt Add atomic_sub_64.S
 1.19 27-Feb-2014  matt Add atomic_cas_64 support for ARM EABI on V5TE and V5TEJ cpus.
(strd is atomic).
 1.18 22-Feb-2014  martin Missed one __sync_* op (or gcc4.8 does inline it, while 4.5 does not?)
 1.17 22-Feb-2014  martin Move the __sync_* ops added in the previous change to a libc-only section
 1.16 21-Feb-2014  martin Provide the missing __sync_* ops for earlier arm versions
 1.15 27-Jan-2014  matt Add _atomic_cas_16_up and _atomic_cas_8_up
 1.14 08-Nov-2013  matt Add support for the gcc __sync builtins.
Note that these need earmv6 or later to get the ldrex/strex instructions
 1.13 19-Aug-2013  matt Rework to allow thumb armv7 compilation.
Add atomic_simplelock.c for thumb
 1.12 06-Aug-2013  matt Only use MACHINE_ARCH if !_STANDALONE and CPPFLAGS/CFLAGS/CPUFLAGS don't
contain -mcpu or -march
 1.11 06-Aug-2013  matt Select ldrex/strex if ${MACHINE_ARCH:Mearmv[67]*}
 1.10 11-Sep-2012  matt branches: 1.10.2;
Add 64bit atomic ops for ARMv6+ (using ldrexd/strexd).
 1.9 16-Aug-2012  matt Actually use the assembly version of the atomic function if compiling
for armv6 or armv7 cpus. Use atomic_cas_ptr instead of _lock_cas so
we pick up the assembly version when it's used.
 1.8 04-Jan-2009  pooka branches: 1.8.8; 1.8.12;
allow inclusion of atomic ops in librump
 1.7 29-Sep-2008  ad branches: 1.7.8;
Allow atomic ops to be built as part of libpthread.
 1.6 29-Apr-2008  scw Implement _atomic_cas_up() in assembly code as the compiler cannot be
trusted to generate fully restartable code sequences.

Addresses lib/38482 for ARM and m68000.
 1.5 11-Feb-2008  ad branches: 1.5.4;
Only build atomic ops for libkern/libc.
 1.4 10-Feb-2008  ad Enable the atomic ops in userspace.
 1.3 29-Nov-2007  ad branches: 1.3.4;
Use the CAS-based inc/dec variants, since these CPUs don't have atomic
add in hardware (does arm?).
 1.2 29-Nov-2007  ad Atomic ops for arm.
 1.1 18-Apr-2007  thorpej branches: 1.1.2;
file Makefile.inc was initially added on branch thorpej-atomic.
 1.1.2.2 26-Aug-2007  matt Add armv6 variants of atomic ops which use ldrex/strex.
Add atomic_swap (all arm architectures) which uses swp and swpb.
Add membar_ops which uses armv6 data {sync, memory} barrier.
XXX let thorpej figure out the Makefile magic.
 1.1.2.1 18-Apr-2007  thorpej Atomic ops implementation for ARM. Everything is built on top of
atomic_cas_32(), which is itself a restartable atomic sequence.
 1.3.4.3 23-Mar-2008  matt sync with HEAD
 1.3.4.2 09-Jan-2008  matt sync with HEAD
 1.3.4.1 29-Nov-2007  matt file Makefile.inc was added on branch matt-armv6 on 2008-01-09 01:20:51 +0000
 1.5.4.1 18-May-2008  yamt sync with head.
 1.7.8.2 15-Feb-2014  matt Add 64-bit atomic ops if ARMv6+
 1.7.8.1 19-Dec-2013  matt Enable ldrex/strex based atomic ops on armv6/armv7
 1.8.12.1 27-Nov-2012  matt Pull atomic ops from HEAD.
 1.8.8.2 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.8.8.1 30-Oct-2012  yamt sync with head
 1.10.2.1 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.22.2.1 10-Aug-2014  tls Rebase.
 1.23.2.1 12-Nov-2014  snj Pull up following revision(s) (requested by martin in ticket #218):
common/lib/libc/arch/arm/atomic/Makefile.inc: revision 1.24-1.26
common/lib/libc/arch/hppa/atomic/Makefile.inc: revision 1.13
common/lib/libc/arch/mips/atomic/Makefile.inc: revision 1.13
common/lib/libc/arch/sh3/atomic/Makefile.inc: revision 1.7
common/lib/libc/arch/sparc/atomic/Makefile.inc: revision 1.18
common/lib/libc/arch/vax/atomic/Makefile.inc: revision 1.7
common/lib/libc/atomic/atomic_and_16_nv_cas.c: revision 1.2
common/lib/libc/atomic/atomic_and_8_nv_cas.c: revision 1.2
common/lib/libc/atomic/atomic_c11_compare_exchange_cas_16.c: revision 1.1-1.2
common/lib/libc/atomic/atomic_c11_compare_exchange_cas_32.c: revision 1.1-1.2
common/lib/libc/atomic/atomic_c11_compare_exchange_cas_8.c: revision 1.1-1.2
common/lib/libc/atomic/atomic_cas_by_cas32.c: revision 1.4
common/lib/libc/atomic/atomic_op_namespace.h: revision 1.7
Add __sync_val_compare_and_swap_{1,2} aliases for _atomic_cas_{8,16}
--
Provide __atomic_compare_exchange_N (as needed for the C11 2011 <atomic>
ops) via the corresponding CAS.
--
Hook __atomic_compare_exchange_N into vax libc.
--
Provide __sync_and_and_fetch_2 and __sync_and_and_fetch_1 for pre-ARMv6,
they are needed for the C++ 2011 <atomic> stuff.
--
Add C++ 2011 <atomic> support functions.
--
Move the and_{16,8}_nv sources into the right (libc only) block.
--
Provide <atomic> C++ 2011 support functions for mips and sh3.
--
Provide C++ 2011 <atomic> support functions for hppa and arm.
--
Provide prototypes to fix build with clang.
 1.29.16.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.29.16.1 21-Apr-2020  martin Sync with HEAD
 1.5 28-Jul-2021  skrll Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)

This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)

This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.4 10-Jul-2021  skrll s/ifdef _ARM_ARCH_6/if defined(_ARM_ARCH_6)/ for consistency. NFCI.
 1.3 23-Jun-2014  joerg branches: 1.3.4; 1.3.26; 1.3.28;
Add aliases for the builtins used to implement C11/C++11 atomics.
 1.2 04-Mar-2014  matt branches: 1.2.2; 1.2.4;
Don't export __sync* if _KERNEL || _STANDALONE are defined.
(except if _RUMPKERNEL is defined)
 1.1 08-Nov-2013  matt Add support for the gcc __sync builtins.
Note that these need earmv6 or later to get the ldrex/strex instructions
 1.2.4.2 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.2.4.1 04-Mar-2014  yamt file atomic_add_16.S was added on branch yamt-pagecache on 2014-05-22 11:26:28 +0000
 1.2.2.1 10-Aug-2014  tls Rebase.
 1.3.28.1 11-Aug-2021  martin Pull up following revision(s) (requested by skrll in ticket #1330):

common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.18
common/lib/libc/arch/arm/atomic/atomic_and_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.19
common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_inc_32.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_and_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_add_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_inc_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_cas_up.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_add_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_op_asm.h: revision 1.9
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.8
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_op_asm.h: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_and_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_cas_16.S: revision 1.3
common/lib/libc/arch/arm/atomic/sync_fetch_and_and_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_32.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_xor_32.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_cas_32.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_xor_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_add_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/sync_fetch_and_add_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_dec_64.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_add_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/sync_fetch_and_sub_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_dec_64.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_add_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_xor_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_add_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.14
common/lib/libc/arch/arm/atomic/atomic_or_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_nand_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_or_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_cas_8.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_inc_64.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_inc_64.S: revision 1.11
common/lib/libc/arch/arm/atomic/atomic_cas_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_dec_32.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_or_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_dec_32.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_xor_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_4.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/atomic_or_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_2.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_swap_64.S: revision 1.14
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_1.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_and_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_and_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_and_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.14

Give the thumb atomic ops a chance of working

s/ifdef _ARM_ARCH_6/if defined(_ARM_ARCH_6)/ for consistency. NFCI.

Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.

That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)
This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)
This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.3.26.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.3.26.1 21-Apr-2020  martin Sync with HEAD
 1.3.4.2 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.3.4.1 23-Jun-2014  tls file atomic_add_16.S was added on branch tls-maxphys on 2014-08-19 23:45:12 +0000
 1.10 28-Jul-2021  skrll Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)

This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)

This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.9 10-Jul-2021  skrll s/ifdef _ARM_ARCH_6/if defined(_ARM_ARCH_6)/ for consistency. NFCI.
 1.8 23-Jun-2014  joerg branches: 1.8.24; 1.8.26;
Add aliases for the builtins used to implement C11/C++11 atomics.
 1.7 22-Feb-2014  martin branches: 1.7.2;
Try to hide the C runtime implementation specific __sync_* ops from librump,
to avoid duplicates.
 1.6 08-Nov-2013  matt Add support for the gcc __sync builtins.
Note that these need earmv6 or later to get the ldrex/strex instructions
 1.5 11-Aug-2013  matt Use foo{s} since it reduces the ifdefs for thumb
 1.4 10-Aug-2013  matt Make these under Thumb2
 1.3 31-Aug-2012  matt branches: 1.3.2;
Add dmb/dsb instructions as required by the armv7 arch man.
 1.2 16-Aug-2008  matt branches: 1.2.2; 1.2.4; 1.2.24;
Add assembly versions of atomic ops with ldrex/strex
 1.1 26-Aug-2007  matt branches: 1.1.2;
file atomic_add_32.S was initially added on branch thorpej-atomic.
 1.1.2.2 27-Aug-2007  matt Don't use r4 (since it's caller saved). Use ip (it's available since we
are leaf functions).
 1.1.2.1 26-Aug-2007  matt Add armv6 variants of atomic ops which use ldrex/strex.
Add atomic_swap (all arm architectures) which uses swp and swpb.
Add membar_ops which uses armv6 data {sync, memory} barrier.
XXX let thorpej figure out the Makefile magic.
 1.2.24.1 27-Nov-2012  matt Pull atomic ops from HEAD.
 1.2.4.2 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.2.4.1 30-Oct-2012  yamt sync with head
 1.2.2.2 18-Sep-2008  wrstuden Sync with wrstuden-revivesa-base-2.
 1.2.2.1 16-Aug-2008  wrstuden file atomic_add_32.S was added on branch wrstuden-revivesa on 2008-09-18 04:54:18 +0000
 1.3.2.1 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.7.2.1 10-Aug-2014  tls Rebase.
 1.8.26.1 11-Aug-2021  martin Pull up following revision(s) (requested by skrll in ticket #1330):

common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.18
common/lib/libc/arch/arm/atomic/atomic_and_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.19
common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_inc_32.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_and_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_add_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_inc_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_cas_up.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_add_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_op_asm.h: revision 1.9
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.8
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_op_asm.h: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_and_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_cas_16.S: revision 1.3
common/lib/libc/arch/arm/atomic/sync_fetch_and_and_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_32.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_xor_32.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_cas_32.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_xor_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_add_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/sync_fetch_and_add_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_dec_64.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_add_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/sync_fetch_and_sub_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_dec_64.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_add_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_xor_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_add_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.14
common/lib/libc/arch/arm/atomic/atomic_or_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_nand_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_or_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_cas_8.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_inc_64.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_inc_64.S: revision 1.11
common/lib/libc/arch/arm/atomic/atomic_cas_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_dec_32.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_or_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_dec_32.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_xor_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_4.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/atomic_or_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_2.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_swap_64.S: revision 1.14
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_1.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_and_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_and_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_and_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.14

Give the thumb atomic ops a chance of working

s/ifdef _ARM_ARCH_6/if defined(_ARM_ARCH_6)/ for consistency. NFCI.

Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.

That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)
This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)
This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.8.24.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.8.24.1 21-Apr-2020  martin Sync with HEAD
 1.14 28-Jul-2021  skrll Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)

This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)

This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.13 10-Jul-2021  skrll s/ifdef _ARM_ARCH_6/if defined(_ARM_ARCH_6)/ for consistency. NFCI.
 1.12 28-Jun-2021  skrll Whitespace
 1.11 04-Mar-2014  matt branches: 1.11.26; 1.11.28;
Don't export __sync* if _KERNEL || _STANDALONE are defined.
(except if _RUMPKERNEL is defined)
 1.10 30-Nov-2013  joerg branches: 1.10.2;
Explicitly name the register pairs.
 1.9 08-Nov-2013  matt Add support for the gcc __sync builtins.
Note that these need earmv6 or later to get the ldrex/strex instructions
 1.8 20-Aug-2013  matt Push two registers to keep stack aligned.
 1.7 11-Aug-2013  matt Use foo{s} since it reduces the ifdefs for thumb
 1.6 10-Aug-2013  matt Make these under Thumb2
 1.5 10-Aug-2013  matt Never modify NLO/NHI (r2,r3) only LO/HI (r0,r1) so subsequent loops will work.
 1.4 10-Aug-2013  matt use push/pop
 1.3 13-Sep-2012  matt branches: 1.3.2; 1.3.4;
Correct copyright/fix comments.
 1.2 12-Sep-2012  matt Fix bas code, use ldr
 1.1 11-Sep-2012  matt branches: 1.1.2;
Add 64bit atomic ops for ARMv6+ (using ldrexd/strexd).
 1.1.2.2 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.1.2.1 20-Nov-2012  tls Resync to 2012-11-19 00:00:00 UTC
 1.3.4.2 27-Nov-2012  matt Pull atomic ops from HEAD.
 1.3.4.1 13-Sep-2012  matt file atomic_add_64.S was added on branch matt-nb6-plus on 2012-11-27 23:42:35 +0000
 1.3.2.3 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.3.2.2 30-Oct-2012  yamt sync with head
 1.3.2.1 13-Sep-2012  yamt file atomic_add_64.S was added on branch yamt-pagecache on 2012-10-30 18:46:10 +0000
 1.10.2.2 15-Feb-2014  matt Add 64-bit atomic ops if ARMv6+
 1.10.2.1 30-Nov-2013  matt file atomic_add_64.S was added on branch matt-nb5-mips64 on 2014-02-15 10:27:44 +0000
 1.11.28.2 11-Aug-2021  martin Pull up following revision(s) (requested by skrll in ticket #1330):

common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.18
common/lib/libc/arch/arm/atomic/atomic_and_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.19
common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_inc_32.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_and_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_add_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_inc_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_cas_up.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_add_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_op_asm.h: revision 1.9
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.8
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_op_asm.h: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_and_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_cas_16.S: revision 1.3
common/lib/libc/arch/arm/atomic/sync_fetch_and_and_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_32.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_xor_32.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_cas_32.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_xor_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_add_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/sync_fetch_and_add_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_dec_64.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_add_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/sync_fetch_and_sub_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_dec_64.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_add_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_xor_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_add_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.14
common/lib/libc/arch/arm/atomic/atomic_or_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_nand_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_or_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_cas_8.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_inc_64.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_inc_64.S: revision 1.11
common/lib/libc/arch/arm/atomic/atomic_cas_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_dec_32.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_or_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_dec_32.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_xor_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_4.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/atomic_or_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_2.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_swap_64.S: revision 1.14
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_1.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_and_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_and_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_and_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.14

Give the thumb atomic ops a chance of working

s/ifdef _ARM_ARCH_6/if defined(_ARM_ARCH_6)/ for consistency. NFCI.

Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.

That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)
This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)
This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.11.28.1 06-Jul-2021  martin Pull up following revision(s) (requested by skrll in ticket #1313):

common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/atomic_cas_8.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.3
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.11
common/lib/libc/arch/arm/atomic/atomic_swap_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.12

Whitespace
 1.11.26.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.11.26.1 21-Apr-2020  martin Sync with HEAD
 1.5 28-Jul-2021  skrll Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)

This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)

This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.4 10-Jul-2021  skrll s/ifdef _ARM_ARCH_6/if defined(_ARM_ARCH_6)/ for consistency. NFCI.
 1.3 23-Jun-2014  joerg branches: 1.3.4; 1.3.26; 1.3.28;
Add aliases for the builtins used to implement C11/C++11 atomics.
 1.2 04-Mar-2014  matt branches: 1.2.2; 1.2.4;
Don't export __sync* if _KERNEL || _STANDALONE are defined.
(except if _RUMPKERNEL is defined)
 1.1 08-Nov-2013  matt Add support for the gcc __sync builtins.
Note that these need earmv6 or later to get the ldrex/strex instructions
 1.2.4.2 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.2.4.1 04-Mar-2014  yamt file atomic_add_8.S was added on branch yamt-pagecache on 2014-05-22 11:26:28 +0000
 1.2.2.1 10-Aug-2014  tls Rebase.
 1.3.28.1 11-Aug-2021  martin Pull up following revision(s) (requested by skrll in ticket #1330):

common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.18
common/lib/libc/arch/arm/atomic/atomic_and_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.19
common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_inc_32.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_and_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_add_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_inc_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_cas_up.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_add_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_op_asm.h: revision 1.9
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.8
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_op_asm.h: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_and_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_cas_16.S: revision 1.3
common/lib/libc/arch/arm/atomic/sync_fetch_and_and_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_32.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_xor_32.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_cas_32.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_xor_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_add_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/sync_fetch_and_add_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_dec_64.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_add_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/sync_fetch_and_sub_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_dec_64.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_add_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_xor_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_add_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.14
common/lib/libc/arch/arm/atomic/atomic_or_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_nand_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_or_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_cas_8.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_inc_64.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_inc_64.S: revision 1.11
common/lib/libc/arch/arm/atomic/atomic_cas_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_dec_32.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_or_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_dec_32.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_xor_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_4.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/atomic_or_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_2.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_swap_64.S: revision 1.14
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_1.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_and_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_and_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_and_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.14

Give the thumb atomic ops a chance of working

s/ifdef _ARM_ARCH_6/if defined(_ARM_ARCH_6)/ for consistency. NFCI.

Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.

That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)
This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)
This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.3.26.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.3.26.1 21-Apr-2020  martin Sync with HEAD
 1.3.4.2 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.3.4.1 23-Jun-2014  tls file atomic_add_8.S was added on branch tls-maxphys on 2014-08-19 23:45:12 +0000
 1.5 28-Jul-2021  skrll Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)

This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)

This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.4 10-Jul-2021  skrll s/ifdef _ARM_ARCH_6/if defined(_ARM_ARCH_6)/ for consistency. NFCI.
 1.3 23-Jun-2014  joerg branches: 1.3.4; 1.3.26; 1.3.28;
Add aliases for the builtins used to implement C11/C++11 atomics.
 1.2 04-Mar-2014  matt branches: 1.2.2; 1.2.4;
Don't export __sync* if _KERNEL || _STANDALONE are defined.
(except if _RUMPKERNEL is defined)
 1.1 08-Nov-2013  matt Add support for the gcc __sync builtins.
Note that these need earmv6 or later to get the ldrex/strex instructions
 1.2.4.2 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.2.4.1 04-Mar-2014  yamt file atomic_and_16.S was added on branch yamt-pagecache on 2014-05-22 11:26:28 +0000
 1.2.2.1 10-Aug-2014  tls Rebase.
 1.3.28.1 11-Aug-2021  martin Pull up following revision(s) (requested by skrll in ticket #1330):

common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.18
common/lib/libc/arch/arm/atomic/atomic_and_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.19
common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_inc_32.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_and_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_add_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_inc_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_cas_up.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_add_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_op_asm.h: revision 1.9
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.8
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_op_asm.h: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_and_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_cas_16.S: revision 1.3
common/lib/libc/arch/arm/atomic/sync_fetch_and_and_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_32.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_xor_32.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_cas_32.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_xor_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_add_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/sync_fetch_and_add_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_dec_64.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_add_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/sync_fetch_and_sub_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_dec_64.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_add_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_xor_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_add_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.14
common/lib/libc/arch/arm/atomic/atomic_or_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_nand_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_or_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_cas_8.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_inc_64.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_inc_64.S: revision 1.11
common/lib/libc/arch/arm/atomic/atomic_cas_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_dec_32.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_or_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_dec_32.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_xor_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_4.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/atomic_or_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_2.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_swap_64.S: revision 1.14
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_1.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_and_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_and_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_and_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.14

Give the thumb atomic ops a chance of working

s/ifdef _ARM_ARCH_6/if defined(_ARM_ARCH_6)/ for consistency. NFCI.

Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.

That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)
This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)
This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.3.26.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.3.26.1 21-Apr-2020  martin Sync with HEAD
 1.3.4.2 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.3.4.1 23-Jun-2014  tls file atomic_and_16.S was added on branch tls-maxphys on 2014-08-19 23:45:12 +0000
 1.10 28-Jul-2021  skrll Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)

This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)

This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.9 10-Jul-2021  skrll s/ifdef _ARM_ARCH_6/if defined(_ARM_ARCH_6)/ for consistency. NFCI.
 1.8 23-Jun-2014  joerg branches: 1.8.24; 1.8.26;
Add aliases for the builtins used to implement C11/C++11 atomics.
 1.7 22-Feb-2014  martin branches: 1.7.2;
Try to hide the C runtime implementation specific __sync_* ops from librump,
to avoid duplicates.
 1.6 08-Nov-2013  matt Add support for the gcc __sync builtins.
Note that these need earmv6 or later to get the ldrex/strex instructions
 1.5 11-Aug-2013  matt Use foo{s} since it reduces the ifdefs for thumb
 1.4 10-Aug-2013  matt Make these under Thumb2
 1.3 31-Aug-2012  matt branches: 1.3.2;
Add dmb/dsb instructions as required by the armv7 arch man.
 1.2 16-Aug-2008  matt branches: 1.2.2; 1.2.4; 1.2.24;
Add assembly versions of atomic ops with ldrex/strex
 1.1 26-Aug-2007  matt branches: 1.1.2;
file atomic_and_32.S was initially added on branch thorpej-atomic.
 1.1.2.2 27-Aug-2007  matt Don't use r4 (since it's caller saved). Use ip (it's available since we
are leaf functions).
 1.1.2.1 26-Aug-2007  matt Add armv6 variants of atomic ops which use ldrex/strex.
Add atomic_swap (all arm architectures) which uses swp and swpb.
Add membar_ops which uses armv6 data {sync, memory} barrier.
XXX let thorpej figure out the Makefile magic.
 1.2.24.1 27-Nov-2012  matt Pull atomic ops from HEAD.
 1.2.4.2 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.2.4.1 30-Oct-2012  yamt sync with head
 1.2.2.2 18-Sep-2008  wrstuden Sync with wrstuden-revivesa-base-2.
 1.2.2.1 16-Aug-2008  wrstuden file atomic_and_32.S was added on branch wrstuden-revivesa on 2008-09-18 04:54:18 +0000
 1.3.2.1 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.7.2.1 10-Aug-2014  tls Rebase.
 1.8.26.1 11-Aug-2021  martin Pull up following revision(s) (requested by skrll in ticket #1330):

common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.18
common/lib/libc/arch/arm/atomic/atomic_and_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.19
common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_inc_32.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_and_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_add_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_inc_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_cas_up.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_add_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_op_asm.h: revision 1.9
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.8
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_op_asm.h: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_and_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_cas_16.S: revision 1.3
common/lib/libc/arch/arm/atomic/sync_fetch_and_and_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_32.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_xor_32.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_cas_32.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_xor_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_add_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/sync_fetch_and_add_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_dec_64.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_add_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/sync_fetch_and_sub_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_dec_64.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_add_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_xor_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_add_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.14
common/lib/libc/arch/arm/atomic/atomic_or_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_nand_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_or_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_cas_8.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_inc_64.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_inc_64.S: revision 1.11
common/lib/libc/arch/arm/atomic/atomic_cas_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_dec_32.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_or_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_dec_32.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_xor_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_4.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/atomic_or_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_2.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_swap_64.S: revision 1.14
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_1.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_and_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_and_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_and_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.14

Give the thumb atomic ops a chance of working

s/ifdef _ARM_ARCH_6/if defined(_ARM_ARCH_6)/ for consistency. NFCI.

Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.

That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)
This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)
This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.8.24.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.8.24.1 21-Apr-2020  martin Sync with HEAD
 1.13 28-Jul-2021  skrll Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)

This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)

This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.12 10-Jul-2021  skrll s/ifdef _ARM_ARCH_6/if defined(_ARM_ARCH_6)/ for consistency. NFCI.
 1.11 28-Jun-2021  skrll Whitespace
 1.10 04-Mar-2014  matt branches: 1.10.26; 1.10.28;
Don't export __sync* if _KERNEL || _STANDALONE are defined.
(except if _RUMPKERNEL is defined)
 1.9 30-Nov-2013  joerg branches: 1.9.2;
Use explicit form of register pair operations by specifying both.
 1.8 08-Nov-2013  matt Add support for the gcc __sync builtins.
Note that these need earmv6 or later to get the ldrex/strex instructions
 1.7 20-Aug-2013  matt Push two registers to keep stack aligned.
 1.6 11-Aug-2013  matt Use foo{s} since it reduces the ifdefs for thumb
 1.5 10-Aug-2013  matt Make these under Thumb2
 1.4 10-Aug-2013  matt Never modify NLO/NHI (r2,r3) only LO/HI (r0,r1) so subsequent loops will work.
 1.3 10-Aug-2013  matt Use push/pop
 1.2 13-Sep-2012  matt branches: 1.2.2; 1.2.4;
Correct copyright/fix comments.
 1.1 11-Sep-2012  matt branches: 1.1.2;
Add 64bit atomic ops for ARMv6+ (using ldrexd/strexd).
 1.1.2.2 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.1.2.1 20-Nov-2012  tls Resync to 2012-11-19 00:00:00 UTC
 1.2.4.2 27-Nov-2012  matt Pull atomic ops from HEAD.
 1.2.4.1 13-Sep-2012  matt file atomic_and_64.S was added on branch matt-nb6-plus on 2012-11-27 23:42:35 +0000
 1.2.2.3 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.2.2.2 30-Oct-2012  yamt sync with head
 1.2.2.1 13-Sep-2012  yamt file atomic_and_64.S was added on branch yamt-pagecache on 2012-10-30 18:46:10 +0000
 1.9.2.2 15-Feb-2014  matt Add 64-bit atomic ops if ARMv6+
 1.9.2.1 30-Nov-2013  matt file atomic_and_64.S was added on branch matt-nb5-mips64 on 2014-02-15 10:27:44 +0000
 1.10.28.2 11-Aug-2021  martin Pull up following revision(s) (requested by skrll in ticket #1330):

common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.18
common/lib/libc/arch/arm/atomic/atomic_and_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.19
common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_inc_32.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_and_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_add_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_inc_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_cas_up.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_add_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_op_asm.h: revision 1.9
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.8
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_op_asm.h: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_and_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_cas_16.S: revision 1.3
common/lib/libc/arch/arm/atomic/sync_fetch_and_and_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_32.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_xor_32.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_cas_32.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_xor_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_add_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/sync_fetch_and_add_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_dec_64.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_add_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/sync_fetch_and_sub_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_dec_64.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_add_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_xor_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_add_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.14
common/lib/libc/arch/arm/atomic/atomic_or_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_nand_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_or_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_cas_8.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_inc_64.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_inc_64.S: revision 1.11
common/lib/libc/arch/arm/atomic/atomic_cas_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_dec_32.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_or_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_dec_32.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_xor_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_4.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/atomic_or_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_2.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_swap_64.S: revision 1.14
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_1.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_and_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_and_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_and_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.14

Give the thumb atomic ops a chance of working

s/ifdef _ARM_ARCH_6/if defined(_ARM_ARCH_6)/ for consistency. NFCI.

Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.

That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)
This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)
This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.10.28.1 06-Jul-2021  martin Pull up following revision(s) (requested by skrll in ticket #1313):

common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/atomic_cas_8.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.3
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.11
common/lib/libc/arch/arm/atomic/atomic_swap_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.12

Whitespace
 1.10.26.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.10.26.1 21-Apr-2020  martin Sync with HEAD
 1.5 28-Jul-2021  skrll Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)

This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)

This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.4 10-Jul-2021  skrll s/ifdef _ARM_ARCH_6/if defined(_ARM_ARCH_6)/ for consistency. NFCI.
 1.3 23-Jun-2014  joerg branches: 1.3.4; 1.3.26; 1.3.28;
Add aliases for the builtins used to implement C11/C++11 atomics.
 1.2 04-Mar-2014  matt branches: 1.2.2; 1.2.4;
Don't export __sync* if _KERNEL || _STANDALONE are defined.
(except if _RUMPKERNEL is defined)
 1.1 08-Nov-2013  matt Add support for the gcc __sync builtins.
Note that these need earmv6 or later to get the ldrex/strex instructions
 1.2.4.2 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.2.4.1 04-Mar-2014  yamt file atomic_and_8.S was added on branch yamt-pagecache on 2014-05-22 11:26:28 +0000
 1.2.2.1 10-Aug-2014  tls Rebase.
 1.3.28.1 11-Aug-2021  martin Pull up following revision(s) (requested by skrll in ticket #1330):

common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.18
common/lib/libc/arch/arm/atomic/atomic_and_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.19
common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_inc_32.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_and_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_add_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_inc_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_cas_up.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_add_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_op_asm.h: revision 1.9
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.8
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_op_asm.h: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_and_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_cas_16.S: revision 1.3
common/lib/libc/arch/arm/atomic/sync_fetch_and_and_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_32.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_xor_32.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_cas_32.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_xor_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_add_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/sync_fetch_and_add_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_dec_64.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_add_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/sync_fetch_and_sub_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_dec_64.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_add_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_xor_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_add_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.14
common/lib/libc/arch/arm/atomic/atomic_or_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_nand_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_or_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_cas_8.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_inc_64.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_inc_64.S: revision 1.11
common/lib/libc/arch/arm/atomic/atomic_cas_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_dec_32.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_or_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_dec_32.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_xor_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_4.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/atomic_or_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_2.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_swap_64.S: revision 1.14
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_1.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_and_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_and_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_and_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.14

Give the thumb atomic ops a chance of working

s/ifdef _ARM_ARCH_6/if defined(_ARM_ARCH_6)/ for consistency. NFCI.

Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.

That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)
This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)
This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.3.26.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.3.26.1 21-Apr-2020  martin Sync with HEAD
 1.3.4.2 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.3.4.1 23-Jun-2014  tls file atomic_and_8.S was added on branch tls-maxphys on 2014-08-19 23:45:12 +0000
 1.3 28-Jul-2021  skrll Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)

This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)

This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.2 04-Mar-2014  matt branches: 1.2.4; 1.2.8; 1.2.30; 1.2.32;
Don't export __sync* if _KERNEL || _STANDALONE are defined.
(except if _RUMPKERNEL is defined)
 1.1 08-Nov-2013  matt Add support for the gcc __sync builtins.
Note that these need earmv6 or later to get the ldrex/strex instructions
 1.2.32.1 11-Aug-2021  martin Pull up following revision(s) (requested by skrll in ticket #1330):

common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.18
common/lib/libc/arch/arm/atomic/atomic_and_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.19
common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_inc_32.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_and_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_add_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_inc_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_cas_up.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_add_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_op_asm.h: revision 1.9
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.8
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_op_asm.h: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_and_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_cas_16.S: revision 1.3
common/lib/libc/arch/arm/atomic/sync_fetch_and_and_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_32.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_xor_32.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_cas_32.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_xor_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_add_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/sync_fetch_and_add_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_dec_64.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_add_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/sync_fetch_and_sub_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_dec_64.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_add_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_xor_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_add_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.14
common/lib/libc/arch/arm/atomic/atomic_or_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_nand_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_or_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_cas_8.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_inc_64.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_inc_64.S: revision 1.11
common/lib/libc/arch/arm/atomic/atomic_cas_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_dec_32.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_or_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_dec_32.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_xor_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_4.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/atomic_or_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_2.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_swap_64.S: revision 1.14
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_1.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_and_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_and_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_and_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.14

Give the thumb atomic ops a chance of working

s/ifdef _ARM_ARCH_6/if defined(_ARM_ARCH_6)/ for consistency. NFCI.

Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.

That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)
This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)
This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.2.30.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.2.30.1 21-Apr-2020  martin Sync with HEAD
 1.2.8.2 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.2.8.1 04-Mar-2014  tls file atomic_cas_16.S was added on branch tls-maxphys on 2014-08-19 23:45:12 +0000
 1.2.4.2 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.2.4.1 04-Mar-2014  yamt file atomic_cas_16.S was added on branch yamt-pagecache on 2014-05-22 11:26:28 +0000
 1.8 28-Jul-2021  skrll Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)

This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)

This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.7 04-Mar-2014  matt branches: 1.7.26; 1.7.28;
Don't export __sync* if _KERNEL || _STANDALONE are defined.
(except if _RUMPKERNEL is defined)
 1.6 08-Nov-2013  matt Add support for the gcc __sync builtins.
Note that these need earmv6 or later to get the ldrex/strex instructions
 1.5 10-Aug-2013  matt Make these under Thumb2
 1.4 29-Oct-2012  chs _ARCH_ARM_6 -> _ARM_ARCH_6
 1.3 31-Aug-2012  matt branches: 1.3.2;
Add dmb/dsb instructions as required by the armv7 arch man.
 1.2 16-Aug-2008  matt branches: 1.2.2; 1.2.4; 1.2.12; 1.2.24;
Add assembly versions of atomic ops with ldrex/strex
 1.1 18-Apr-2007  thorpej branches: 1.1.2;
file atomic_cas_32.S was initially added on branch thorpej-atomic.
 1.1.2.5 29-Aug-2007  matt Typo. cmpeq should just be cmp
 1.1.2.4 27-Aug-2007  matt Don't use r4 (since it's caller saved). Use ip (it's available since we
are leaf functions).
 1.1.2.3 26-Aug-2007  matt Add armv6 variants of atomic ops which use ldrex/strex.
Add atomic_swap (all arm architectures) which uses swp and swpb.
Add membar_ops which uses armv6 data {sync, memory} barrier.
XXX let thorpej figure out the Makefile magic.
 1.1.2.2 22-Apr-2007  thorpej Make sure namespace-cleansed aliases are avaialble for all atomic ops.
 1.1.2.1 18-Apr-2007  thorpej Atomic ops implementation for ARM. Everything is built on top of
atomic_cas_32(), which is itself a restartable atomic sequence.
 1.2.24.1 27-Nov-2012  matt Pull atomic ops from HEAD.
 1.2.12.1 19-Dec-2013  matt Enable ldrex/strex based atomic ops on armv6/armv7
 1.2.4.2 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.2.4.1 30-Oct-2012  yamt sync with head
 1.2.2.2 18-Sep-2008  wrstuden Sync with wrstuden-revivesa-base-2.
 1.2.2.1 16-Aug-2008  wrstuden file atomic_cas_32.S was added on branch wrstuden-revivesa on 2008-09-18 04:54:18 +0000
 1.3.2.2 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.3.2.1 20-Nov-2012  tls Resync to 2012-11-19 00:00:00 UTC
 1.7.28.1 11-Aug-2021  martin Pull up following revision(s) (requested by skrll in ticket #1330):

common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.18
common/lib/libc/arch/arm/atomic/atomic_and_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.19
common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_inc_32.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_and_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_add_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_inc_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_cas_up.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_add_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_op_asm.h: revision 1.9
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.8
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_op_asm.h: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_and_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_cas_16.S: revision 1.3
common/lib/libc/arch/arm/atomic/sync_fetch_and_and_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_32.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_xor_32.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_cas_32.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_xor_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_add_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/sync_fetch_and_add_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_dec_64.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_add_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/sync_fetch_and_sub_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_dec_64.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_add_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_xor_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_add_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.14
common/lib/libc/arch/arm/atomic/atomic_or_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_nand_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_or_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_cas_8.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_inc_64.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_inc_64.S: revision 1.11
common/lib/libc/arch/arm/atomic/atomic_cas_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_dec_32.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_or_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_dec_32.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_xor_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_4.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/atomic_or_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_2.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_swap_64.S: revision 1.14
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_1.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_and_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_and_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_and_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.14

Give the thumb atomic ops a chance of working

s/ifdef _ARM_ARCH_6/if defined(_ARM_ARCH_6)/ for consistency. NFCI.

Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.

That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)
This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)
This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.7.26.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.7.26.1 21-Apr-2020  martin Sync with HEAD
 1.12 28-Jul-2021  skrll Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)

This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)

This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.11 18-Feb-2019  martin branches: 1.11.2;
Add some atomic_cas_64_ni aliases
 1.10 05-Mar-2014  matt branches: 1.10.26;
apcs-gnu only passes one register on the stack.
ldrd always loads little endian (low address, low register).
 1.9 04-Mar-2014  matt Don't export __sync* if _KERNEL || _STANDALONE are defined.
(except if _RUMPKERNEL is defined)
 1.8 04-Mar-2014  matt Fix #if/#endif nesting
 1.7 04-Mar-2014  matt Fix non-EABI loading of argument. Deal with endian issues.
Fixes PR/48635
 1.6 04-Mar-2014  matt Fetch value from correct stack location. Push an even number of registers
so ldrd won't fail.
 1.5 30-Nov-2013  joerg branches: 1.5.2;
Use explicit form of register pair operations by specifying both.
 1.4 08-Nov-2013  matt Add support for the gcc __sync builtins.
Note that these need earmv6 or later to get the ldrex/strex instructions
 1.3 10-Aug-2013  matt Make these under Thumb2
 1.2 29-Oct-2012  chs branches: 1.2.2; 1.2.4;
_ARCH_ARM_6 -> _ARM_ARCH_6
 1.1 11-Sep-2012  matt branches: 1.1.2;
Add 64bit atomic ops for ARMv6+ (using ldrexd/strexd).
 1.1.2.2 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.1.2.1 20-Nov-2012  tls Resync to 2012-11-19 00:00:00 UTC
 1.2.4.2 27-Nov-2012  matt Pull atomic ops from HEAD.
 1.2.4.1 29-Oct-2012  matt file atomic_cas_64.S was added on branch matt-nb6-plus on 2012-11-27 23:42:35 +0000
 1.2.2.3 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.2.2.2 30-Oct-2012  yamt sync with head
 1.2.2.1 29-Oct-2012  yamt file atomic_cas_64.S was added on branch yamt-pagecache on 2012-10-30 18:46:10 +0000
 1.5.2.2 15-Feb-2014  matt Add 64-bit atomic ops if ARMv6+
 1.5.2.1 30-Nov-2013  matt file atomic_cas_64.S was added on branch matt-nb5-mips64 on 2014-02-15 10:27:44 +0000
 1.10.26.2 21-Apr-2020  martin Sync with HEAD
 1.10.26.1 10-Jun-2019  christos Sync with HEAD
 1.11.2.1 11-Aug-2021  martin Pull up following revision(s) (requested by skrll in ticket #1330):

common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.18
common/lib/libc/arch/arm/atomic/atomic_and_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.19
common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_inc_32.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_and_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_add_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_inc_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_cas_up.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_add_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_op_asm.h: revision 1.9
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.8
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_op_asm.h: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_and_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_cas_16.S: revision 1.3
common/lib/libc/arch/arm/atomic/sync_fetch_and_and_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_32.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_xor_32.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_cas_32.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_xor_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_add_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/sync_fetch_and_add_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_dec_64.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_add_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/sync_fetch_and_sub_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_dec_64.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_add_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_xor_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_add_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.14
common/lib/libc/arch/arm/atomic/atomic_or_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_nand_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_or_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_cas_8.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_inc_64.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_inc_64.S: revision 1.11
common/lib/libc/arch/arm/atomic/atomic_cas_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_dec_32.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_or_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_dec_32.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_xor_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_4.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/atomic_or_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_2.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_swap_64.S: revision 1.14
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_1.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_and_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_and_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_and_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.14

Give the thumb atomic ops a chance of working

s/ifdef _ARM_ARCH_6/if defined(_ARM_ARCH_6)/ for consistency. NFCI.

Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.

That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)
This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)
This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.9 28-Jul-2021  skrll Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)

This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)

This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.8 29-Jun-2021  skrll Whitespace
 1.7 04-Mar-2014  matt branches: 1.7.26; 1.7.28;
Don't export __sync* if _KERNEL || _STANDALONE are defined.
(except if _RUMPKERNEL is defined)
 1.6 08-Nov-2013  matt Add support for the gcc __sync builtins.
Note that these need earmv6 or later to get the ldrex/strex instructions
 1.5 10-Aug-2013  matt Make these under Thumb2
 1.4 29-Oct-2012  chs _ARCH_ARM_6 -> _ARM_ARCH_6
 1.3 31-Aug-2012  matt branches: 1.3.2;
Add dmb/dsb instructions as required by the armv7 arch man.
 1.2 16-Aug-2012  matt Actually use the assembly version of the atomic function if compiling
for armv6 or armv7 cpus. Use atomic_cas_ptr instead of _lock_cas so
we pick up the assembly version when it's used.
 1.1 18-Nov-2008  matt branches: 1.1.8; 1.1.12;
Add an atomic_cas_8 which uses ldrex/strex.
 1.1.12.1 27-Nov-2012  matt Pull atomic ops from HEAD.
 1.1.8.2 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.1.8.1 30-Oct-2012  yamt sync with head
 1.3.2.2 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.3.2.1 20-Nov-2012  tls Resync to 2012-11-19 00:00:00 UTC
 1.7.28.2 11-Aug-2021  martin Pull up following revision(s) (requested by skrll in ticket #1330):

common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.18
common/lib/libc/arch/arm/atomic/atomic_and_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.19
common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_inc_32.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_and_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_add_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_inc_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_cas_up.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_add_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_op_asm.h: revision 1.9
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.8
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_op_asm.h: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_and_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_cas_16.S: revision 1.3
common/lib/libc/arch/arm/atomic/sync_fetch_and_and_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_32.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_xor_32.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_cas_32.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_xor_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_add_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/sync_fetch_and_add_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_dec_64.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_add_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/sync_fetch_and_sub_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_dec_64.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_add_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_xor_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_add_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.14
common/lib/libc/arch/arm/atomic/atomic_or_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_nand_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_or_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_cas_8.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_inc_64.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_inc_64.S: revision 1.11
common/lib/libc/arch/arm/atomic/atomic_cas_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_dec_32.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_or_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_dec_32.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_xor_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_4.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/atomic_or_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_2.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_swap_64.S: revision 1.14
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_1.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_and_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_and_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_and_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.14

Give the thumb atomic ops a chance of working

s/ifdef _ARM_ARCH_6/if defined(_ARM_ARCH_6)/ for consistency. NFCI.

Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.

That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)
This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)
This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.7.28.1 06-Jul-2021  martin Pull up following revision(s) (requested by skrll in ticket #1313):

common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/atomic_cas_8.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.3
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.11
common/lib/libc/arch/arm/atomic/atomic_swap_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.12

Whitespace
 1.7.26.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.7.26.1 21-Apr-2020  martin Sync with HEAD
 1.8 09-Mar-2020  skrll Give the thumb atomic ops a chance of working
 1.7 04-Mar-2014  matt branches: 1.7.26; 1.7.28;
Load new value from correct stack location in _atomic_cas_64_up
 1.6 27-Feb-2014  matt Add atomic_cas_64 support for ARM EABI on V5TE and V5TEJ cpus.
(strd is atomic).
 1.5 27-Jan-2014  matt Add _atomic_cas_16_up and _atomic_cas_8_up
 1.4 19-Aug-2013  matt Thumbify
 1.3 10-Aug-2013  matt Make these under Thumb2
 1.2 25-May-2008  chs branches: 1.2.2; 1.2.24;
enable profiling of assembly functions.
 1.1 29-Apr-2008  scw branches: 1.1.2; 1.1.4;
Implement _atomic_cas_up() in assembly code as the compiler cannot be
trusted to generate fully restartable code sequences.

Addresses lib/38482 for ARM and m68000.
 1.1.4.3 04-Jun-2008  yamt sync with head
 1.1.4.2 18-May-2008  yamt sync with head.
 1.1.4.1 29-Apr-2008  yamt file atomic_cas_up.S was added on branch yamt-pf42 on 2008-05-18 12:28:44 +0000
 1.1.2.1 23-Jun-2008  wrstuden Sync w/ -current. 34 merge conflicts to follow.
 1.2.24.1 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.2.2.1 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.7.28.1 11-Aug-2021  martin Pull up following revision(s) (requested by skrll in ticket #1330):

common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.18
common/lib/libc/arch/arm/atomic/atomic_and_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.19
common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_inc_32.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_and_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_add_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_inc_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_cas_up.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_add_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_op_asm.h: revision 1.9
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.8
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_op_asm.h: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_and_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_cas_16.S: revision 1.3
common/lib/libc/arch/arm/atomic/sync_fetch_and_and_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_32.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_xor_32.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_cas_32.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_xor_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_add_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/sync_fetch_and_add_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_dec_64.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_add_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/sync_fetch_and_sub_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_dec_64.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_add_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_xor_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_add_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.14
common/lib/libc/arch/arm/atomic/atomic_or_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_nand_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_or_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_cas_8.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_inc_64.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_inc_64.S: revision 1.11
common/lib/libc/arch/arm/atomic/atomic_cas_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_dec_32.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_or_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_dec_32.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_xor_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_4.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/atomic_or_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_2.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_swap_64.S: revision 1.14
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_1.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_and_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_and_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_and_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.14

Give the thumb atomic ops a chance of working

s/ifdef _ARM_ARCH_6/if defined(_ARM_ARCH_6)/ for consistency. NFCI.

Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.

That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)
This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)
This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.7.26.2 21-Apr-2020  martin Sync with HEAD
 1.7.26.1 08-Apr-2020  martin Merge changes from current as of 20200406
 1.7 28-Jul-2021  skrll Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)

This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)

This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.6 10-Jul-2021  skrll s/ifdef _ARM_ARCH_6/if defined(_ARM_ARCH_6)/ for consistency. NFCI.
 1.5 11-Aug-2013  matt branches: 1.5.26; 1.5.28;
Use foo{s} since it reduces the ifdefs for thumb
 1.4 10-Aug-2013  matt Make these under Thumb2
 1.3 31-Aug-2012  matt branches: 1.3.2;
Add dmb/dsb instructions as required by the armv7 arch man.
 1.2 16-Aug-2008  matt branches: 1.2.2; 1.2.4; 1.2.24;
Add assembly versions of atomic ops with ldrex/strex
 1.1 26-Aug-2007  matt branches: 1.1.2;
file atomic_dec_32.S was initially added on branch thorpej-atomic.
 1.1.2.1 26-Aug-2007  matt Add armv6 variants of atomic ops which use ldrex/strex.
Add atomic_swap (all arm architectures) which uses swp and swpb.
Add membar_ops which uses armv6 data {sync, memory} barrier.
XXX let thorpej figure out the Makefile magic.
 1.2.24.1 27-Nov-2012  matt Pull atomic ops from HEAD.
 1.2.4.2 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.2.4.1 30-Oct-2012  yamt sync with head
 1.2.2.2 18-Sep-2008  wrstuden Sync with wrstuden-revivesa-base-2.
 1.2.2.1 16-Aug-2008  wrstuden file atomic_dec_32.S was added on branch wrstuden-revivesa on 2008-09-18 04:54:18 +0000
 1.3.2.1 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.5.28.1 11-Aug-2021  martin Pull up following revision(s) (requested by skrll in ticket #1330):

common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.18
common/lib/libc/arch/arm/atomic/atomic_and_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.19
common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_inc_32.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_and_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_add_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_inc_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_cas_up.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_add_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_op_asm.h: revision 1.9
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.8
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_op_asm.h: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_and_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_cas_16.S: revision 1.3
common/lib/libc/arch/arm/atomic/sync_fetch_and_and_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_32.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_xor_32.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_cas_32.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_xor_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_add_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/sync_fetch_and_add_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_dec_64.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_add_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/sync_fetch_and_sub_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_dec_64.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_add_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_xor_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_add_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.14
common/lib/libc/arch/arm/atomic/atomic_or_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_nand_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_or_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_cas_8.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_inc_64.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_inc_64.S: revision 1.11
common/lib/libc/arch/arm/atomic/atomic_cas_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_dec_32.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_or_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_dec_32.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_xor_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_4.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/atomic_or_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_2.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_swap_64.S: revision 1.14
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_1.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_and_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_and_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_and_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.14

Give the thumb atomic ops a chance of working

s/ifdef _ARM_ARCH_6/if defined(_ARM_ARCH_6)/ for consistency. NFCI.

Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.

That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)
This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)
This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.5.26.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.5.26.1 21-Apr-2020  martin Sync with HEAD
 1.9 28-Jul-2021  skrll Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)

This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)

This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.8 10-Jul-2021  skrll s/ifdef _ARM_ARCH_6/if defined(_ARM_ARCH_6)/ for consistency. NFCI.
 1.7 30-Nov-2013  joerg branches: 1.7.2; 1.7.28; 1.7.30;
Use explicit form of register pair operations by specifying both.
 1.6 11-Aug-2013  matt Use foo{s} since it reduces the ifdefs for thumb
 1.5 10-Aug-2013  matt Make these under Thumb2
 1.4 10-Aug-2013  matt Use r2 instead of ip
 1.3 10-Aug-2013  matt Use subs/sbc
Use push/pop
Don't adjust return for atomic_dec_64 since it's void and returns nothing
 1.2 13-Sep-2012  matt branches: 1.2.2; 1.2.4;
Correct copyright/fix comments.
 1.1 11-Sep-2012  matt branches: 1.1.2;
Add 64bit atomic ops for ARMv6+ (using ldrexd/strexd).
 1.1.2.2 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.1.2.1 20-Nov-2012  tls Resync to 2012-11-19 00:00:00 UTC
 1.2.4.2 27-Nov-2012  matt Pull atomic ops from HEAD.
 1.2.4.1 13-Sep-2012  matt file atomic_dec_64.S was added on branch matt-nb6-plus on 2012-11-27 23:42:35 +0000
 1.2.2.3 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.2.2.2 30-Oct-2012  yamt sync with head
 1.2.2.1 13-Sep-2012  yamt file atomic_dec_64.S was added on branch yamt-pagecache on 2012-10-30 18:46:11 +0000
 1.7.30.1 11-Aug-2021  martin Pull up following revision(s) (requested by skrll in ticket #1330):

common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.18
common/lib/libc/arch/arm/atomic/atomic_and_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.19
common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_inc_32.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_and_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_add_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_inc_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_cas_up.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_add_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_op_asm.h: revision 1.9
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.8
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_op_asm.h: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_and_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_cas_16.S: revision 1.3
common/lib/libc/arch/arm/atomic/sync_fetch_and_and_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_32.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_xor_32.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_cas_32.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_xor_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_add_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/sync_fetch_and_add_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_dec_64.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_add_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/sync_fetch_and_sub_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_dec_64.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_add_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_xor_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_add_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.14
common/lib/libc/arch/arm/atomic/atomic_or_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_nand_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_or_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_cas_8.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_inc_64.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_inc_64.S: revision 1.11
common/lib/libc/arch/arm/atomic/atomic_cas_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_dec_32.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_or_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_dec_32.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_xor_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_4.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/atomic_or_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_2.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_swap_64.S: revision 1.14
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_1.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_and_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_and_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_and_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.14

Give the thumb atomic ops a chance of working

s/ifdef _ARM_ARCH_6/if defined(_ARM_ARCH_6)/ for consistency. NFCI.

Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.

That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)
This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)
This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.7.28.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.7.28.1 21-Apr-2020  martin Sync with HEAD
 1.7.2.2 15-Feb-2014  matt Add 64-bit atomic ops if ARMv6+
 1.7.2.1 30-Nov-2013  matt file atomic_dec_64.S was added on branch matt-nb5-mips64 on 2014-02-15 10:27:44 +0000
 1.9 28-Jul-2021  skrll Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)

This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)

This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.8 10-Jul-2021  skrll s/ifdef _ARM_ARCH_6/if defined(_ARM_ARCH_6)/ for consistency. NFCI.
 1.7 08-Nov-2013  matt branches: 1.7.26; 1.7.28;
Add support for the gcc __sync builtins.
Note that these need earmv6 or later to get the ldrex/strex instructions
 1.6 11-Aug-2013  matt Use foo{s} since it reduces the ifdefs for thumb
 1.5 10-Aug-2013  matt Make these under Thumb2
 1.4 29-Oct-2012  chs _ARCH_ARM_6 -> _ARM_ARCH_6
 1.3 31-Aug-2012  matt branches: 1.3.2;
Add dmb/dsb instructions as required by the armv7 arch man.
 1.2 16-Aug-2008  matt branches: 1.2.2; 1.2.4; 1.2.24;
Add assembly versions of atomic ops with ldrex/strex
 1.1 26-Aug-2007  matt branches: 1.1.2;
file atomic_inc_32.S was initially added on branch thorpej-atomic.
 1.1.2.1 26-Aug-2007  matt Add armv6 variants of atomic ops which use ldrex/strex.
Add atomic_swap (all arm architectures) which uses swp and swpb.
Add membar_ops which uses armv6 data {sync, memory} barrier.
XXX let thorpej figure out the Makefile magic.
 1.2.24.1 27-Nov-2012  matt Pull atomic ops from HEAD.
 1.2.4.2 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.2.4.1 30-Oct-2012  yamt sync with head
 1.2.2.2 18-Sep-2008  wrstuden Sync with wrstuden-revivesa-base-2.
 1.2.2.1 16-Aug-2008  wrstuden file atomic_inc_32.S was added on branch wrstuden-revivesa on 2008-09-18 04:54:18 +0000
 1.3.2.2 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.3.2.1 20-Nov-2012  tls Resync to 2012-11-19 00:00:00 UTC
 1.7.28.1 11-Aug-2021  martin Pull up following revision(s) (requested by skrll in ticket #1330):

common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.18
common/lib/libc/arch/arm/atomic/atomic_and_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.19
common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_inc_32.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_and_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_add_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_inc_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_cas_up.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_add_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_op_asm.h: revision 1.9
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.8
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_op_asm.h: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_and_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_cas_16.S: revision 1.3
common/lib/libc/arch/arm/atomic/sync_fetch_and_and_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_32.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_xor_32.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_cas_32.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_xor_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_add_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/sync_fetch_and_add_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_dec_64.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_add_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/sync_fetch_and_sub_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_dec_64.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_add_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_xor_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_add_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.14
common/lib/libc/arch/arm/atomic/atomic_or_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_nand_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_or_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_cas_8.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_inc_64.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_inc_64.S: revision 1.11
common/lib/libc/arch/arm/atomic/atomic_cas_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_dec_32.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_or_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_dec_32.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_xor_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_4.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/atomic_or_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_2.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_swap_64.S: revision 1.14
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_1.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_and_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_and_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_and_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.14

Give the thumb atomic ops a chance of working

s/ifdef _ARM_ARCH_6/if defined(_ARM_ARCH_6)/ for consistency. NFCI.

Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.

That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)
This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)
This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.7.26.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.7.26.1 21-Apr-2020  martin Sync with HEAD
 1.11 28-Jul-2021  skrll Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)

This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)

This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.10 10-Jul-2021  skrll s/ifdef _ARM_ARCH_6/if defined(_ARM_ARCH_6)/ for consistency. NFCI.
 1.9 30-Nov-2013  joerg branches: 1.9.2; 1.9.28; 1.9.30;
Use explicit form of register pair operations by specifying both.
 1.8 11-Aug-2013  matt Use foo{s} since it reduces the ifdefs for thumb
 1.7 10-Aug-2013  matt Make these under Thumb2
 1.6 10-Aug-2013  matt Use r2 instead of ip
 1.5 10-Aug-2013  matt Don't adjust return of atomic_inc_64 since it's void and doesn't return
anything.
 1.4 10-Aug-2013  matt Fix add -> adds
 1.3 29-Oct-2012  chs branches: 1.3.2; 1.3.4;
_ARCH_ARM_6 -> _ARM_ARCH_6
 1.2 13-Sep-2012  matt Correct copyright/fix comments.
 1.1 11-Sep-2012  matt branches: 1.1.2;
Add 64bit atomic ops for ARMv6+ (using ldrexd/strexd).
 1.1.2.2 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.1.2.1 20-Nov-2012  tls Resync to 2012-11-19 00:00:00 UTC
 1.3.4.2 27-Nov-2012  matt Pull atomic ops from HEAD.
 1.3.4.1 29-Oct-2012  matt file atomic_inc_64.S was added on branch matt-nb6-plus on 2012-11-27 23:42:35 +0000
 1.3.2.3 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.3.2.2 30-Oct-2012  yamt sync with head
 1.3.2.1 29-Oct-2012  yamt file atomic_inc_64.S was added on branch yamt-pagecache on 2012-10-30 18:46:11 +0000
 1.9.30.1 11-Aug-2021  martin Pull up following revision(s) (requested by skrll in ticket #1330):

common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.18
common/lib/libc/arch/arm/atomic/atomic_and_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.19
common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_inc_32.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_and_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_add_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_inc_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_cas_up.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_add_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_op_asm.h: revision 1.9
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.8
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_op_asm.h: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_and_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_cas_16.S: revision 1.3
common/lib/libc/arch/arm/atomic/sync_fetch_and_and_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_32.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_xor_32.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_cas_32.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_xor_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_add_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/sync_fetch_and_add_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_dec_64.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_add_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/sync_fetch_and_sub_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_dec_64.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_add_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_xor_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_add_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.14
common/lib/libc/arch/arm/atomic/atomic_or_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_nand_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_or_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_cas_8.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_inc_64.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_inc_64.S: revision 1.11
common/lib/libc/arch/arm/atomic/atomic_cas_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_dec_32.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_or_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_dec_32.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_xor_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_4.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/atomic_or_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_2.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_swap_64.S: revision 1.14
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_1.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_and_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_and_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_and_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.14

Give the thumb atomic ops a chance of working

s/ifdef _ARM_ARCH_6/if defined(_ARM_ARCH_6)/ for consistency. NFCI.

Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.

That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)
This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)
This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.9.28.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.9.28.1 21-Apr-2020  martin Sync with HEAD
 1.9.2.2 15-Feb-2014  matt Add 64-bit atomic ops if ARMv6+
 1.9.2.1 30-Nov-2013  matt file atomic_inc_64.S was added on branch matt-nb5-mips64 on 2014-02-15 10:27:44 +0000
 1.5 28-Jul-2021  skrll Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)

This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)

This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.4 10-Jul-2021  skrll s/ifdef _ARM_ARCH_6/if defined(_ARM_ARCH_6)/ for consistency. NFCI.
 1.3 11-Dec-2015  skrll branches: 1.3.16; 1.3.18;
Use gcc 4.4 and later operation for nand, i.e.
*ptr = ~(tmp & value) instead of *ptr = ~tmp & value

There was also another bug in sync_fetch_and_nand_8 which I've also fixed.

PR port-arm32/50513: Incorrect logic for atomic_nand_xx.S
 1.2 04-Mar-2014  matt branches: 1.2.4; 1.2.6; 1.2.8;
Don't export __sync* if _KERNEL || _STANDALONE are defined.
(except if _RUMPKERNEL is defined)
 1.1 08-Nov-2013  matt Add support for the gcc __sync builtins.
Note that these need earmv6 or later to get the ldrex/strex instructions
 1.2.8.2 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.2.8.1 04-Mar-2014  tls file atomic_nand_16.S was added on branch tls-maxphys on 2014-08-19 23:45:12 +0000
 1.2.6.1 26-Feb-2016  snj Pull up following revision(s) (requested by skrll in ticket #1105):
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.3
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.3
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.3
common/lib/libc/arch/arm/atomic/sync_fetch_and_nand_8.S: revision 1.5
Use gcc 4.4 and later operation for nand, i.e.
*ptr = ~(tmp & value) instead of *ptr = ~tmp & value
There was also another bug in sync_fetch_and_nand_8 which I've also fixed.
PR port-arm32/50513: Incorrect logic for atomic_nand_xx.S
 1.2.4.2 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.2.4.1 04-Mar-2014  yamt file atomic_nand_16.S was added on branch yamt-pagecache on 2014-05-22 11:26:28 +0000
 1.3.18.1 11-Aug-2021  martin Pull up following revision(s) (requested by skrll in ticket #1330):

common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.18
common/lib/libc/arch/arm/atomic/atomic_and_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.19
common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_inc_32.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_and_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_add_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_inc_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_cas_up.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_add_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_op_asm.h: revision 1.9
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.8
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_op_asm.h: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_and_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_cas_16.S: revision 1.3
common/lib/libc/arch/arm/atomic/sync_fetch_and_and_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_32.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_xor_32.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_cas_32.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_xor_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_add_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/sync_fetch_and_add_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_dec_64.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_add_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/sync_fetch_and_sub_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_dec_64.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_add_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_xor_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_add_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.14
common/lib/libc/arch/arm/atomic/atomic_or_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_nand_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_or_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_cas_8.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_inc_64.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_inc_64.S: revision 1.11
common/lib/libc/arch/arm/atomic/atomic_cas_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_dec_32.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_or_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_dec_32.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_xor_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_4.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/atomic_or_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_2.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_swap_64.S: revision 1.14
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_1.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_and_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_and_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_and_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.14

Give the thumb atomic ops a chance of working

s/ifdef _ARM_ARCH_6/if defined(_ARM_ARCH_6)/ for consistency. NFCI.

Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.

That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)
This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)
This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.3.16.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.3.16.1 21-Apr-2020  martin Sync with HEAD
 1.5 28-Jul-2021  skrll Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)

This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)

This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.4 10-Jul-2021  skrll s/ifdef _ARM_ARCH_6/if defined(_ARM_ARCH_6)/ for consistency. NFCI.
 1.3 11-Dec-2015  skrll branches: 1.3.16; 1.3.18;
Use gcc 4.4 and later operation for nand, i.e.
*ptr = ~(tmp & value) instead of *ptr = ~tmp & value

There was also another bug in sync_fetch_and_nand_8 which I've also fixed.

PR port-arm32/50513: Incorrect logic for atomic_nand_xx.S
 1.2 04-Mar-2014  matt branches: 1.2.4; 1.2.6; 1.2.8;
Don't export __sync* if _KERNEL || _STANDALONE are defined.
(except if _RUMPKERNEL is defined)
 1.1 08-Nov-2013  matt Add support for the gcc __sync builtins.
Note that these need earmv6 or later to get the ldrex/strex instructions
 1.2.8.2 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.2.8.1 04-Mar-2014  tls file atomic_nand_32.S was added on branch tls-maxphys on 2014-08-19 23:45:12 +0000
 1.2.6.1 26-Feb-2016  snj Pull up following revision(s) (requested by skrll in ticket #1105):
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.3
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.3
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.3
common/lib/libc/arch/arm/atomic/sync_fetch_and_nand_8.S: revision 1.5
Use gcc 4.4 and later operation for nand, i.e.
*ptr = ~(tmp & value) instead of *ptr = ~tmp & value
There was also another bug in sync_fetch_and_nand_8 which I've also fixed.
PR port-arm32/50513: Incorrect logic for atomic_nand_xx.S
 1.2.4.2 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.2.4.1 04-Mar-2014  yamt file atomic_nand_32.S was added on branch yamt-pagecache on 2014-05-22 11:26:28 +0000
 1.3.18.1 11-Aug-2021  martin Pull up following revision(s) (requested by skrll in ticket #1330):

common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.18
common/lib/libc/arch/arm/atomic/atomic_and_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.19
common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_inc_32.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_and_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_add_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_inc_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_cas_up.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_add_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_op_asm.h: revision 1.9
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.8
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_op_asm.h: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_and_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_cas_16.S: revision 1.3
common/lib/libc/arch/arm/atomic/sync_fetch_and_and_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_32.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_xor_32.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_cas_32.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_xor_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_add_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/sync_fetch_and_add_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_dec_64.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_add_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/sync_fetch_and_sub_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_dec_64.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_add_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_xor_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_add_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.14
common/lib/libc/arch/arm/atomic/atomic_or_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_nand_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_or_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_cas_8.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_inc_64.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_inc_64.S: revision 1.11
common/lib/libc/arch/arm/atomic/atomic_cas_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_dec_32.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_or_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_dec_32.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_xor_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_4.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/atomic_or_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_2.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_swap_64.S: revision 1.14
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_1.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_and_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_and_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_and_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.14

Give the thumb atomic ops a chance of working

s/ifdef _ARM_ARCH_6/if defined(_ARM_ARCH_6)/ for consistency. NFCI.

Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.

That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)
This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)
This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.3.16.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.3.16.1 21-Apr-2020  martin Sync with HEAD
 1.7 28-Jul-2021  skrll Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)

This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)

This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.6 10-Jul-2021  skrll s/ifdef _ARM_ARCH_6/if defined(_ARM_ARCH_6)/ for consistency. NFCI.
 1.5 28-Jun-2021  skrll Whitespace
 1.4 11-Dec-2015  skrll branches: 1.4.16; 1.4.18;
Use gcc 4.4 and later operation for nand, i.e.
*ptr = ~(tmp & value) instead of *ptr = ~tmp & value

There was also another bug in sync_fetch_and_nand_8 which I've also fixed.

PR port-arm32/50513: Incorrect logic for atomic_nand_xx.S
 1.3 04-Mar-2014  matt branches: 1.3.4; 1.3.6; 1.3.8;
Don't export __sync* if _KERNEL || _STANDALONE are defined.
(except if _RUMPKERNEL is defined)
 1.2 30-Nov-2013  joerg Use explicit form of register pair operations by specifying both.
 1.1 08-Nov-2013  matt Add support for the gcc __sync builtins.
Note that these need earmv6 or later to get the ldrex/strex instructions
 1.3.8.2 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.3.8.1 04-Mar-2014  tls file atomic_nand_64.S was added on branch tls-maxphys on 2014-08-19 23:45:12 +0000
 1.3.6.1 26-Feb-2016  snj Pull up following revision(s) (requested by skrll in ticket #1105):
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.3
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.3
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.3
common/lib/libc/arch/arm/atomic/sync_fetch_and_nand_8.S: revision 1.5
Use gcc 4.4 and later operation for nand, i.e.
*ptr = ~(tmp & value) instead of *ptr = ~tmp & value
There was also another bug in sync_fetch_and_nand_8 which I've also fixed.
PR port-arm32/50513: Incorrect logic for atomic_nand_xx.S
 1.3.4.2 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.3.4.1 04-Mar-2014  yamt file atomic_nand_64.S was added on branch yamt-pagecache on 2014-05-22 11:26:28 +0000
 1.4.18.2 11-Aug-2021  martin Pull up following revision(s) (requested by skrll in ticket #1330):

common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.18
common/lib/libc/arch/arm/atomic/atomic_and_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.19
common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_inc_32.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_and_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_add_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_inc_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_cas_up.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_add_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_op_asm.h: revision 1.9
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.8
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_op_asm.h: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_and_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_cas_16.S: revision 1.3
common/lib/libc/arch/arm/atomic/sync_fetch_and_and_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_32.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_xor_32.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_cas_32.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_xor_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_add_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/sync_fetch_and_add_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_dec_64.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_add_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/sync_fetch_and_sub_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_dec_64.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_add_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_xor_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_add_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.14
common/lib/libc/arch/arm/atomic/atomic_or_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_nand_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_or_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_cas_8.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_inc_64.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_inc_64.S: revision 1.11
common/lib/libc/arch/arm/atomic/atomic_cas_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_dec_32.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_or_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_dec_32.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_xor_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_4.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/atomic_or_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_2.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_swap_64.S: revision 1.14
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_1.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_and_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_and_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_and_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.14

Give the thumb atomic ops a chance of working

s/ifdef _ARM_ARCH_6/if defined(_ARM_ARCH_6)/ for consistency. NFCI.

Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.

That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)
This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)
This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.4.18.1 06-Jul-2021  martin Pull up following revision(s) (requested by skrll in ticket #1313):

common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/atomic_cas_8.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.3
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.11
common/lib/libc/arch/arm/atomic/atomic_swap_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.12

Whitespace
 1.4.16.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.4.16.1 21-Apr-2020  martin Sync with HEAD
 1.5 28-Jul-2021  skrll Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)

This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)

This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.4 10-Jul-2021  skrll s/ifdef _ARM_ARCH_6/if defined(_ARM_ARCH_6)/ for consistency. NFCI.
 1.3 11-Dec-2015  skrll branches: 1.3.16; 1.3.18;
Use gcc 4.4 and later operation for nand, i.e.
*ptr = ~(tmp & value) instead of *ptr = ~tmp & value

There was also another bug in sync_fetch_and_nand_8 which I've also fixed.

PR port-arm32/50513: Incorrect logic for atomic_nand_xx.S
 1.2 04-Mar-2014  matt branches: 1.2.4; 1.2.6; 1.2.8;
Don't export __sync* if _KERNEL || _STANDALONE are defined.
(except if _RUMPKERNEL is defined)
 1.1 08-Nov-2013  matt Add support for the gcc __sync builtins.
Note that these need earmv6 or later to get the ldrex/strex instructions
 1.2.8.2 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.2.8.1 04-Mar-2014  tls file atomic_nand_8.S was added on branch tls-maxphys on 2014-08-19 23:45:12 +0000
 1.2.6.1 26-Feb-2016  snj Pull up following revision(s) (requested by skrll in ticket #1105):
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.3
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.3
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.3
common/lib/libc/arch/arm/atomic/sync_fetch_and_nand_8.S: revision 1.5
Use gcc 4.4 and later operation for nand, i.e.
*ptr = ~(tmp & value) instead of *ptr = ~tmp & value
There was also another bug in sync_fetch_and_nand_8 which I've also fixed.
PR port-arm32/50513: Incorrect logic for atomic_nand_xx.S
 1.2.4.2 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.2.4.1 04-Mar-2014  yamt file atomic_nand_8.S was added on branch yamt-pagecache on 2014-05-22 11:26:28 +0000
 1.3.18.1 11-Aug-2021  martin Pull up following revision(s) (requested by skrll in ticket #1330):

common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.18
common/lib/libc/arch/arm/atomic/atomic_and_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.19
common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_inc_32.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_and_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_add_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_inc_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_cas_up.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_add_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_op_asm.h: revision 1.9
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.8
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_op_asm.h: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_and_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_cas_16.S: revision 1.3
common/lib/libc/arch/arm/atomic/sync_fetch_and_and_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_32.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_xor_32.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_cas_32.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_xor_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_add_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/sync_fetch_and_add_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_dec_64.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_add_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/sync_fetch_and_sub_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_dec_64.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_add_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_xor_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_add_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.14
common/lib/libc/arch/arm/atomic/atomic_or_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_nand_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_or_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_cas_8.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_inc_64.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_inc_64.S: revision 1.11
common/lib/libc/arch/arm/atomic/atomic_cas_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_dec_32.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_or_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_dec_32.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_xor_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_4.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/atomic_or_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_2.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_swap_64.S: revision 1.14
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_1.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_and_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_and_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_and_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.14

Give the thumb atomic ops a chance of working

s/ifdef _ARM_ARCH_6/if defined(_ARM_ARCH_6)/ for consistency. NFCI.

Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.

That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)
This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)
This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.3.16.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.3.16.1 21-Apr-2020  martin Sync with HEAD
 1.10 28-Jul-2021  simonb #define<tab> consistency.
 1.9 28-Jul-2021  skrll Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)

This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)

This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.8 16-Sep-2019  skrll Traiing whitespace.
 1.7 17-May-2015  justin branches: 1.7.16; 1.7.18;
Do not build arm toolchain symbols in the rump kernel

These symbols will be provided at link time and will be
duplicate symbols in rump kernel and libc if defined.

Many have been fixed previously, but these were missed
as did not have a test.
 1.6 04-Mar-2014  matt Don't export __sync* if _KERNEL || _STANDALONE are defined.
(except if _RUMPKERNEL is defined)
 1.5 22-Feb-2014  martin Try to hide the C runtime implementation specific __sync_* ops from librump,
to avoid duplicates.
 1.4 08-Nov-2013  matt Add support for the gcc __sync builtins.
Note that these need earmv6 or later to get the ldrex/strex instructions
 1.3 11-Sep-2012  matt branches: 1.3.2;
Add 64bit atomic ops for ARMv6+ (using ldrexd/strexd).
 1.2 16-Aug-2008  matt branches: 1.2.2; 1.2.4; 1.2.12; 1.2.24;
Add assembly versions of atomic ops with ldrex/strex
 1.1 18-Apr-2007  thorpej branches: 1.1.2;
file atomic_op_asm.h was initially added on branch thorpej-atomic.
 1.1.2.1 18-Apr-2007  thorpej Atomic ops implementation for ARM. Everything is built on top of
atomic_cas_32(), which is itself a restartable atomic sequence.
 1.2.24.1 27-Nov-2012  matt Pull atomic ops from HEAD.
 1.2.12.1 15-Feb-2014  matt Add 64-bit atomic ops if ARMv6+
 1.2.4.2 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.2.4.1 30-Oct-2012  yamt sync with head
 1.2.2.2 18-Sep-2008  wrstuden Sync with wrstuden-revivesa-base-2.
 1.2.2.1 16-Aug-2008  wrstuden file atomic_op_asm.h was added on branch wrstuden-revivesa on 2008-09-18 04:54:18 +0000
 1.3.2.1 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.7.18.1 11-Aug-2021  martin Pull up following revision(s) (requested by skrll in ticket #1330):

common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.18
common/lib/libc/arch/arm/atomic/atomic_and_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.19
common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_inc_32.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_and_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_add_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_inc_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_cas_up.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_add_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_op_asm.h: revision 1.9
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.8
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_op_asm.h: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_and_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_cas_16.S: revision 1.3
common/lib/libc/arch/arm/atomic/sync_fetch_and_and_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_32.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_xor_32.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_cas_32.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_xor_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_add_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/sync_fetch_and_add_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_dec_64.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_add_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/sync_fetch_and_sub_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_dec_64.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_add_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_xor_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_add_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.14
common/lib/libc/arch/arm/atomic/atomic_or_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_nand_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_or_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_cas_8.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_inc_64.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_inc_64.S: revision 1.11
common/lib/libc/arch/arm/atomic/atomic_cas_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_dec_32.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_or_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_dec_32.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_xor_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_4.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/atomic_or_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_2.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_swap_64.S: revision 1.14
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_1.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_and_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_and_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_and_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.14

Give the thumb atomic ops a chance of working

s/ifdef _ARM_ARCH_6/if defined(_ARM_ARCH_6)/ for consistency. NFCI.

Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.

That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)
This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)
This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.7.16.2 21-Apr-2020  martin Sync with HEAD
 1.7.16.1 13-Apr-2020  martin Mostly merge changes from HEAD upto 20200411
 1.5 28-Jul-2021  skrll Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)

This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)

This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.4 10-Jul-2021  skrll s/ifdef _ARM_ARCH_6/if defined(_ARM_ARCH_6)/ for consistency. NFCI.
 1.3 23-Jun-2014  joerg branches: 1.3.4; 1.3.26; 1.3.28;
Add aliases for the builtins used to implement C11/C++11 atomics.
 1.2 04-Mar-2014  matt branches: 1.2.2; 1.2.4;
Don't export __sync* if _KERNEL || _STANDALONE are defined.
(except if _RUMPKERNEL is defined)
 1.1 08-Nov-2013  matt Add support for the gcc __sync builtins.
Note that these need earmv6 or later to get the ldrex/strex instructions
 1.2.4.2 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.2.4.1 04-Mar-2014  yamt file atomic_or_16.S was added on branch yamt-pagecache on 2014-05-22 11:26:28 +0000
 1.2.2.1 10-Aug-2014  tls Rebase.
 1.3.28.1 11-Aug-2021  martin Pull up following revision(s) (requested by skrll in ticket #1330):

common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.18
common/lib/libc/arch/arm/atomic/atomic_and_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.19
common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_inc_32.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_and_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_add_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_inc_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_cas_up.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_add_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_op_asm.h: revision 1.9
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.8
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_op_asm.h: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_and_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_cas_16.S: revision 1.3
common/lib/libc/arch/arm/atomic/sync_fetch_and_and_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_32.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_xor_32.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_cas_32.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_xor_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_add_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/sync_fetch_and_add_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_dec_64.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_add_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/sync_fetch_and_sub_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_dec_64.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_add_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_xor_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_add_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.14
common/lib/libc/arch/arm/atomic/atomic_or_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_nand_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_or_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_cas_8.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_inc_64.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_inc_64.S: revision 1.11
common/lib/libc/arch/arm/atomic/atomic_cas_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_dec_32.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_or_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_dec_32.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_xor_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_4.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/atomic_or_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_2.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_swap_64.S: revision 1.14
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_1.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_and_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_and_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_and_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.14

Give the thumb atomic ops a chance of working

s/ifdef _ARM_ARCH_6/if defined(_ARM_ARCH_6)/ for consistency. NFCI.

Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.

That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)
This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)
This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.3.26.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.3.26.1 21-Apr-2020  martin Sync with HEAD
 1.3.4.2 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.3.4.1 23-Jun-2014  tls file atomic_or_16.S was added on branch tls-maxphys on 2014-08-19 23:45:12 +0000
 1.10 28-Jul-2021  skrll Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)

This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)

This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.9 10-Jul-2021  skrll s/ifdef _ARM_ARCH_6/if defined(_ARM_ARCH_6)/ for consistency. NFCI.
 1.8 23-Jun-2014  joerg branches: 1.8.24; 1.8.26;
Add aliases for the builtins used to implement C11/C++11 atomics.
 1.7 22-Feb-2014  martin branches: 1.7.2;
Try to hide the C runtime implementation specific __sync_* ops from librump,
to avoid duplicates.
 1.6 08-Nov-2013  matt Add support for the gcc __sync builtins.
Note that these need earmv6 or later to get the ldrex/strex instructions
 1.5 11-Aug-2013  matt Use foo{s} since it reduces the ifdefs for thumb
 1.4 10-Aug-2013  matt Make these under Thumb2
 1.3 31-Aug-2012  matt branches: 1.3.2;
Add dmb/dsb instructions as required by the armv7 arch man.
 1.2 16-Aug-2008  matt branches: 1.2.2; 1.2.4; 1.2.24;
Add assembly versions of atomic ops with ldrex/strex
 1.1 26-Aug-2007  matt branches: 1.1.2;
file atomic_or_32.S was initially added on branch thorpej-atomic.
 1.1.2.2 27-Aug-2007  matt Don't use r4 (since it's caller saved). Use ip (it's available since we
are leaf functions).
 1.1.2.1 26-Aug-2007  matt Add armv6 variants of atomic ops which use ldrex/strex.
Add atomic_swap (all arm architectures) which uses swp and swpb.
Add membar_ops which uses armv6 data {sync, memory} barrier.
XXX let thorpej figure out the Makefile magic.
 1.2.24.1 27-Nov-2012  matt Pull atomic ops from HEAD.
 1.2.4.2 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.2.4.1 30-Oct-2012  yamt sync with head
 1.2.2.2 18-Sep-2008  wrstuden Sync with wrstuden-revivesa-base-2.
 1.2.2.1 16-Aug-2008  wrstuden file atomic_or_32.S was added on branch wrstuden-revivesa on 2008-09-18 04:54:18 +0000
 1.3.2.1 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.7.2.1 10-Aug-2014  tls Rebase.
 1.8.26.1 11-Aug-2021  martin Pull up following revision(s) (requested by skrll in ticket #1330):

common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.18
common/lib/libc/arch/arm/atomic/atomic_and_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.19
common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_inc_32.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_and_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_add_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_inc_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_cas_up.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_add_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_op_asm.h: revision 1.9
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.8
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_op_asm.h: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_and_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_cas_16.S: revision 1.3
common/lib/libc/arch/arm/atomic/sync_fetch_and_and_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_32.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_xor_32.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_cas_32.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_xor_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_add_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/sync_fetch_and_add_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_dec_64.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_add_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/sync_fetch_and_sub_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_dec_64.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_add_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_xor_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_add_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.14
common/lib/libc/arch/arm/atomic/atomic_or_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_nand_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_or_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_cas_8.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_inc_64.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_inc_64.S: revision 1.11
common/lib/libc/arch/arm/atomic/atomic_cas_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_dec_32.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_or_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_dec_32.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_xor_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_4.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/atomic_or_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_2.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_swap_64.S: revision 1.14
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_1.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_and_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_and_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_and_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.14

Give the thumb atomic ops a chance of working

s/ifdef _ARM_ARCH_6/if defined(_ARM_ARCH_6)/ for consistency. NFCI.

Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.

That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)
This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)
This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.8.24.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.8.24.1 21-Apr-2020  martin Sync with HEAD
 1.14 28-Jul-2021  skrll Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)

This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)

This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.13 10-Jul-2021  skrll s/ifdef _ARM_ARCH_6/if defined(_ARM_ARCH_6)/ for consistency. NFCI.
 1.12 28-Jun-2021  skrll Whitespace
 1.11 15-Sep-2019  skrll __sync_{,x}or_and_fetch_8 should return new value... make it so.
 1.10 04-Mar-2014  matt branches: 1.10.18; 1.10.26; 1.10.28;
Don't export __sync* if _KERNEL || _STANDALONE are defined.
(except if _RUMPKERNEL is defined)
 1.9 30-Nov-2013  joerg branches: 1.9.2;
Use explicit form of register pair operations by specifying both.
 1.8 08-Nov-2013  matt Add support for the gcc __sync builtins.
Note that these need earmv6 or later to get the ldrex/strex instructions
 1.7 20-Aug-2013  matt Push two registers to keep stack aligned.
 1.6 11-Aug-2013  matt Use foo{s} since it reduces the ifdefs for thumb
 1.5 10-Aug-2013  matt Make these under Thumb2
 1.4 10-Aug-2013  matt Never modify NLO/NHI (r2,r3) only LO/HI (r0,r1) so subsequent loops will work.
 1.3 10-Aug-2013  matt use push/pop
 1.2 13-Sep-2012  matt branches: 1.2.2; 1.2.4;
Correct copyright/fix comments.
 1.1 11-Sep-2012  matt branches: 1.1.2;
Add 64bit atomic ops for ARMv6+ (using ldrexd/strexd).
 1.1.2.2 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.1.2.1 20-Nov-2012  tls Resync to 2012-11-19 00:00:00 UTC
 1.2.4.2 27-Nov-2012  matt Pull atomic ops from HEAD.
 1.2.4.1 13-Sep-2012  matt file atomic_or_64.S was added on branch matt-nb6-plus on 2012-11-27 23:42:35 +0000
 1.2.2.3 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.2.2.2 30-Oct-2012  yamt sync with head
 1.2.2.1 13-Sep-2012  yamt file atomic_or_64.S was added on branch yamt-pagecache on 2012-10-30 18:46:11 +0000
 1.9.2.2 15-Feb-2014  matt Add 64-bit atomic ops if ARMv6+
 1.9.2.1 30-Nov-2013  matt file atomic_or_64.S was added on branch matt-nb5-mips64 on 2014-02-15 10:27:44 +0000
 1.10.28.3 11-Aug-2021  martin Pull up following revision(s) (requested by skrll in ticket #1330):

common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.18
common/lib/libc/arch/arm/atomic/atomic_and_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.19
common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_inc_32.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_and_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_add_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_inc_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_cas_up.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_add_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_op_asm.h: revision 1.9
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.8
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_op_asm.h: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_and_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_cas_16.S: revision 1.3
common/lib/libc/arch/arm/atomic/sync_fetch_and_and_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_32.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_xor_32.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_cas_32.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_xor_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_add_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/sync_fetch_and_add_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_dec_64.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_add_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/sync_fetch_and_sub_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_dec_64.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_add_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_xor_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_add_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.14
common/lib/libc/arch/arm/atomic/atomic_or_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_nand_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_or_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_cas_8.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_inc_64.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_inc_64.S: revision 1.11
common/lib/libc/arch/arm/atomic/atomic_cas_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_dec_32.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_or_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_dec_32.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_xor_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_4.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/atomic_or_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_2.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_swap_64.S: revision 1.14
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_1.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_and_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_and_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_and_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.14

Give the thumb atomic ops a chance of working

s/ifdef _ARM_ARCH_6/if defined(_ARM_ARCH_6)/ for consistency. NFCI.

Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.

That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)
This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)
This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.10.28.2 06-Jul-2021  martin Pull up following revision(s) (requested by skrll in ticket #1313):

common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/atomic_cas_8.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.3
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.11
common/lib/libc/arch/arm/atomic/atomic_swap_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.12

Whitespace
 1.10.28.1 17-Sep-2019  martin Pull up following revision(s) (requested by skrll in ticket #202):

common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.11

__sync_{,x}or_and_fetch_8 should return new value... make it so.
 1.10.26.2 21-Apr-2020  martin Sync with HEAD
 1.10.26.1 13-Apr-2020  martin Mostly merge changes from HEAD upto 20200411
 1.10.18.1 17-Sep-2019  martin Pull up following revision(s) (requested by skrll in ticket #1375):

common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.11

__sync_{,x}or_and_fetch_8 should return new value... make it so.
 1.5 28-Jul-2021  skrll Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)

This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)

This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.4 10-Jul-2021  skrll s/ifdef _ARM_ARCH_6/if defined(_ARM_ARCH_6)/ for consistency. NFCI.
 1.3 23-Jun-2014  joerg branches: 1.3.4; 1.3.26; 1.3.28;
Add aliases for the builtins used to implement C11/C++11 atomics.
 1.2 04-Mar-2014  matt branches: 1.2.2; 1.2.4;
Don't export __sync* if _KERNEL || _STANDALONE are defined.
(except if _RUMPKERNEL is defined)
 1.1 08-Nov-2013  matt Add support for the gcc __sync builtins.
Note that these need earmv6 or later to get the ldrex/strex instructions
 1.2.4.2 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.2.4.1 04-Mar-2014  yamt file atomic_or_8.S was added on branch yamt-pagecache on 2014-05-22 11:26:28 +0000
 1.2.2.1 10-Aug-2014  tls Rebase.
 1.3.28.1 11-Aug-2021  martin Pull up following revision(s) (requested by skrll in ticket #1330):

common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.18
common/lib/libc/arch/arm/atomic/atomic_and_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.19
common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_inc_32.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_and_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_add_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_inc_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_cas_up.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_add_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_op_asm.h: revision 1.9
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.8
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_op_asm.h: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_and_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_cas_16.S: revision 1.3
common/lib/libc/arch/arm/atomic/sync_fetch_and_and_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_32.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_xor_32.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_cas_32.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_xor_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_add_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/sync_fetch_and_add_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_dec_64.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_add_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/sync_fetch_and_sub_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_dec_64.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_add_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_xor_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_add_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.14
common/lib/libc/arch/arm/atomic/atomic_or_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_nand_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_or_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_cas_8.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_inc_64.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_inc_64.S: revision 1.11
common/lib/libc/arch/arm/atomic/atomic_cas_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_dec_32.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_or_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_dec_32.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_xor_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_4.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/atomic_or_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_2.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_swap_64.S: revision 1.14
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_1.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_and_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_and_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_and_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.14

Give the thumb atomic ops a chance of working

s/ifdef _ARM_ARCH_6/if defined(_ARM_ARCH_6)/ for consistency. NFCI.

Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.

That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)
This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)
This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.3.26.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.3.26.1 21-Apr-2020  martin Sync with HEAD
 1.3.4.2 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.3.4.1 23-Jun-2014  tls file atomic_or_8.S was added on branch tls-maxphys on 2014-08-19 23:45:12 +0000
 1.2 16-Aug-2013  matt branches: 1.2.4; 1.2.8; 1.2.30;
Add a hidden version for libpthread.
 1.1 15-Aug-2013  matt When compiling for Thumb1, the swp instruction is not availble. So this
stub is added to provide __cpu_simple_lock and __cpu_simple_lock_try via
thumb interwork support. It is compiled with -mno-thumb.
 1.2.30.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.2.30.1 21-Apr-2020  martin Sync with HEAD
 1.2.8.2 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.2.8.1 16-Aug-2013  tls file atomic_simplelock.c was added on branch tls-maxphys on 2014-08-19 23:45:12 +0000
 1.2.4.2 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.2.4.1 16-Aug-2013  yamt file atomic_simplelock.c was added on branch yamt-pagecache on 2014-05-22 11:26:28 +0000
 1.5 28-Jul-2021  skrll Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)

This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)

This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.4 10-Jul-2021  skrll s/ifdef _ARM_ARCH_6/if defined(_ARM_ARCH_6)/ for consistency. NFCI.
 1.3 28-Jun-2021  skrll Whitespace
 1.2 04-Mar-2014  matt branches: 1.2.4; 1.2.8; 1.2.30; 1.2.32;
Don't export __sync* if _KERNEL || _STANDALONE are defined.
(except if _RUMPKERNEL is defined)
 1.1 08-Nov-2013  matt Add support for the gcc __sync builtins.
Note that these need earmv6 or later to get the ldrex/strex instructions
 1.2.32.2 11-Aug-2021  martin Pull up following revision(s) (requested by skrll in ticket #1330):

common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.18
common/lib/libc/arch/arm/atomic/atomic_and_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.19
common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_inc_32.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_and_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_add_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_inc_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_cas_up.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_add_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_op_asm.h: revision 1.9
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.8
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_op_asm.h: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_and_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_cas_16.S: revision 1.3
common/lib/libc/arch/arm/atomic/sync_fetch_and_and_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_32.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_xor_32.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_cas_32.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_xor_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_add_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/sync_fetch_and_add_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_dec_64.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_add_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/sync_fetch_and_sub_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_dec_64.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_add_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_xor_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_add_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.14
common/lib/libc/arch/arm/atomic/atomic_or_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_nand_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_or_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_cas_8.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_inc_64.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_inc_64.S: revision 1.11
common/lib/libc/arch/arm/atomic/atomic_cas_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_dec_32.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_or_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_dec_32.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_xor_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_4.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/atomic_or_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_2.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_swap_64.S: revision 1.14
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_1.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_and_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_and_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_and_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.14

Give the thumb atomic ops a chance of working

s/ifdef _ARM_ARCH_6/if defined(_ARM_ARCH_6)/ for consistency. NFCI.

Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.

That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)
This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)
This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.2.32.1 06-Jul-2021  martin Pull up following revision(s) (requested by skrll in ticket #1313):

common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/atomic_cas_8.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.3
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.11
common/lib/libc/arch/arm/atomic/atomic_swap_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.12

Whitespace
 1.2.30.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.2.30.1 21-Apr-2020  martin Sync with HEAD
 1.2.8.2 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.2.8.1 04-Mar-2014  tls file atomic_sub_64.S was added on branch tls-maxphys on 2014-08-19 23:45:12 +0000
 1.2.4.2 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.2.4.1 04-Mar-2014  yamt file atomic_sub_64.S was added on branch yamt-pagecache on 2014-05-22 11:26:28 +0000
 1.19 28-Jul-2021  skrll Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)

This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)

This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.18 10-Jul-2021  skrll s/ifdef _ARM_ARCH_6/if defined(_ARM_ARCH_6)/ for consistency. NFCI.
 1.17 26-Apr-2021  skrll Add the appropriate memory barrier before the lock is cleared in
__sync_lock_release_{1,2,4,8}. That is, all reads and write for in inner
shareability domain before the lock clear store.
 1.16 24-Apr-2021  skrll Trailing whitespace
 1.15 24-Apr-2021  skrll Fix __sync_lock_release_4 to actually zeroise the whole 4bytes/32bits.
 1.14 17-May-2015  justin branches: 1.14.8; 1.14.16; 1.14.18;
Do not build arm toolchain symbols in the rump kernel

These symbols will be provided at link time and will be
duplicate symbols in rump kernel and libc if defined.

Many have been fixed previously, but these were missed
as did not have a test.
 1.13 17-Apr-2015  skrll Use the right register in previous. Spotted by matt@
 1.12 17-Apr-2015  skrll ARM ARM D7.3.2 - ensure all previous accesses are observed before
the lock is cleared
 1.11 28-Jun-2014  joerg Add aliases for the C11/C++11 spelling of the CAS primitives.
 1.10 04-Mar-2014  matt branches: 1.10.2;
Don't export __sync* if _KERNEL || _STANDALONE are defined.
(except if _RUMPKERNEL is defined)
 1.9 30-Jan-2014  matt switch to unified syntax
 1.8 08-Nov-2013  matt Add support for the gcc __sync builtins.
Note that these need earmv6 or later to get the ldrex/strex instructions
 1.7 11-Aug-2013  matt Use foo{s} since it reduces the ifdefs for thumb
 1.6 10-Aug-2013  matt Make these under Thumb2
 1.5 28-Jan-2013  matt Change movsne to movnes for clang.
 1.4 31-Aug-2012  matt branches: 1.4.2;
Add dmb/dsb instructions as required by the armv7 arch man.
 1.3 16-Aug-2012  matt Actually use the assembly version of the atomic function if compiling
for armv6 or armv7 cpus. Use atomic_cas_ptr instead of _lock_cas so
we pick up the assembly version when it's used.
 1.2 16-Aug-2008  matt branches: 1.2.2; 1.2.4; 1.2.12; 1.2.24;
Add assembly versions of atomic ops with ldrex/strex
 1.1 26-Aug-2007  matt branches: 1.1.2;
file atomic_swap.S was initially added on branch thorpej-atomic.
 1.1.2.1 26-Aug-2007  matt Add armv6 variants of atomic ops which use ldrex/strex.
Add atomic_swap (all arm architectures) which uses swp and swpb.
Add membar_ops which uses armv6 data {sync, memory} barrier.
XXX let thorpej figure out the Makefile magic.
 1.2.24.1 27-Nov-2012  matt Pull atomic ops from HEAD.
 1.2.12.1 24-Mar-2014  matt Use ldrex/strex instead of swp when possible.
 1.2.4.2 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.2.4.1 30-Oct-2012  yamt sync with head
 1.2.2.2 18-Sep-2008  wrstuden Sync with wrstuden-revivesa-base-2.
 1.2.2.1 16-Aug-2008  wrstuden file atomic_swap.S was added on branch wrstuden-revivesa on 2008-09-18 04:54:18 +0000
 1.4.2.2 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.4.2.1 25-Feb-2013  tls resync with head
 1.10.2.1 10-Aug-2014  tls Rebase.
 1.14.18.3 11-Aug-2021  martin Pull up following revision(s) (requested by skrll in ticket #1330):

common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.18
common/lib/libc/arch/arm/atomic/atomic_and_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.19
common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_inc_32.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_and_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_add_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_inc_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_cas_up.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_add_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_op_asm.h: revision 1.9
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.8
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_op_asm.h: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_and_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_cas_16.S: revision 1.3
common/lib/libc/arch/arm/atomic/sync_fetch_and_and_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_32.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_xor_32.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_cas_32.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_xor_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_add_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/sync_fetch_and_add_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_dec_64.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_add_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/sync_fetch_and_sub_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_dec_64.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_add_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_xor_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_add_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.14
common/lib/libc/arch/arm/atomic/atomic_or_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_nand_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_or_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_cas_8.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_inc_64.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_inc_64.S: revision 1.11
common/lib/libc/arch/arm/atomic/atomic_cas_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_dec_32.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_or_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_dec_32.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_xor_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_4.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/atomic_or_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_2.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_swap_64.S: revision 1.14
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_1.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_and_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_and_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_and_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.14

Give the thumb atomic ops a chance of working

s/ifdef _ARM_ARCH_6/if defined(_ARM_ARCH_6)/ for consistency. NFCI.

Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.

That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)
This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)
This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.14.18.2 30-Apr-2021  martin Pull up following revision(s) (requested by skrll in ticket #1261):

sys/arch/arm/include/lock.h: revision 1.38
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.17

Add the appropriate memory barrier before the lock is cleared in
__sync_lock_release_{1,2,4,8}. That is, all reads and write for the
inner shareability domain before the lock clear store.

Improve the membar_ops barriers - no need to use dsb and wait for
completion. Also, we only to act on the inner shareability domain.

Fix the barrier confusion. From Riastradh - thanks!.
 1.14.18.1 26-Apr-2021  martin Pull up following revision(s) (requested by skrll in ticket #1254):

sys/arch/arm/include/lock.h: revision 1.37
common/lib/libc/arch/arm/atomic/atomic_swap_64.S: revision 1.11
common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.15
common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.16
sys/arch/arm/include/lock.h: revision 1.36

Fix ARMv8 instructions

Fix __sync_lock_release_4 to actually zeroise the whole 4bytes/32bits.

Trailing whitespace

Change #ifdef FOO to #if defined(FOO). NFCI.
 1.14.16.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.14.16.1 21-Apr-2020  martin Sync with HEAD
 1.14.8.1 26-Apr-2021  martin Pull up following revision(s) (requested by skrll in ticket #1672):

sys/arch/arm/include/lock.h: revision 1.37
common/lib/libc/arch/arm/atomic/atomic_swap_64.S: revision 1.11
common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.15
common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.16
sys/arch/arm/include/lock.h: revision 1.36

Fix ARMv8 instructions

Fix __sync_lock_release_4 to actually zeroise the whole 4bytes/32bits.

Trailing whitespace

Change #ifdef FOO to #if defined(FOO). NFCI.
 1.7 28-Jul-2021  skrll Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)

This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)

This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.6 10-Jul-2021  skrll s/ifdef _ARM_ARCH_6/if defined(_ARM_ARCH_6)/ for consistency. NFCI.
 1.5 26-Apr-2021  skrll Add the appropriate memory barrier before the lock is cleared in
__sync_lock_release_{1,2,4,8}. That is, all reads and write for in inner
shareability domain before the lock clear store.
 1.4 17-May-2015  justin branches: 1.4.16; 1.4.18;
Do not build arm toolchain symbols in the rump kernel

These symbols will be provided at link time and will be
duplicate symbols in rump kernel and libc if defined.

Many have been fixed previously, but these were missed
as did not have a test.
 1.3 28-Jun-2014  joerg branches: 1.3.4;
Add aliases for the C11/C++11 spelling of the CAS primitives.
 1.2 04-Mar-2014  matt branches: 1.2.2; 1.2.4;
Don't export __sync* if _KERNEL || _STANDALONE are defined.
(except if _RUMPKERNEL is defined)
 1.1 08-Nov-2013  matt Add support for the gcc __sync builtins.
Note that these need earmv6 or later to get the ldrex/strex instructions
 1.2.4.2 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.2.4.1 04-Mar-2014  yamt file atomic_swap_16.S was added on branch yamt-pagecache on 2014-05-22 11:26:28 +0000
 1.2.2.1 10-Aug-2014  tls Rebase.
 1.3.4.2 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.3.4.1 28-Jun-2014  tls file atomic_swap_16.S was added on branch tls-maxphys on 2014-08-19 23:45:12 +0000
 1.4.18.2 11-Aug-2021  martin Pull up following revision(s) (requested by skrll in ticket #1330):

common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.18
common/lib/libc/arch/arm/atomic/atomic_and_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.19
common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_inc_32.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_and_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_add_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_inc_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_cas_up.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_add_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_op_asm.h: revision 1.9
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.8
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_op_asm.h: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_and_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_cas_16.S: revision 1.3
common/lib/libc/arch/arm/atomic/sync_fetch_and_and_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_32.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_xor_32.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_cas_32.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_xor_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_add_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/sync_fetch_and_add_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_dec_64.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_add_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/sync_fetch_and_sub_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_dec_64.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_add_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_xor_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_add_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.14
common/lib/libc/arch/arm/atomic/atomic_or_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_nand_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_or_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_cas_8.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_inc_64.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_inc_64.S: revision 1.11
common/lib/libc/arch/arm/atomic/atomic_cas_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_dec_32.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_or_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_dec_32.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_xor_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_4.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/atomic_or_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_2.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_swap_64.S: revision 1.14
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_1.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_and_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_and_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_and_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.14

Give the thumb atomic ops a chance of working

s/ifdef _ARM_ARCH_6/if defined(_ARM_ARCH_6)/ for consistency. NFCI.

Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.

That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)
This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)
This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.4.18.1 30-Apr-2021  martin Pull up following revision(s) (requested by skrll in ticket #1261):

sys/arch/arm/include/lock.h: revision 1.38
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.17

Add the appropriate memory barrier before the lock is cleared in
__sync_lock_release_{1,2,4,8}. That is, all reads and write for the
inner shareability domain before the lock clear store.

Improve the membar_ops barriers - no need to use dsb and wait for
completion. Also, we only to act on the inner shareability domain.

Fix the barrier confusion. From Riastradh - thanks!.
 1.4.16.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.4.16.1 21-Apr-2020  martin Sync with HEAD
 1.15 01-Aug-2021  andvar s/overwriten/overwritten/ in comments.
 1.14 28-Jul-2021  skrll Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)

This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)

This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.13 28-Jun-2021  skrll Whitespace
 1.12 26-Apr-2021  skrll Add the appropriate memory barrier before the lock is cleared in
__sync_lock_release_{1,2,4,8}. That is, all reads and write for in inner
shareability domain before the lock clear store.
 1.11 24-Apr-2021  skrll Trailing whitespace
 1.10 17-May-2015  justin branches: 1.10.8; 1.10.16; 1.10.18;
Do not build arm toolchain symbols in the rump kernel

These symbols will be provided at link time and will be
duplicate symbols in rump kernel and libc if defined.

Many have been fixed previously, but these were missed
as did not have a test.
 1.9 28-Jun-2014  joerg Add aliases for the C11/C++11 spelling of the CAS primitives.
 1.8 04-Mar-2014  matt branches: 1.8.2;
Don't export __sync* if _KERNEL || _STANDALONE are defined.
(except if _RUMPKERNEL is defined)
 1.7 30-Nov-2013  joerg Use explicit form of register pair operations by specifying both.
 1.6 08-Nov-2013  matt Add support for the gcc __sync builtins.
Note that these need earmv6 or later to get the ldrex/strex instructions
 1.5 20-Aug-2013  matt Push two registers to keep stack aligned.
 1.4 10-Aug-2013  matt Make these under Thumb2
 1.3 10-Aug-2013  matt cmpne -> cmp
Use push/pop
 1.2 13-Sep-2012  matt branches: 1.2.2; 1.2.4;
Correct copyright/fix comments.
 1.1 11-Sep-2012  matt branches: 1.1.2;
Add 64bit atomic ops for ARMv6+ (using ldrexd/strexd).
 1.1.2.2 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.1.2.1 20-Nov-2012  tls Resync to 2012-11-19 00:00:00 UTC
 1.2.4.2 27-Nov-2012  matt Pull atomic ops from HEAD.
 1.2.4.1 13-Sep-2012  matt file atomic_swap_64.S was added on branch matt-nb6-plus on 2012-11-27 23:42:35 +0000
 1.2.2.3 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.2.2.2 30-Oct-2012  yamt sync with head
 1.2.2.1 13-Sep-2012  yamt file atomic_swap_64.S was added on branch yamt-pagecache on 2012-10-30 18:46:12 +0000
 1.8.2.1 10-Aug-2014  tls Rebase.
 1.10.18.4 11-Aug-2021  martin Pull up following revision(s) (requested by skrll in ticket #1330):

common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.18
common/lib/libc/arch/arm/atomic/atomic_and_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.19
common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_inc_32.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_and_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_add_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_inc_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_cas_up.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_add_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_op_asm.h: revision 1.9
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.8
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_op_asm.h: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_and_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_cas_16.S: revision 1.3
common/lib/libc/arch/arm/atomic/sync_fetch_and_and_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_32.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_xor_32.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_cas_32.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_xor_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_add_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/sync_fetch_and_add_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_dec_64.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_add_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/sync_fetch_and_sub_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_dec_64.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_add_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_xor_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_add_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.14
common/lib/libc/arch/arm/atomic/atomic_or_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_nand_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_or_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_cas_8.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_inc_64.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_inc_64.S: revision 1.11
common/lib/libc/arch/arm/atomic/atomic_cas_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_dec_32.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_or_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_dec_32.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_xor_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_4.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/atomic_or_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_2.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_swap_64.S: revision 1.14
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_1.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_and_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_and_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_and_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.14

Give the thumb atomic ops a chance of working

s/ifdef _ARM_ARCH_6/if defined(_ARM_ARCH_6)/ for consistency. NFCI.

Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.

That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)
This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)
This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.10.18.3 06-Jul-2021  martin Pull up following revision(s) (requested by skrll in ticket #1313):

common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/atomic_cas_8.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.3
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.11
common/lib/libc/arch/arm/atomic/atomic_swap_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.12

Whitespace
 1.10.18.2 30-Apr-2021  martin Pull up following revision(s) (requested by skrll in ticket #1261):

sys/arch/arm/include/lock.h: revision 1.38
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.17

Add the appropriate memory barrier before the lock is cleared in
__sync_lock_release_{1,2,4,8}. That is, all reads and write for the
inner shareability domain before the lock clear store.

Improve the membar_ops barriers - no need to use dsb and wait for
completion. Also, we only to act on the inner shareability domain.

Fix the barrier confusion. From Riastradh - thanks!.
 1.10.18.1 26-Apr-2021  martin Pull up following revision(s) (requested by skrll in ticket #1254):

sys/arch/arm/include/lock.h: revision 1.37
common/lib/libc/arch/arm/atomic/atomic_swap_64.S: revision 1.11
common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.15
common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.16
sys/arch/arm/include/lock.h: revision 1.36

Fix ARMv8 instructions

Fix __sync_lock_release_4 to actually zeroise the whole 4bytes/32bits.

Trailing whitespace

Change #ifdef FOO to #if defined(FOO). NFCI.
 1.10.16.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.10.16.1 21-Apr-2020  martin Sync with HEAD
 1.10.8.1 26-Apr-2021  martin Pull up following revision(s) (requested by skrll in ticket #1672):

sys/arch/arm/include/lock.h: revision 1.37
common/lib/libc/arch/arm/atomic/atomic_swap_64.S: revision 1.11
common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.15
common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.16
sys/arch/arm/include/lock.h: revision 1.36

Fix ARMv8 instructions

Fix __sync_lock_release_4 to actually zeroise the whole 4bytes/32bits.

Trailing whitespace

Change #ifdef FOO to #if defined(FOO). NFCI.
 1.5 28-Jul-2021  skrll Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)

This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)

This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.4 10-Jul-2021  skrll s/ifdef _ARM_ARCH_6/if defined(_ARM_ARCH_6)/ for consistency. NFCI.
 1.3 23-Jun-2014  joerg branches: 1.3.4; 1.3.26; 1.3.28;
Add aliases for the builtins used to implement C11/C++11 atomics.
 1.2 04-Mar-2014  matt branches: 1.2.2; 1.2.4;
Don't export __sync* if _KERNEL || _STANDALONE are defined.
(except if _RUMPKERNEL is defined)
 1.1 08-Nov-2013  matt Add support for the gcc __sync builtins.
Note that these need earmv6 or later to get the ldrex/strex instructions
 1.2.4.2 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.2.4.1 04-Mar-2014  yamt file atomic_xor_16.S was added on branch yamt-pagecache on 2014-05-22 11:26:28 +0000
 1.2.2.1 10-Aug-2014  tls Rebase.
 1.3.28.1 11-Aug-2021  martin Pull up following revision(s) (requested by skrll in ticket #1330):

common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.18
common/lib/libc/arch/arm/atomic/atomic_and_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.19
common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_inc_32.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_and_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_add_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_inc_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_cas_up.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_add_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_op_asm.h: revision 1.9
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.8
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_op_asm.h: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_and_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_cas_16.S: revision 1.3
common/lib/libc/arch/arm/atomic/sync_fetch_and_and_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_32.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_xor_32.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_cas_32.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_xor_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_add_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/sync_fetch_and_add_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_dec_64.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_add_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/sync_fetch_and_sub_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_dec_64.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_add_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_xor_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_add_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.14
common/lib/libc/arch/arm/atomic/atomic_or_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_nand_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_or_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_cas_8.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_inc_64.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_inc_64.S: revision 1.11
common/lib/libc/arch/arm/atomic/atomic_cas_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_dec_32.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_or_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_dec_32.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_xor_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_4.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/atomic_or_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_2.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_swap_64.S: revision 1.14
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_1.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_and_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_and_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_and_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.14

Give the thumb atomic ops a chance of working

s/ifdef _ARM_ARCH_6/if defined(_ARM_ARCH_6)/ for consistency. NFCI.

Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.

That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)
This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)
This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.3.26.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.3.26.1 21-Apr-2020  martin Sync with HEAD
 1.3.4.2 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.3.4.1 23-Jun-2014  tls file atomic_xor_16.S was added on branch tls-maxphys on 2014-08-19 23:45:12 +0000
 1.5 28-Jul-2021  skrll Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)

This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)

This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.4 10-Jul-2021  skrll s/ifdef _ARM_ARCH_6/if defined(_ARM_ARCH_6)/ for consistency. NFCI.
 1.3 23-Jun-2014  joerg branches: 1.3.4; 1.3.26; 1.3.28;
Add aliases for the builtins used to implement C11/C++11 atomics.
 1.2 04-Mar-2014  matt branches: 1.2.2; 1.2.4;
Don't export __sync* if _KERNEL || _STANDALONE are defined.
(except if _RUMPKERNEL is defined)
 1.1 08-Nov-2013  matt Add support for the gcc __sync builtins.
Note that these need earmv6 or later to get the ldrex/strex instructions
 1.2.4.2 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.2.4.1 04-Mar-2014  yamt file atomic_xor_32.S was added on branch yamt-pagecache on 2014-05-22 11:26:28 +0000
 1.2.2.1 10-Aug-2014  tls Rebase.
 1.3.28.1 11-Aug-2021  martin Pull up following revision(s) (requested by skrll in ticket #1330):

common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.18
common/lib/libc/arch/arm/atomic/atomic_and_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.19
common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_inc_32.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_and_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_add_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_inc_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_cas_up.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_add_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_op_asm.h: revision 1.9
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.8
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_op_asm.h: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_and_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_cas_16.S: revision 1.3
common/lib/libc/arch/arm/atomic/sync_fetch_and_and_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_32.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_xor_32.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_cas_32.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_xor_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_add_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/sync_fetch_and_add_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_dec_64.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_add_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/sync_fetch_and_sub_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_dec_64.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_add_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_xor_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_add_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.14
common/lib/libc/arch/arm/atomic/atomic_or_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_nand_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_or_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_cas_8.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_inc_64.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_inc_64.S: revision 1.11
common/lib/libc/arch/arm/atomic/atomic_cas_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_dec_32.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_or_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_dec_32.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_xor_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_4.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/atomic_or_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_2.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_swap_64.S: revision 1.14
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_1.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_and_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_and_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_and_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.14

Give the thumb atomic ops a chance of working

s/ifdef _ARM_ARCH_6/if defined(_ARM_ARCH_6)/ for consistency. NFCI.

Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.

That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)
This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)
This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.3.26.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.3.26.1 21-Apr-2020  martin Sync with HEAD
 1.3.4.2 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.3.4.1 23-Jun-2014  tls file atomic_xor_32.S was added on branch tls-maxphys on 2014-08-19 23:45:12 +0000
 1.7 28-Jul-2021  skrll Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)

This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)

This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.6 10-Jul-2021  skrll s/ifdef _ARM_ARCH_6/if defined(_ARM_ARCH_6)/ for consistency. NFCI.
 1.5 28-Jun-2021  skrll Whitespace
 1.4 15-Sep-2019  skrll __sync_{,x}or_and_fetch_8 should return new value... make it so.
 1.3 04-Mar-2014  matt branches: 1.3.4; 1.3.8; 1.3.22; 1.3.30; 1.3.32;
Don't export __sync* if _KERNEL || _STANDALONE are defined.
(except if _RUMPKERNEL is defined)
 1.2 30-Nov-2013  joerg Use explicit form of register pair operations by specifying both.
 1.1 08-Nov-2013  matt Add support for the gcc __sync builtins.
Note that these need earmv6 or later to get the ldrex/strex instructions
 1.3.32.3 11-Aug-2021  martin Pull up following revision(s) (requested by skrll in ticket #1330):

common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.18
common/lib/libc/arch/arm/atomic/atomic_and_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.19
common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_inc_32.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_and_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_add_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_inc_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_cas_up.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_add_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_op_asm.h: revision 1.9
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.8
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_op_asm.h: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_and_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_cas_16.S: revision 1.3
common/lib/libc/arch/arm/atomic/sync_fetch_and_and_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_32.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_xor_32.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_cas_32.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_xor_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_add_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/sync_fetch_and_add_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_dec_64.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_add_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/sync_fetch_and_sub_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_dec_64.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_add_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_xor_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_add_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.14
common/lib/libc/arch/arm/atomic/atomic_or_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_nand_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_or_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_cas_8.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_inc_64.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_inc_64.S: revision 1.11
common/lib/libc/arch/arm/atomic/atomic_cas_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_dec_32.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_or_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_dec_32.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_xor_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_4.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/atomic_or_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_2.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_swap_64.S: revision 1.14
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_1.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_and_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_and_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_and_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.14

Give the thumb atomic ops a chance of working

s/ifdef _ARM_ARCH_6/if defined(_ARM_ARCH_6)/ for consistency. NFCI.

Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.

That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)
This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)
This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.3.32.2 06-Jul-2021  martin Pull up following revision(s) (requested by skrll in ticket #1313):

common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/atomic_cas_8.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.3
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.11
common/lib/libc/arch/arm/atomic/atomic_swap_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.12

Whitespace
 1.3.32.1 17-Sep-2019  martin Pull up following revision(s) (requested by skrll in ticket #202):

common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.11

__sync_{,x}or_and_fetch_8 should return new value... make it so.
 1.3.30.2 21-Apr-2020  martin Sync with HEAD
 1.3.30.1 13-Apr-2020  martin Mostly merge changes from HEAD upto 20200411
 1.3.22.1 17-Sep-2019  martin Pull up following revision(s) (requested by skrll in ticket #1375):

common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.11

__sync_{,x}or_and_fetch_8 should return new value... make it so.
 1.3.8.2 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.3.8.1 04-Mar-2014  tls file atomic_xor_64.S was added on branch tls-maxphys on 2014-08-19 23:45:12 +0000
 1.3.4.2 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.3.4.1 04-Mar-2014  yamt file atomic_xor_64.S was added on branch yamt-pagecache on 2014-05-22 11:26:28 +0000
 1.5 28-Jul-2021  skrll Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)

This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)

This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.4 10-Jul-2021  skrll s/ifdef _ARM_ARCH_6/if defined(_ARM_ARCH_6)/ for consistency. NFCI.
 1.3 23-Jun-2014  joerg branches: 1.3.4; 1.3.26; 1.3.28;
Add aliases for the builtins used to implement C11/C++11 atomics.
 1.2 04-Mar-2014  matt branches: 1.2.2; 1.2.4;
Don't export __sync* if _KERNEL || _STANDALONE are defined.
(except if _RUMPKERNEL is defined)
 1.1 08-Nov-2013  matt Add support for the gcc __sync builtins.
Note that these need earmv6 or later to get the ldrex/strex instructions
 1.2.4.2 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.2.4.1 04-Mar-2014  yamt file atomic_xor_8.S was added on branch yamt-pagecache on 2014-05-22 11:26:28 +0000
 1.2.2.1 10-Aug-2014  tls Rebase.
 1.3.28.1 11-Aug-2021  martin Pull up following revision(s) (requested by skrll in ticket #1330):

common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.18
common/lib/libc/arch/arm/atomic/atomic_and_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.19
common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_inc_32.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_and_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_add_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_inc_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_cas_up.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_add_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_op_asm.h: revision 1.9
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.8
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_op_asm.h: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_and_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_cas_16.S: revision 1.3
common/lib/libc/arch/arm/atomic/sync_fetch_and_and_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_32.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_xor_32.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_cas_32.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_xor_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_add_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/sync_fetch_and_add_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_dec_64.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_add_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/sync_fetch_and_sub_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_dec_64.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_add_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_xor_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_add_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.14
common/lib/libc/arch/arm/atomic/atomic_or_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_nand_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_or_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_cas_8.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_inc_64.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_inc_64.S: revision 1.11
common/lib/libc/arch/arm/atomic/atomic_cas_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_dec_32.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_or_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_dec_32.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_xor_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_4.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/atomic_or_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_2.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_swap_64.S: revision 1.14
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_1.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_and_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_and_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_and_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.14

Give the thumb atomic ops a chance of working

s/ifdef _ARM_ARCH_6/if defined(_ARM_ARCH_6)/ for consistency. NFCI.

Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.

That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)
This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)
This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.3.26.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.3.26.1 21-Apr-2020  martin Sync with HEAD
 1.3.4.2 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.3.4.1 23-Jun-2014  tls file atomic_xor_8.S was added on branch tls-maxphys on 2014-08-19 23:45:12 +0000
 1.10 09-Apr-2022  riastradh Introduce membar_acquire/release. Deprecate membar_enter/exit.

The names membar_enter/exit were unclear, and the documentation of
membar_enter has disagreed with the implementations on sparc,
powerpc, and even x86(!) for the entire time it has been in NetBSD.

The terms `acquire' and `release' are ubiquitous in the literature
today, and have been adopted in the C and C++ standards to mean
load-before-load/store and load/store-before-store, respectively,
which are exactly the orderings required by acquiring and releasing a
mutex, as well as other useful applications like decrementing a
reference count and then freeing the underlying object if it went to
zero.

Originally I proposed changing one word in the documentation for
membar_enter to make it load-before-load/store instead of
store-before-load/store, i.e., to make it an acquire barrier. I
proposed this on the grounds that

(a) all implementations guarantee load-before-load/store,
(b) some implementations fail to guarantee store-before-load/store,
and
(c) all uses in-tree assume load-before-load/store.

I verified parts (a) and (b) (except, for (a), powerpc didn't even
guarantee load-before-load/store -- isync isn't necessarily enough;
need lwsync in general -- but it _almost_ did, and it certainly didn't
guarantee store-before-load/store).

Part (c) might not be correct, however: under the mistaken assumption
that atomic-r/m/w then membar-w/rw is equivalent to atomic-r/m/w then
membar-r/rw, I only audited the cases of membar_enter that _aren't_
immediately after an atomic-r/m/w. All of those cases assume
load-before-load/store. But my assumption was wrong -- there are
cases of atomic-r/m/w then membar-w/rw that would be broken by
changing to atomic-r/m/w then membar-r/rw:

https://mail-index.netbsd.org/tech-kern/2022/03/29/msg028044.html

Furthermore, the name membar_enter has been adopted in other places
like OpenBSD where it actually does follow the documentation and
guarantee store-before-load/store, even if that order is not useful.
So the name membar_enter currently lives in a bad place where it
means either of two things -- r/rw or w/rw.

With this change, we deprecate membar_enter/exit, introduce
membar_acquire/release as better names for the useful pair (r/rw and
rw/w), and make sure the implementation of membar_enter guarantees
both what was documented _and_ what was implemented, making it an
alias for membar_sync.

While here, rework all of the membar_* definitions and aliases. The
new logic follows a rule to make it easier to audit:

membar_X is defined as an alias for membar_Y iff membar_X is
guaranteed by membar_Y.

The `no stronger than' relation is (the transitive closure of):

- membar_consumer (r/r) is guaranteed by membar_acquire (r/rw)
- membar_producer (w/w) is guaranteed by membar_release (rw/w)
- membar_acquire (r/rw) is guaranteed by membar_sync (rw/rw)
- membar_release (rw/w) is guaranteed by membar_sync (rw/rw)

And, for the deprecated membars:

- membar_enter (whether r/rw, w/rw, or rw/rw) is guaranteed by
membar_sync (rw/rw)
- membar_exit (rw/w) is guaranteed by membar_release (rw/w)

(membar_exit is identical to membar_release, but the name is
deprecated.)

Finally, while here, annotate some of the instructions with their
semantics. For powerpc, leave an essay with citations on the
unfortunate but -- as far as I can tell -- necessary decision to use
lwsync, not isync, for membar_acquire and membar_consumer.

Also add membar(3) and atomic(3) man page links.
 1.9 28-Jul-2021  skrll Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)

This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)

This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.8 10-Jul-2021  skrll s/ifdef _ARM_ARCH_6/if defined(_ARM_ARCH_6)/ for consistency. NFCI.
 1.7 27-Apr-2021  skrll Improve the membar_ops barriers - no need to use dsb and wait for
completion. Also, we only to act on the inner shareability domain.
 1.6 28-Mar-2014  skrll branches: 1.6.26; 1.6.28;
Ensure SBZ register is zero
 1.5 04-Mar-2014  matt Don't export __sync* if _KERNEL || _STANDALONE are defined.
(except if _RUMPKERNEL is defined)
 1.4 08-Nov-2013  matt Add support for the gcc __sync builtins.
Note that these need earmv6 or later to get the ldrex/strex instructions
 1.3 16-Aug-2012  matt branches: 1.3.2;
Actually use the assembly version of the atomic function if compiling
for armv6 or armv7 cpus. Use atomic_cas_ptr instead of _lock_cas so
we pick up the assembly version when it's used.
 1.2 16-Aug-2008  matt branches: 1.2.2; 1.2.4; 1.2.24;
Add assembly versions of atomic ops with ldrex/strex
 1.1 26-Aug-2007  matt branches: 1.1.2;
file membar_ops.S was initially added on branch thorpej-atomic.
 1.1.2.1 26-Aug-2007  matt Add armv6 variants of atomic ops which use ldrex/strex.
Add atomic_swap (all arm architectures) which uses swp and swpb.
Add membar_ops which uses armv6 data {sync, memory} barrier.
XXX let thorpej figure out the Makefile magic.
 1.2.24.1 27-Nov-2012  matt Pull atomic ops from HEAD.
 1.2.4.2 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.2.4.1 30-Oct-2012  yamt sync with head
 1.2.2.2 18-Sep-2008  wrstuden Sync with wrstuden-revivesa-base-2.
 1.2.2.1 16-Aug-2008  wrstuden file membar_ops.S was added on branch wrstuden-revivesa on 2008-09-18 04:54:18 +0000
 1.3.2.1 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.6.28.2 11-Aug-2021  martin Pull up following revision(s) (requested by skrll in ticket #1330):

common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.18
common/lib/libc/arch/arm/atomic/atomic_and_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.19
common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_inc_32.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_and_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_add_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_inc_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_cas_up.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_add_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_op_asm.h: revision 1.9
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.8
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_op_asm.h: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_and_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_cas_16.S: revision 1.3
common/lib/libc/arch/arm/atomic/sync_fetch_and_and_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_32.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_xor_32.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_cas_32.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_xor_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_add_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/sync_fetch_and_add_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_dec_64.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_add_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/sync_fetch_and_sub_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_dec_64.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_add_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_xor_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_add_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.14
common/lib/libc/arch/arm/atomic/atomic_or_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_nand_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_or_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_cas_8.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_inc_64.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_inc_64.S: revision 1.11
common/lib/libc/arch/arm/atomic/atomic_cas_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_dec_32.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_or_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_dec_32.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_xor_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_4.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/atomic_or_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_2.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_swap_64.S: revision 1.14
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_1.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_and_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_and_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_and_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.14

Give the thumb atomic ops a chance of working

s/ifdef _ARM_ARCH_6/if defined(_ARM_ARCH_6)/ for consistency. NFCI.

Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.

That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)
This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)
This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.6.28.1 30-Apr-2021  martin Pull up following revision(s) (requested by skrll in ticket #1261):

sys/arch/arm/include/lock.h: revision 1.38
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.17

Add the appropriate memory barrier before the lock is cleared in
__sync_lock_release_{1,2,4,8}. That is, all reads and write for the
inner shareability domain before the lock clear store.

Improve the membar_ops barriers - no need to use dsb and wait for
completion. Also, we only to act on the inner shareability domain.

Fix the barrier confusion. From Riastradh - thanks!.
 1.6.26.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.6.26.1 21-Apr-2020  martin Sync with HEAD
 1.4 28-Jul-2021  skrll Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)

This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)

This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.3 17-May-2015  justin branches: 1.3.16; 1.3.18;
Move arm sync_* changes to Makefile.inc
 1.2 17-May-2015  justin Do not build arm toolchain symbols in the rump kernel

These symbols will be provided at link time and will be
duplicate symbols in rump kernel and libc if defined.

Many have been fixed previously, but these were missed
as did not have a test.
 1.1 08-Nov-2013  matt branches: 1.1.4; 1.1.8;
Add support for the gcc __sync builtins.
Note that these need earmv6 or later to get the ldrex/strex instructions
 1.1.8.2 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.1.8.1 08-Nov-2013  tls file sync_bool_compare_and_swap_1.S was added on branch tls-maxphys on 2014-08-19 23:45:12 +0000
 1.1.4.2 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.1.4.1 08-Nov-2013  yamt file sync_bool_compare_and_swap_1.S was added on branch yamt-pagecache on 2014-05-22 11:26:28 +0000
 1.3.18.1 11-Aug-2021  martin Pull up following revision(s) (requested by skrll in ticket #1330):

common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.18
common/lib/libc/arch/arm/atomic/atomic_and_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.19
common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_inc_32.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_and_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_add_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_inc_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_cas_up.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_add_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_op_asm.h: revision 1.9
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.8
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_op_asm.h: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_and_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_cas_16.S: revision 1.3
common/lib/libc/arch/arm/atomic/sync_fetch_and_and_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_32.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_xor_32.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_cas_32.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_xor_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_add_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/sync_fetch_and_add_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_dec_64.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_add_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/sync_fetch_and_sub_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_dec_64.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_add_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_xor_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_add_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.14
common/lib/libc/arch/arm/atomic/atomic_or_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_nand_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_or_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_cas_8.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_inc_64.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_inc_64.S: revision 1.11
common/lib/libc/arch/arm/atomic/atomic_cas_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_dec_32.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_or_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_dec_32.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_xor_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_4.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/atomic_or_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_2.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_swap_64.S: revision 1.14
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_1.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_and_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_and_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_and_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.14

Give the thumb atomic ops a chance of working

s/ifdef _ARM_ARCH_6/if defined(_ARM_ARCH_6)/ for consistency. NFCI.

Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.

That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)
This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)
This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.3.16.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.3.16.1 21-Apr-2020  martin Sync with HEAD
 1.4 28-Jul-2021  skrll Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)

This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)

This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.3 17-May-2015  justin branches: 1.3.16; 1.3.18;
Move arm sync_* changes to Makefile.inc
 1.2 17-May-2015  justin Do not build arm toolchain symbols in the rump kernel

These symbols will be provided at link time and will be
duplicate symbols in rump kernel and libc if defined.

Many have been fixed previously, but these were missed
as did not have a test.
 1.1 08-Nov-2013  matt branches: 1.1.4; 1.1.8;
Add support for the gcc __sync builtins.
Note that these need earmv6 or later to get the ldrex/strex instructions
 1.1.8.2 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.1.8.1 08-Nov-2013  tls file sync_bool_compare_and_swap_2.S was added on branch tls-maxphys on 2014-08-19 23:45:12 +0000
 1.1.4.2 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.1.4.1 08-Nov-2013  yamt file sync_bool_compare_and_swap_2.S was added on branch yamt-pagecache on 2014-05-22 11:26:28 +0000
 1.3.18.1 11-Aug-2021  martin Pull up following revision(s) (requested by skrll in ticket #1330):

common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.18
common/lib/libc/arch/arm/atomic/atomic_and_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.19
common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_inc_32.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_and_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_add_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_inc_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_cas_up.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_add_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_op_asm.h: revision 1.9
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.8
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_op_asm.h: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_and_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_cas_16.S: revision 1.3
common/lib/libc/arch/arm/atomic/sync_fetch_and_and_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_32.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_xor_32.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_cas_32.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_xor_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_add_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/sync_fetch_and_add_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_dec_64.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_add_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/sync_fetch_and_sub_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_dec_64.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_add_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_xor_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_add_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.14
common/lib/libc/arch/arm/atomic/atomic_or_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_nand_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_or_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_cas_8.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_inc_64.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_inc_64.S: revision 1.11
common/lib/libc/arch/arm/atomic/atomic_cas_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_dec_32.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_or_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_dec_32.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_xor_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_4.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/atomic_or_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_2.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_swap_64.S: revision 1.14
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_1.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_and_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_and_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_and_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.14

Give the thumb atomic ops a chance of working

s/ifdef _ARM_ARCH_6/if defined(_ARM_ARCH_6)/ for consistency. NFCI.

Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.

That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)
This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)
This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.3.16.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.3.16.1 21-Apr-2020  martin Sync with HEAD
 1.4 28-Jul-2021  skrll Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)

This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)

This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.3 17-May-2015  justin branches: 1.3.16; 1.3.18;
Move arm sync_* changes to Makefile.inc
 1.2 17-May-2015  justin Do not build arm toolchain symbols in the rump kernel

These symbols will be provided at link time and will be
duplicate symbols in rump kernel and libc if defined.

Many have been fixed previously, but these were missed
as did not have a test.
 1.1 08-Nov-2013  matt branches: 1.1.4; 1.1.8;
Add support for the gcc __sync builtins.
Note that these need earmv6 or later to get the ldrex/strex instructions
 1.1.8.2 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.1.8.1 08-Nov-2013  tls file sync_bool_compare_and_swap_4.S was added on branch tls-maxphys on 2014-08-19 23:45:12 +0000
 1.1.4.2 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.1.4.1 08-Nov-2013  yamt file sync_bool_compare_and_swap_4.S was added on branch yamt-pagecache on 2014-05-22 11:26:28 +0000
 1.3.18.1 11-Aug-2021  martin Pull up following revision(s) (requested by skrll in ticket #1330):

common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.18
common/lib/libc/arch/arm/atomic/atomic_and_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.19
common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_inc_32.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_and_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_add_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_inc_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_cas_up.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_add_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_op_asm.h: revision 1.9
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.8
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_op_asm.h: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_and_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_cas_16.S: revision 1.3
common/lib/libc/arch/arm/atomic/sync_fetch_and_and_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_32.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_xor_32.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_cas_32.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_xor_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_add_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/sync_fetch_and_add_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_dec_64.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_add_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/sync_fetch_and_sub_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_dec_64.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_add_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_xor_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_add_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.14
common/lib/libc/arch/arm/atomic/atomic_or_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_nand_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_or_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_cas_8.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_inc_64.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_inc_64.S: revision 1.11
common/lib/libc/arch/arm/atomic/atomic_cas_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_dec_32.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_or_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_dec_32.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_xor_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_4.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/atomic_or_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_2.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_swap_64.S: revision 1.14
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_1.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_and_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_and_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_and_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.14

Give the thumb atomic ops a chance of working

s/ifdef _ARM_ARCH_6/if defined(_ARM_ARCH_6)/ for consistency. NFCI.

Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.

That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)
This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)
This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.3.16.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.3.16.1 21-Apr-2020  martin Sync with HEAD
 1.5 28-Jul-2021  skrll Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)

This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)

This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.4 17-May-2015  justin branches: 1.4.16; 1.4.18;
Move arm sync_* changes to Makefile.inc
 1.3 17-May-2015  justin Do not build arm toolchain symbols in the rump kernel

These symbols will be provided at link time and will be
duplicate symbols in rump kernel and libc if defined.

Many have been fixed previously, but these were missed
as did not have a test.
 1.2 30-Nov-2013  joerg branches: 1.2.4; 1.2.8;
Use explicit form of register pair operations by specifying both.
 1.1 08-Nov-2013  matt Add support for the gcc __sync builtins.
Note that these need earmv6 or later to get the ldrex/strex instructions
 1.2.8.2 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.2.8.1 30-Nov-2013  tls file sync_bool_compare_and_swap_8.S was added on branch tls-maxphys on 2014-08-19 23:45:12 +0000
 1.2.4.2 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.2.4.1 30-Nov-2013  yamt file sync_bool_compare_and_swap_8.S was added on branch yamt-pagecache on 2014-05-22 11:26:28 +0000
 1.4.18.1 11-Aug-2021  martin Pull up following revision(s) (requested by skrll in ticket #1330):

common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.18
common/lib/libc/arch/arm/atomic/atomic_and_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.19
common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_inc_32.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_and_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_add_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_inc_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_cas_up.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_add_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_op_asm.h: revision 1.9
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.8
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_op_asm.h: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_and_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_cas_16.S: revision 1.3
common/lib/libc/arch/arm/atomic/sync_fetch_and_and_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_32.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_xor_32.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_cas_32.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_xor_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_add_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/sync_fetch_and_add_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_dec_64.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_add_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/sync_fetch_and_sub_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_dec_64.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_add_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_xor_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_add_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.14
common/lib/libc/arch/arm/atomic/atomic_or_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_nand_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_or_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_cas_8.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_inc_64.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_inc_64.S: revision 1.11
common/lib/libc/arch/arm/atomic/atomic_cas_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_dec_32.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_or_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_dec_32.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_xor_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_4.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/atomic_or_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_2.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_swap_64.S: revision 1.14
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_1.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_and_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_and_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_and_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.14

Give the thumb atomic ops a chance of working

s/ifdef _ARM_ARCH_6/if defined(_ARM_ARCH_6)/ for consistency. NFCI.

Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.

That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)
This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)
This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.4.16.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.4.16.1 21-Apr-2020  martin Sync with HEAD
 1.6 28-Jul-2021  skrll Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)

This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)

This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.5 17-May-2015  justin branches: 1.5.16; 1.5.18;
Move arm sync_* changes to Makefile.inc
 1.4 17-May-2015  justin Do not build arm toolchain symbols in the rump kernel

These symbols will be provided at link time and will be
duplicate symbols in rump kernel and libc if defined.

Many have been fixed previously, but these were missed
as did not have a test.
 1.3 23-Jun-2014  joerg branches: 1.3.4;
Add aliases for the builtins used to implement C11/C++11 atomics.
 1.2 30-Nov-2013  joerg branches: 1.2.2; 1.2.4;
Use explicit form of register pair operations by specifying both.
 1.1 08-Nov-2013  matt Add support for the gcc __sync builtins.
Note that these need earmv6 or later to get the ldrex/strex instructions
 1.2.4.2 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.2.4.1 30-Nov-2013  yamt file sync_fetch_and_add_8.S was added on branch yamt-pagecache on 2014-05-22 11:26:28 +0000
 1.2.2.1 10-Aug-2014  tls Rebase.
 1.3.4.2 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.3.4.1 23-Jun-2014  tls file sync_fetch_and_add_8.S was added on branch tls-maxphys on 2014-08-19 23:45:12 +0000
 1.5.18.1 11-Aug-2021  martin Pull up following revision(s) (requested by skrll in ticket #1330):

common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.18
common/lib/libc/arch/arm/atomic/atomic_and_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.19
common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_inc_32.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_and_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_add_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_inc_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_cas_up.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_add_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_op_asm.h: revision 1.9
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.8
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_op_asm.h: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_and_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_cas_16.S: revision 1.3
common/lib/libc/arch/arm/atomic/sync_fetch_and_and_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_32.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_xor_32.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_cas_32.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_xor_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_add_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/sync_fetch_and_add_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_dec_64.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_add_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/sync_fetch_and_sub_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_dec_64.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_add_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_xor_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_add_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.14
common/lib/libc/arch/arm/atomic/atomic_or_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_nand_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_or_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_cas_8.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_inc_64.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_inc_64.S: revision 1.11
common/lib/libc/arch/arm/atomic/atomic_cas_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_dec_32.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_or_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_dec_32.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_xor_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_4.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/atomic_or_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_2.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_swap_64.S: revision 1.14
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_1.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_and_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_and_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_and_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.14

Give the thumb atomic ops a chance of working

s/ifdef _ARM_ARCH_6/if defined(_ARM_ARCH_6)/ for consistency. NFCI.

Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.

That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)
This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)
This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.5.16.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.5.16.1 21-Apr-2020  martin Sync with HEAD
 1.6 28-Jul-2021  skrll Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)

This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)

This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.5 17-May-2015  justin branches: 1.5.16; 1.5.18;
Move arm sync_* changes to Makefile.inc
 1.4 17-May-2015  justin Do not build arm toolchain symbols in the rump kernel

These symbols will be provided at link time and will be
duplicate symbols in rump kernel and libc if defined.

Many have been fixed previously, but these were missed
as did not have a test.
 1.3 23-Jun-2014  joerg branches: 1.3.4;
Add aliases for the builtins used to implement C11/C++11 atomics.
 1.2 30-Nov-2013  joerg branches: 1.2.2; 1.2.4;
Use explicit form of register pair operations by specifying both.
 1.1 08-Nov-2013  matt Add support for the gcc __sync builtins.
Note that these need earmv6 or later to get the ldrex/strex instructions
 1.2.4.2 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.2.4.1 30-Nov-2013  yamt file sync_fetch_and_and_8.S was added on branch yamt-pagecache on 2014-05-22 11:26:28 +0000
 1.2.2.1 10-Aug-2014  tls Rebase.
 1.3.4.2 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.3.4.1 23-Jun-2014  tls file sync_fetch_and_and_8.S was added on branch tls-maxphys on 2014-08-19 23:45:12 +0000
 1.5.18.1 11-Aug-2021  martin Pull up following revision(s) (requested by skrll in ticket #1330):

common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.18
common/lib/libc/arch/arm/atomic/atomic_and_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.19
common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_inc_32.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_and_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_add_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_inc_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_cas_up.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_add_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_op_asm.h: revision 1.9
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.8
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_op_asm.h: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_and_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_cas_16.S: revision 1.3
common/lib/libc/arch/arm/atomic/sync_fetch_and_and_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_32.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_xor_32.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_cas_32.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_xor_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_add_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/sync_fetch_and_add_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_dec_64.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_add_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/sync_fetch_and_sub_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_dec_64.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_add_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_xor_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_add_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.14
common/lib/libc/arch/arm/atomic/atomic_or_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_nand_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_or_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_cas_8.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_inc_64.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_inc_64.S: revision 1.11
common/lib/libc/arch/arm/atomic/atomic_cas_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_dec_32.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_or_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_dec_32.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_xor_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_4.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/atomic_or_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_2.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_swap_64.S: revision 1.14
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_1.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_and_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_and_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_and_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.14

Give the thumb atomic ops a chance of working

s/ifdef _ARM_ARCH_6/if defined(_ARM_ARCH_6)/ for consistency. NFCI.

Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.

That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)
This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)
This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.5.16.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.5.16.1 21-Apr-2020  martin Sync with HEAD
 1.6 28-Jul-2021  skrll Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)

This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)

This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.5 11-Dec-2015  skrll branches: 1.5.16; 1.5.18;
Use gcc 4.4 and later operation for nand, i.e.
*ptr = ~(tmp & value) instead of *ptr = ~tmp & value

There was also another bug in sync_fetch_and_nand_8 which I've also fixed.

PR port-arm32/50513: Incorrect logic for atomic_nand_xx.S
 1.4 17-May-2015  justin Move arm sync_* changes to Makefile.inc
 1.3 17-May-2015  justin Do not build arm toolchain symbols in the rump kernel

These symbols will be provided at link time and will be
duplicate symbols in rump kernel and libc if defined.

Many have been fixed previously, but these were missed
as did not have a test.
 1.2 30-Nov-2013  joerg branches: 1.2.4; 1.2.6; 1.2.8;
Use explicit form of register pair operations by specifying both.
 1.1 08-Nov-2013  matt Add support for the gcc __sync builtins.
Note that these need earmv6 or later to get the ldrex/strex instructions
 1.2.8.2 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.2.8.1 30-Nov-2013  tls file sync_fetch_and_nand_8.S was added on branch tls-maxphys on 2014-08-19 23:45:12 +0000
 1.2.6.1 26-Feb-2016  snj Pull up following revision(s) (requested by skrll in ticket #1105):
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.3
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.3
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.3
common/lib/libc/arch/arm/atomic/sync_fetch_and_nand_8.S: revision 1.5
Use gcc 4.4 and later operation for nand, i.e.
*ptr = ~(tmp & value) instead of *ptr = ~tmp & value
There was also another bug in sync_fetch_and_nand_8 which I've also fixed.
PR port-arm32/50513: Incorrect logic for atomic_nand_xx.S
 1.2.4.2 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.2.4.1 30-Nov-2013  yamt file sync_fetch_and_nand_8.S was added on branch yamt-pagecache on 2014-05-22 11:26:28 +0000
 1.5.18.1 11-Aug-2021  martin Pull up following revision(s) (requested by skrll in ticket #1330):

common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.18
common/lib/libc/arch/arm/atomic/atomic_and_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.19
common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_inc_32.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_and_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_add_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_inc_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_cas_up.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_add_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_op_asm.h: revision 1.9
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.8
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_op_asm.h: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_and_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_cas_16.S: revision 1.3
common/lib/libc/arch/arm/atomic/sync_fetch_and_and_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_32.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_xor_32.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_cas_32.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_xor_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_add_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/sync_fetch_and_add_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_dec_64.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_add_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/sync_fetch_and_sub_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_dec_64.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_add_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_xor_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_add_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.14
common/lib/libc/arch/arm/atomic/atomic_or_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_nand_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_or_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_cas_8.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_inc_64.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_inc_64.S: revision 1.11
common/lib/libc/arch/arm/atomic/atomic_cas_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_dec_32.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_or_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_dec_32.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_xor_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_4.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/atomic_or_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_2.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_swap_64.S: revision 1.14
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_1.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_and_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_and_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_and_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.14

Give the thumb atomic ops a chance of working

s/ifdef _ARM_ARCH_6/if defined(_ARM_ARCH_6)/ for consistency. NFCI.

Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.

That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)
This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)
This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.5.16.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.5.16.1 21-Apr-2020  martin Sync with HEAD
 1.6 28-Jul-2021  skrll Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)

This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)

This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.5 17-May-2015  justin branches: 1.5.16; 1.5.18;
Move arm sync_* changes to Makefile.inc
 1.4 17-May-2015  justin Do not build arm toolchain symbols in the rump kernel

These symbols will be provided at link time and will be
duplicate symbols in rump kernel and libc if defined.

Many have been fixed previously, but these were missed
as did not have a test.
 1.3 23-Jun-2014  joerg branches: 1.3.4;
Add aliases for the builtins used to implement C11/C++11 atomics.
 1.2 30-Nov-2013  joerg branches: 1.2.2; 1.2.4;
Use explicit form of register pair operations by specifying both.
 1.1 08-Nov-2013  matt Add support for the gcc __sync builtins.
Note that these need earmv6 or later to get the ldrex/strex instructions
 1.2.4.2 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.2.4.1 30-Nov-2013  yamt file sync_fetch_and_or_8.S was added on branch yamt-pagecache on 2014-05-22 11:26:28 +0000
 1.2.2.1 10-Aug-2014  tls Rebase.
 1.3.4.2 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.3.4.1 23-Jun-2014  tls file sync_fetch_and_or_8.S was added on branch tls-maxphys on 2014-08-19 23:45:12 +0000
 1.5.18.1 11-Aug-2021  martin Pull up following revision(s) (requested by skrll in ticket #1330):

common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.18
common/lib/libc/arch/arm/atomic/atomic_and_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.19
common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_inc_32.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_and_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_add_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_inc_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_cas_up.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_add_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_op_asm.h: revision 1.9
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.8
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_op_asm.h: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_and_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_cas_16.S: revision 1.3
common/lib/libc/arch/arm/atomic/sync_fetch_and_and_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_32.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_xor_32.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_cas_32.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_xor_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_add_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/sync_fetch_and_add_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_dec_64.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_add_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/sync_fetch_and_sub_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_dec_64.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_add_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_xor_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_add_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.14
common/lib/libc/arch/arm/atomic/atomic_or_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_nand_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_or_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_cas_8.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_inc_64.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_inc_64.S: revision 1.11
common/lib/libc/arch/arm/atomic/atomic_cas_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_dec_32.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_or_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_dec_32.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_xor_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_4.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/atomic_or_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_2.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_swap_64.S: revision 1.14
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_1.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_and_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_and_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_and_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.14

Give the thumb atomic ops a chance of working

s/ifdef _ARM_ARCH_6/if defined(_ARM_ARCH_6)/ for consistency. NFCI.

Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.

That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)
This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)
This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.5.16.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.5.16.1 21-Apr-2020  martin Sync with HEAD
 1.6 28-Jul-2021  skrll Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)

This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)

This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.5 17-May-2015  justin branches: 1.5.16; 1.5.18;
Move arm sync_* changes to Makefile.inc
 1.4 17-May-2015  justin Do not build arm toolchain symbols in the rump kernel

These symbols will be provided at link time and will be
duplicate symbols in rump kernel and libc if defined.

Many have been fixed previously, but these were missed
as did not have a test.
 1.3 23-Jun-2014  joerg branches: 1.3.4;
Add aliases for the builtins used to implement C11/C++11 atomics.
 1.2 30-Nov-2013  joerg branches: 1.2.2; 1.2.4;
Use explicit form of register pair operations by specifying both.
 1.1 08-Nov-2013  matt Add support for the gcc __sync builtins.
Note that these need earmv6 or later to get the ldrex/strex instructions
 1.2.4.2 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.2.4.1 30-Nov-2013  yamt file sync_fetch_and_sub_8.S was added on branch yamt-pagecache on 2014-05-22 11:26:28 +0000
 1.2.2.1 10-Aug-2014  tls Rebase.
 1.3.4.2 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.3.4.1 23-Jun-2014  tls file sync_fetch_and_sub_8.S was added on branch tls-maxphys on 2014-08-19 23:45:12 +0000
 1.5.18.1 11-Aug-2021  martin Pull up following revision(s) (requested by skrll in ticket #1330):

common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.18
common/lib/libc/arch/arm/atomic/atomic_and_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.19
common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_inc_32.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_and_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_add_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_inc_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_cas_up.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_add_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_op_asm.h: revision 1.9
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.8
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_op_asm.h: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_and_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_cas_16.S: revision 1.3
common/lib/libc/arch/arm/atomic/sync_fetch_and_and_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_32.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_xor_32.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_cas_32.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_xor_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_add_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/sync_fetch_and_add_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_dec_64.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_add_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/sync_fetch_and_sub_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_dec_64.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_add_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_xor_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_add_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.14
common/lib/libc/arch/arm/atomic/atomic_or_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_nand_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_or_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_cas_8.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_inc_64.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_inc_64.S: revision 1.11
common/lib/libc/arch/arm/atomic/atomic_cas_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_dec_32.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_or_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_dec_32.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_xor_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_4.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/atomic_or_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_2.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_swap_64.S: revision 1.14
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_1.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_and_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_and_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_and_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.14

Give the thumb atomic ops a chance of working

s/ifdef _ARM_ARCH_6/if defined(_ARM_ARCH_6)/ for consistency. NFCI.

Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.

That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)
This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)
This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.5.16.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.5.16.1 21-Apr-2020  martin Sync with HEAD
 1.6 28-Jul-2021  skrll Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)

This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)

This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.5 17-May-2015  justin branches: 1.5.16; 1.5.18;
Move arm sync_* changes to Makefile.inc
 1.4 17-May-2015  justin Do not build arm toolchain symbols in the rump kernel

These symbols will be provided at link time and will be
duplicate symbols in rump kernel and libc if defined.

Many have been fixed previously, but these were missed
as did not have a test.
 1.3 23-Jun-2014  joerg branches: 1.3.4;
Add aliases for the builtins used to implement C11/C++11 atomics.
 1.2 30-Nov-2013  joerg branches: 1.2.2; 1.2.4;
Use explicit form of register pair operations by specifying both.
 1.1 08-Nov-2013  matt Add support for the gcc __sync builtins.
Note that these need earmv6 or later to get the ldrex/strex instructions
 1.2.4.2 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.2.4.1 30-Nov-2013  yamt file sync_fetch_and_xor_8.S was added on branch yamt-pagecache on 2014-05-22 11:26:28 +0000
 1.2.2.1 10-Aug-2014  tls Rebase.
 1.3.4.2 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.3.4.1 23-Jun-2014  tls file sync_fetch_and_xor_8.S was added on branch tls-maxphys on 2014-08-19 23:45:12 +0000
 1.5.18.1 11-Aug-2021  martin Pull up following revision(s) (requested by skrll in ticket #1330):

common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.18
common/lib/libc/arch/arm/atomic/atomic_and_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.19
common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_inc_32.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_and_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_add_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_inc_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_cas_up.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_add_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_op_asm.h: revision 1.9
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.8
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_op_asm.h: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_and_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_cas_16.S: revision 1.3
common/lib/libc/arch/arm/atomic/sync_fetch_and_and_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_32.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_xor_32.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_cas_32.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_xor_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_add_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/sync_fetch_and_add_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_dec_64.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_add_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/sync_fetch_and_sub_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_dec_64.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_add_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_xor_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_add_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.14
common/lib/libc/arch/arm/atomic/atomic_or_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_nand_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_or_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_cas_8.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_inc_64.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_inc_64.S: revision 1.11
common/lib/libc/arch/arm/atomic/atomic_cas_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_dec_32.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_or_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_dec_32.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_xor_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_4.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/atomic_or_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_2.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_swap_64.S: revision 1.14
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_1.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_and_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_and_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_and_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.14

Give the thumb atomic ops a chance of working

s/ifdef _ARM_ARCH_6/if defined(_ARM_ARCH_6)/ for consistency. NFCI.

Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.

That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)
This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)
This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.5.16.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.5.16.1 21-Apr-2020  martin Sync with HEAD

RSS XML Feed