Define _ARM_ARCH_8 when __ARM_ARCH_8A (no trailing double underscore) as
it is defined by gcc.

__ARM_ARCH_8A__ (with trailing double underscore) seems to be a typo (or
maybe historical)

ARMv6KZ has been misspelled by GCC since forever, but clang only
provides the correct name. Support both.

branches: 1.1.4; 1.1.8; 1.1.30;
Add a method to test the compiler for things like LDREX availability,
LDRD availability, THUMB2, EABI.

branches: 1.2.16;
Use ${COPT} so it properly emits the right features doing MKCOMPAT

branches: 1.1.4; 1.1.6; 1.1.8;
Add a method to test the compiler for things like LDREX availability,
LDRD availability, THUMB2, EABI.

branches: 1.29.16;
Back out last change; fixed in the correct place.

Require ARMv5TE to assemble.

Move arm sync_* changes to Makefile.inc

Provide C++ 2011 <atomic> support functions for hppa and arm.

Move the and_{16,8}_nv sources into the right (libc only) block.

Provide __sync_and_and_fetch_2 and __sync_and_and_fetch_1 for pre-ARMv6,
they are needed for the C++ 2011 <atomic> stuff.

branches: 1.23.2;
Provide a basic implementation of __atomic_load_* and __atomic_store_*,
used by GCC and LLVM as backing for C11/C++11 atomics, if the hardware
is not known to have corresponding features. Include it on ARM as LLVM
and libc++ hit it when compiled for ARMv4.

branches: 1.22.2;
Don't export __sync* if _KERNEL || _STANDALONE are defined.
(except if _RUMPKERNEL is defined)

fix typo.

Add atomic_sub_64.S

Add atomic_cas_64 support for ARM EABI on V5TE and V5TEJ cpus.
(strd is atomic).

Missed one __sync_* op (or gcc4.8 does inline it, while 4.5 does not?)

Move the __sync_* ops added in the previous change to a libc-only section

Provide the missing __sync_* ops for earlier arm versions

Add _atomic_cas_16_up and _atomic_cas_8_up

Add support for the gcc __sync builtins.
Note that these need earmv6 or later to get the ldrex/strex instructions

Rework to allow thumb armv7 compilation.
Add atomic_simplelock.c for thumb

Only use MACHINE_ARCH if !_STANDALONE and CPPFLAGS/CFLAGS/CPUFLAGS don't
contain -mcpu or -march

Select ldrex/strex if ${MACHINE_ARCH:Mearmv[67]*}

branches: 1.10.2;
Add 64bit atomic ops for ARMv6+ (using ldrexd/strexd).

Actually use the assembly version of the atomic function if compiling
for armv6 or armv7 cpus. Use atomic_cas_ptr instead of _lock_cas so
we pick up the assembly version when it's used.

branches: 1.8.8; 1.8.12;
allow inclusion of atomic ops in librump

branches: 1.7.8;
Allow atomic ops to be built as part of libpthread.

Implement _atomic_cas_up() in assembly code as the compiler cannot be
trusted to generate fully restartable code sequences.

Addresses lib/38482 for ARM and m68000.

branches: 1.5.4;
Only build atomic ops for libkern/libc.

Enable the atomic ops in userspace.

branches: 1.3.4;
Use the CAS-based inc/dec variants, since these CPUs don't have atomic
add in hardware (does arm?).

Atomic ops for arm.

branches: 1.1.2;
file Makefile.inc was initially added on branch thorpej-atomic.

Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)

This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)

This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.

s/ifdef _ARM_ARCH_6/if defined(_ARM_ARCH_6)/ for consistency. NFCI.

branches: 1.3.4; 1.3.26; 1.3.28;
Add aliases for the builtins used to implement C11/C++11 atomics.

branches: 1.2.2; 1.2.4;
Don't export __sync* if _KERNEL || _STANDALONE are defined.
(except if _RUMPKERNEL is defined)

Add support for the gcc __sync builtins.
Note that these need earmv6 or later to get the ldrex/strex instructions

Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)

This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)

This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.

s/ifdef _ARM_ARCH_6/if defined(_ARM_ARCH_6)/ for consistency. NFCI.

branches: 1.8.24; 1.8.26;
Add aliases for the builtins used to implement C11/C++11 atomics.

branches: 1.7.2;
Try to hide the C runtime implementation specific __sync_* ops from librump,
to avoid duplicates.

Add support for the gcc __sync builtins.
Note that these need earmv6 or later to get the ldrex/strex instructions

Use foo{s} since it reduces the ifdefs for thumb

Make these under Thumb2

branches: 1.3.2;
Add dmb/dsb instructions as required by the armv7 arch man.

branches: 1.2.2; 1.2.4; 1.2.24;
Add assembly versions of atomic ops with ldrex/strex

branches: 1.1.2;
file atomic_add_32.S was initially added on branch thorpej-atomic.

Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)

This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)

This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.

s/ifdef _ARM_ARCH_6/if defined(_ARM_ARCH_6)/ for consistency. NFCI.

Whitespace

branches: 1.11.26; 1.11.28;
Don't export __sync* if _KERNEL || _STANDALONE are defined.
(except if _RUMPKERNEL is defined)

branches: 1.10.2;
Explicitly name the register pairs.

Add support for the gcc __sync builtins.
Note that these need earmv6 or later to get the ldrex/strex instructions

Push two registers to keep stack aligned.

Use foo{s} since it reduces the ifdefs for thumb

Make these under Thumb2

Never modify NLO/NHI (r2,r3) only LO/HI (r0,r1) so subsequent loops will work.

use push/pop

branches: 1.3.2; 1.3.4;
Correct copyright/fix comments.

Fix bas code, use ldr

branches: 1.1.2;
Add 64bit atomic ops for ARMv6+ (using ldrexd/strexd).

Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)

This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)

This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.

s/ifdef _ARM_ARCH_6/if defined(_ARM_ARCH_6)/ for consistency. NFCI.

branches: 1.3.4; 1.3.26; 1.3.28;
Add aliases for the builtins used to implement C11/C++11 atomics.

branches: 1.2.2; 1.2.4;
Don't export __sync* if _KERNEL || _STANDALONE are defined.
(except if _RUMPKERNEL is defined)

Add support for the gcc __sync builtins.
Note that these need earmv6 or later to get the ldrex/strex instructions

Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)

This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)

This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.

s/ifdef _ARM_ARCH_6/if defined(_ARM_ARCH_6)/ for consistency. NFCI.

branches: 1.3.4; 1.3.26; 1.3.28;
Add aliases for the builtins used to implement C11/C++11 atomics.

branches: 1.2.2; 1.2.4;
Don't export __sync* if _KERNEL || _STANDALONE are defined.
(except if _RUMPKERNEL is defined)

Add support for the gcc __sync builtins.
Note that these need earmv6 or later to get the ldrex/strex instructions

Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)

This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)

This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.

s/ifdef _ARM_ARCH_6/if defined(_ARM_ARCH_6)/ for consistency. NFCI.

branches: 1.8.24; 1.8.26;
Add aliases for the builtins used to implement C11/C++11 atomics.

branches: 1.7.2;
Try to hide the C runtime implementation specific __sync_* ops from librump,
to avoid duplicates.

Add support for the gcc __sync builtins.
Note that these need earmv6 or later to get the ldrex/strex instructions

Use foo{s} since it reduces the ifdefs for thumb

Make these under Thumb2

branches: 1.3.2;
Add dmb/dsb instructions as required by the armv7 arch man.

branches: 1.2.2; 1.2.4; 1.2.24;
Add assembly versions of atomic ops with ldrex/strex

branches: 1.1.2;
file atomic_and_32.S was initially added on branch thorpej-atomic.

Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)

This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)

This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.

s/ifdef _ARM_ARCH_6/if defined(_ARM_ARCH_6)/ for consistency. NFCI.

Whitespace

branches: 1.10.26; 1.10.28;
Don't export __sync* if _KERNEL || _STANDALONE are defined.
(except if _RUMPKERNEL is defined)

branches: 1.9.2;
Use explicit form of register pair operations by specifying both.

Add support for the gcc __sync builtins.
Note that these need earmv6 or later to get the ldrex/strex instructions

Push two registers to keep stack aligned.

Use foo{s} since it reduces the ifdefs for thumb

Make these under Thumb2

Never modify NLO/NHI (r2,r3) only LO/HI (r0,r1) so subsequent loops will work.

Use push/pop

branches: 1.2.2; 1.2.4;
Correct copyright/fix comments.

branches: 1.1.2;
Add 64bit atomic ops for ARMv6+ (using ldrexd/strexd).

Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)

This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)

This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.

s/ifdef _ARM_ARCH_6/if defined(_ARM_ARCH_6)/ for consistency. NFCI.

branches: 1.3.4; 1.3.26; 1.3.28;
Add aliases for the builtins used to implement C11/C++11 atomics.

branches: 1.2.2; 1.2.4;
Don't export __sync* if _KERNEL || _STANDALONE are defined.
(except if _RUMPKERNEL is defined)

Add support for the gcc __sync builtins.
Note that these need earmv6 or later to get the ldrex/strex instructions

Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)

This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)

This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.

branches: 1.2.4; 1.2.8; 1.2.30; 1.2.32;
Don't export __sync* if _KERNEL || _STANDALONE are defined.
(except if _RUMPKERNEL is defined)

Add support for the gcc __sync builtins.
Note that these need earmv6 or later to get the ldrex/strex instructions

Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)

This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)

This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.

branches: 1.7.26; 1.7.28;
Don't export __sync* if _KERNEL || _STANDALONE are defined.
(except if _RUMPKERNEL is defined)

Add support for the gcc __sync builtins.
Note that these need earmv6 or later to get the ldrex/strex instructions

Make these under Thumb2

_ARCH_ARM_6 -> _ARM_ARCH_6

branches: 1.3.2;
Add dmb/dsb instructions as required by the armv7 arch man.

branches: 1.2.2; 1.2.4; 1.2.12; 1.2.24;
Add assembly versions of atomic ops with ldrex/strex

branches: 1.1.2;
file atomic_cas_32.S was initially added on branch thorpej-atomic.

Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)

This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)

This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.

branches: 1.11.2;
Add some atomic_cas_64_ni aliases

branches: 1.10.26;
apcs-gnu only passes one register on the stack.
ldrd always loads little endian (low address, low register).

Don't export __sync* if _KERNEL || _STANDALONE are defined.
(except if _RUMPKERNEL is defined)

Fix #if/#endif nesting

Fix non-EABI loading of argument. Deal with endian issues.
Fixes PR/48635

Fetch value from correct stack location. Push an even number of registers
so ldrd won't fail.

branches: 1.5.2;
Use explicit form of register pair operations by specifying both.

Add support for the gcc __sync builtins.
Note that these need earmv6 or later to get the ldrex/strex instructions

Make these under Thumb2

branches: 1.2.2; 1.2.4;
_ARCH_ARM_6 -> _ARM_ARCH_6

branches: 1.1.2;
Add 64bit atomic ops for ARMv6+ (using ldrexd/strexd).

Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)

This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)

This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.

Whitespace

branches: 1.7.26; 1.7.28;
Don't export __sync* if _KERNEL || _STANDALONE are defined.
(except if _RUMPKERNEL is defined)

Add support for the gcc __sync builtins.
Note that these need earmv6 or later to get the ldrex/strex instructions

Make these under Thumb2

_ARCH_ARM_6 -> _ARM_ARCH_6

branches: 1.3.2;
Add dmb/dsb instructions as required by the armv7 arch man.

Actually use the assembly version of the atomic function if compiling
for armv6 or armv7 cpus. Use atomic_cas_ptr instead of _lock_cas so
we pick up the assembly version when it's used.

branches: 1.1.8; 1.1.12;
Add an atomic_cas_8 which uses ldrex/strex.

Give the thumb atomic ops a chance of working

branches: 1.7.26; 1.7.28;
Load new value from correct stack location in _atomic_cas_64_up

Add atomic_cas_64 support for ARM EABI on V5TE and V5TEJ cpus.
(strd is atomic).

Add _atomic_cas_16_up and _atomic_cas_8_up

Thumbify

Make these under Thumb2

branches: 1.2.2; 1.2.24;
enable profiling of assembly functions.

branches: 1.1.2; 1.1.4;
Implement _atomic_cas_up() in assembly code as the compiler cannot be
trusted to generate fully restartable code sequences.

Addresses lib/38482 for ARM and m68000.

Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)

This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)

This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.

s/ifdef _ARM_ARCH_6/if defined(_ARM_ARCH_6)/ for consistency. NFCI.

branches: 1.5.26; 1.5.28;
Use foo{s} since it reduces the ifdefs for thumb

Make these under Thumb2

branches: 1.3.2;
Add dmb/dsb instructions as required by the armv7 arch man.

branches: 1.2.2; 1.2.4; 1.2.24;
Add assembly versions of atomic ops with ldrex/strex

branches: 1.1.2;
file atomic_dec_32.S was initially added on branch thorpej-atomic.

Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)

This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)

This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.

s/ifdef _ARM_ARCH_6/if defined(_ARM_ARCH_6)/ for consistency. NFCI.

branches: 1.7.2; 1.7.28; 1.7.30;
Use explicit form of register pair operations by specifying both.

Use foo{s} since it reduces the ifdefs for thumb

Make these under Thumb2

Use r2 instead of ip

Use subs/sbc
Use push/pop
Don't adjust return for atomic_dec_64 since it's void and returns nothing

branches: 1.2.2; 1.2.4;
Correct copyright/fix comments.

branches: 1.1.2;
Add 64bit atomic ops for ARMv6+ (using ldrexd/strexd).

Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)

This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)

This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.

s/ifdef _ARM_ARCH_6/if defined(_ARM_ARCH_6)/ for consistency. NFCI.

branches: 1.7.26; 1.7.28;
Add support for the gcc __sync builtins.
Note that these need earmv6 or later to get the ldrex/strex instructions

Use foo{s} since it reduces the ifdefs for thumb

Make these under Thumb2

_ARCH_ARM_6 -> _ARM_ARCH_6

branches: 1.3.2;
Add dmb/dsb instructions as required by the armv7 arch man.

branches: 1.2.2; 1.2.4; 1.2.24;
Add assembly versions of atomic ops with ldrex/strex

branches: 1.1.2;
file atomic_inc_32.S was initially added on branch thorpej-atomic.

Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)

This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)

This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.

s/ifdef _ARM_ARCH_6/if defined(_ARM_ARCH_6)/ for consistency. NFCI.

branches: 1.9.2; 1.9.28; 1.9.30;
Use explicit form of register pair operations by specifying both.

Use foo{s} since it reduces the ifdefs for thumb

Make these under Thumb2

Use r2 instead of ip

Don't adjust return of atomic_inc_64 since it's void and doesn't return
anything.

Fix add -> adds

branches: 1.3.2; 1.3.4;
_ARCH_ARM_6 -> _ARM_ARCH_6

Correct copyright/fix comments.

branches: 1.1.2;
Add 64bit atomic ops for ARMv6+ (using ldrexd/strexd).

Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)

This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)

This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.

s/ifdef _ARM_ARCH_6/if defined(_ARM_ARCH_6)/ for consistency. NFCI.

branches: 1.3.16; 1.3.18;
Use gcc 4.4 and later operation for nand, i.e.
*ptr = ~(tmp & value) instead of *ptr = ~tmp & value

There was also another bug in sync_fetch_and_nand_8 which I've also fixed.

PR port-arm32/50513: Incorrect logic for atomic_nand_xx.S

branches: 1.2.4; 1.2.6; 1.2.8;
Don't export __sync* if _KERNEL || _STANDALONE are defined.
(except if _RUMPKERNEL is defined)

Add support for the gcc __sync builtins.
Note that these need earmv6 or later to get the ldrex/strex instructions

Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)

This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)

This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.

s/ifdef _ARM_ARCH_6/if defined(_ARM_ARCH_6)/ for consistency. NFCI.

branches: 1.3.16; 1.3.18;
Use gcc 4.4 and later operation for nand, i.e.
*ptr = ~(tmp & value) instead of *ptr = ~tmp & value

There was also another bug in sync_fetch_and_nand_8 which I've also fixed.

PR port-arm32/50513: Incorrect logic for atomic_nand_xx.S

branches: 1.2.4; 1.2.6; 1.2.8;
Don't export __sync* if _KERNEL || _STANDALONE are defined.
(except if _RUMPKERNEL is defined)

Add support for the gcc __sync builtins.
Note that these need earmv6 or later to get the ldrex/strex instructions

Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)

This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)

This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.

s/ifdef _ARM_ARCH_6/if defined(_ARM_ARCH_6)/ for consistency. NFCI.

Whitespace

branches: 1.4.16; 1.4.18;
Use gcc 4.4 and later operation for nand, i.e.
*ptr = ~(tmp & value) instead of *ptr = ~tmp & value

There was also another bug in sync_fetch_and_nand_8 which I've also fixed.

PR port-arm32/50513: Incorrect logic for atomic_nand_xx.S

branches: 1.3.4; 1.3.6; 1.3.8;
Don't export __sync* if _KERNEL || _STANDALONE are defined.
(except if _RUMPKERNEL is defined)

Use explicit form of register pair operations by specifying both.

Add support for the gcc __sync builtins.
Note that these need earmv6 or later to get the ldrex/strex instructions

Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)

This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)

This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.

s/ifdef _ARM_ARCH_6/if defined(_ARM_ARCH_6)/ for consistency. NFCI.

branches: 1.3.16; 1.3.18;
Use gcc 4.4 and later operation for nand, i.e.
*ptr = ~(tmp & value) instead of *ptr = ~tmp & value

There was also another bug in sync_fetch_and_nand_8 which I've also fixed.

PR port-arm32/50513: Incorrect logic for atomic_nand_xx.S

branches: 1.2.4; 1.2.6; 1.2.8;
Don't export __sync* if _KERNEL || _STANDALONE are defined.
(except if _RUMPKERNEL is defined)

Add support for the gcc __sync builtins.
Note that these need earmv6 or later to get the ldrex/strex instructions

#define<tab> consistency.

Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)

This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)

This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.

Traiing whitespace.

branches: 1.7.16; 1.7.18;
Do not build arm toolchain symbols in the rump kernel

These symbols will be provided at link time and will be
duplicate symbols in rump kernel and libc if defined.

Many have been fixed previously, but these were missed
as did not have a test.

Don't export __sync* if _KERNEL || _STANDALONE are defined.
(except if _RUMPKERNEL is defined)

Try to hide the C runtime implementation specific __sync_* ops from librump,
to avoid duplicates.

Add support for the gcc __sync builtins.
Note that these need earmv6 or later to get the ldrex/strex instructions

branches: 1.3.2;
Add 64bit atomic ops for ARMv6+ (using ldrexd/strexd).

branches: 1.2.2; 1.2.4; 1.2.12; 1.2.24;
Add assembly versions of atomic ops with ldrex/strex

branches: 1.1.2;
file atomic_op_asm.h was initially added on branch thorpej-atomic.

Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)

This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)

This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.

s/ifdef _ARM_ARCH_6/if defined(_ARM_ARCH_6)/ for consistency. NFCI.

branches: 1.3.4; 1.3.26; 1.3.28;
Add aliases for the builtins used to implement C11/C++11 atomics.

branches: 1.2.2; 1.2.4;
Don't export __sync* if _KERNEL || _STANDALONE are defined.
(except if _RUMPKERNEL is defined)

Add support for the gcc __sync builtins.
Note that these need earmv6 or later to get the ldrex/strex instructions

Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)

This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)

This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.

s/ifdef _ARM_ARCH_6/if defined(_ARM_ARCH_6)/ for consistency. NFCI.

branches: 1.8.24; 1.8.26;
Add aliases for the builtins used to implement C11/C++11 atomics.

branches: 1.7.2;
Try to hide the C runtime implementation specific __sync_* ops from librump,
to avoid duplicates.

Add support for the gcc __sync builtins.
Note that these need earmv6 or later to get the ldrex/strex instructions

Use foo{s} since it reduces the ifdefs for thumb

Make these under Thumb2

branches: 1.3.2;
Add dmb/dsb instructions as required by the armv7 arch man.

branches: 1.2.2; 1.2.4; 1.2.24;
Add assembly versions of atomic ops with ldrex/strex

branches: 1.1.2;
file atomic_or_32.S was initially added on branch thorpej-atomic.

Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)

This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)

This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.

s/ifdef _ARM_ARCH_6/if defined(_ARM_ARCH_6)/ for consistency. NFCI.

Whitespace

__sync_{,x}or_and_fetch_8 should return new value... make it so.

branches: 1.10.18; 1.10.26; 1.10.28;
Don't export __sync* if _KERNEL || _STANDALONE are defined.
(except if _RUMPKERNEL is defined)

branches: 1.9.2;
Use explicit form of register pair operations by specifying both.

Add support for the gcc __sync builtins.
Note that these need earmv6 or later to get the ldrex/strex instructions

Push two registers to keep stack aligned.

Use foo{s} since it reduces the ifdefs for thumb

Make these under Thumb2

Never modify NLO/NHI (r2,r3) only LO/HI (r0,r1) so subsequent loops will work.

use push/pop

branches: 1.2.2; 1.2.4;
Correct copyright/fix comments.

branches: 1.1.2;
Add 64bit atomic ops for ARMv6+ (using ldrexd/strexd).

Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)

This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)

This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.

s/ifdef _ARM_ARCH_6/if defined(_ARM_ARCH_6)/ for consistency. NFCI.

branches: 1.3.4; 1.3.26; 1.3.28;
Add aliases for the builtins used to implement C11/C++11 atomics.

branches: 1.2.2; 1.2.4;
Don't export __sync* if _KERNEL || _STANDALONE are defined.
(except if _RUMPKERNEL is defined)

Add support for the gcc __sync builtins.
Note that these need earmv6 or later to get the ldrex/strex instructions

branches: 1.2.4; 1.2.8; 1.2.30;
Add a hidden version for libpthread.

When compiling for Thumb1, the swp instruction is not availble. So this
stub is added to provide __cpu_simple_lock and __cpu_simple_lock_try via
thumb interwork support. It is compiled with -mno-thumb.

Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)

This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)

This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.

s/ifdef _ARM_ARCH_6/if defined(_ARM_ARCH_6)/ for consistency. NFCI.

Whitespace

branches: 1.2.4; 1.2.8; 1.2.30; 1.2.32;
Don't export __sync* if _KERNEL || _STANDALONE are defined.
(except if _RUMPKERNEL is defined)

Add support for the gcc __sync builtins.
Note that these need earmv6 or later to get the ldrex/strex instructions

Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)

This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)

This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.

s/ifdef _ARM_ARCH_6/if defined(_ARM_ARCH_6)/ for consistency. NFCI.

Add the appropriate memory barrier before the lock is cleared in
__sync_lock_release_{1,2,4,8}. That is, all reads and write for in inner
shareability domain before the lock clear store.

Trailing whitespace

Fix __sync_lock_release_4 to actually zeroise the whole 4bytes/32bits.

branches: 1.14.8; 1.14.16; 1.14.18;
Do not build arm toolchain symbols in the rump kernel

These symbols will be provided at link time and will be
duplicate symbols in rump kernel and libc if defined.

Many have been fixed previously, but these were missed
as did not have a test.

Use the right register in previous. Spotted by matt@

ARM ARM D7.3.2 - ensure all previous accesses are observed before
the lock is cleared

Add aliases for the C11/C++11 spelling of the CAS primitives.

branches: 1.10.2;
Don't export __sync* if _KERNEL || _STANDALONE are defined.
(except if _RUMPKERNEL is defined)

switch to unified syntax

Add support for the gcc __sync builtins.
Note that these need earmv6 or later to get the ldrex/strex instructions

Use foo{s} since it reduces the ifdefs for thumb

Make these under Thumb2

Change movsne to movnes for clang.

branches: 1.4.2;
Add dmb/dsb instructions as required by the armv7 arch man.

Actually use the assembly version of the atomic function if compiling
for armv6 or armv7 cpus. Use atomic_cas_ptr instead of _lock_cas so
we pick up the assembly version when it's used.

branches: 1.2.2; 1.2.4; 1.2.12; 1.2.24;
Add assembly versions of atomic ops with ldrex/strex

branches: 1.1.2;
file atomic_swap.S was initially added on branch thorpej-atomic.

Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)

This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)

This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.

s/ifdef _ARM_ARCH_6/if defined(_ARM_ARCH_6)/ for consistency. NFCI.

Add the appropriate memory barrier before the lock is cleared in
__sync_lock_release_{1,2,4,8}. That is, all reads and write for in inner
shareability domain before the lock clear store.

branches: 1.4.16; 1.4.18;
Do not build arm toolchain symbols in the rump kernel

These symbols will be provided at link time and will be
duplicate symbols in rump kernel and libc if defined.

Many have been fixed previously, but these were missed
as did not have a test.

branches: 1.3.4;
Add aliases for the C11/C++11 spelling of the CAS primitives.

branches: 1.2.2; 1.2.4;
Don't export __sync* if _KERNEL || _STANDALONE are defined.
(except if _RUMPKERNEL is defined)

Add support for the gcc __sync builtins.
Note that these need earmv6 or later to get the ldrex/strex instructions

s/overwriten/overwritten/ in comments.

Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)

This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)

This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.

Whitespace

Add the appropriate memory barrier before the lock is cleared in
__sync_lock_release_{1,2,4,8}. That is, all reads and write for in inner
shareability domain before the lock clear store.

Trailing whitespace

branches: 1.10.8; 1.10.16; 1.10.18;
Do not build arm toolchain symbols in the rump kernel

These symbols will be provided at link time and will be
duplicate symbols in rump kernel and libc if defined.

Many have been fixed previously, but these were missed
as did not have a test.

Add aliases for the C11/C++11 spelling of the CAS primitives.

branches: 1.8.2;
Don't export __sync* if _KERNEL || _STANDALONE are defined.
(except if _RUMPKERNEL is defined)

Use explicit form of register pair operations by specifying both.

Add support for the gcc __sync builtins.
Note that these need earmv6 or later to get the ldrex/strex instructions

Push two registers to keep stack aligned.

Make these under Thumb2

cmpne -> cmp
Use push/pop

branches: 1.2.2; 1.2.4;
Correct copyright/fix comments.

branches: 1.1.2;
Add 64bit atomic ops for ARMv6+ (using ldrexd/strexd).

Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)

This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)

This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.

s/ifdef _ARM_ARCH_6/if defined(_ARM_ARCH_6)/ for consistency. NFCI.

branches: 1.3.4; 1.3.26; 1.3.28;
Add aliases for the builtins used to implement C11/C++11 atomics.

branches: 1.2.2; 1.2.4;
Don't export __sync* if _KERNEL || _STANDALONE are defined.
(except if _RUMPKERNEL is defined)

Add support for the gcc __sync builtins.
Note that these need earmv6 or later to get the ldrex/strex instructions

Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)

This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)

This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.

s/ifdef _ARM_ARCH_6/if defined(_ARM_ARCH_6)/ for consistency. NFCI.

branches: 1.3.4; 1.3.26; 1.3.28;
Add aliases for the builtins used to implement C11/C++11 atomics.

branches: 1.2.2; 1.2.4;
Don't export __sync* if _KERNEL || _STANDALONE are defined.
(except if _RUMPKERNEL is defined)

Add support for the gcc __sync builtins.
Note that these need earmv6 or later to get the ldrex/strex instructions

Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)

This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)

This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.

s/ifdef _ARM_ARCH_6/if defined(_ARM_ARCH_6)/ for consistency. NFCI.

Whitespace

__sync_{,x}or_and_fetch_8 should return new value... make it so.

branches: 1.3.4; 1.3.8; 1.3.22; 1.3.30; 1.3.32;
Don't export __sync* if _KERNEL || _STANDALONE are defined.
(except if _RUMPKERNEL is defined)

Use explicit form of register pair operations by specifying both.

Add support for the gcc __sync builtins.
Note that these need earmv6 or later to get the ldrex/strex instructions

Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)

This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)

This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.

s/ifdef _ARM_ARCH_6/if defined(_ARM_ARCH_6)/ for consistency. NFCI.

branches: 1.3.4; 1.3.26; 1.3.28;
Add aliases for the builtins used to implement C11/C++11 atomics.

branches: 1.2.2; 1.2.4;
Don't export __sync* if _KERNEL || _STANDALONE are defined.
(except if _RUMPKERNEL is defined)

Add support for the gcc __sync builtins.
Note that these need earmv6 or later to get the ldrex/strex instructions

Introduce membar_acquire/release. Deprecate membar_enter/exit.

The names membar_enter/exit were unclear, and the documentation of
membar_enter has disagreed with the implementations on sparc,
powerpc, and even x86(!) for the entire time it has been in NetBSD.

The terms `acquire' and `release' are ubiquitous in the literature
today, and have been adopted in the C and C++ standards to mean
load-before-load/store and load/store-before-store, respectively,
which are exactly the orderings required by acquiring and releasing a
mutex, as well as other useful applications like decrementing a
reference count and then freeing the underlying object if it went to
zero.

Originally I proposed changing one word in the documentation for
membar_enter to make it load-before-load/store instead of
store-before-load/store, i.e., to make it an acquire barrier. I
proposed this on the grounds that

(a) all implementations guarantee load-before-load/store,
(b) some implementations fail to guarantee store-before-load/store,
and
(c) all uses in-tree assume load-before-load/store.

I verified parts (a) and (b) (except, for (a), powerpc didn't even
guarantee load-before-load/store -- isync isn't necessarily enough;
need lwsync in general -- but it _almost_ did, and it certainly didn't
guarantee store-before-load/store).

Part (c) might not be correct, however: under the mistaken assumption
that atomic-r/m/w then membar-w/rw is equivalent to atomic-r/m/w then
membar-r/rw, I only audited the cases of membar_enter that _aren't_
immediately after an atomic-r/m/w. All of those cases assume
load-before-load/store. But my assumption was wrong -- there are
cases of atomic-r/m/w then membar-w/rw that would be broken by
changing to atomic-r/m/w then membar-r/rw:

https://mail-index.netbsd.org/tech-kern/2022/03/29/msg028044.html

Furthermore, the name membar_enter has been adopted in other places
like OpenBSD where it actually does follow the documentation and
guarantee store-before-load/store, even if that order is not useful.
So the name membar_enter currently lives in a bad place where it
means either of two things -- r/rw or w/rw.

With this change, we deprecate membar_enter/exit, introduce
membar_acquire/release as better names for the useful pair (r/rw and
rw/w), and make sure the implementation of membar_enter guarantees
both what was documented _and_ what was implemented, making it an
alias for membar_sync.

While here, rework all of the membar_* definitions and aliases. The
new logic follows a rule to make it easier to audit:

membar_X is defined as an alias for membar_Y iff membar_X is
guaranteed by membar_Y.

The `no stronger than' relation is (the transitive closure of):

- membar_consumer (r/r) is guaranteed by membar_acquire (r/rw)
- membar_producer (w/w) is guaranteed by membar_release (rw/w)
- membar_acquire (r/rw) is guaranteed by membar_sync (rw/rw)
- membar_release (rw/w) is guaranteed by membar_sync (rw/rw)

And, for the deprecated membars:

- membar_enter (whether r/rw, w/rw, or rw/rw) is guaranteed by
membar_sync (rw/rw)
- membar_exit (rw/w) is guaranteed by membar_release (rw/w)

(membar_exit is identical to membar_release, but the name is
deprecated.)

Finally, while here, annotate some of the instructions with their
semantics. For powerpc, leave an essay with citations on the
unfortunate but -- as far as I can tell -- necessary decision to use
lwsync, not isync, for membar_acquire and membar_consumer.

Also add membar(3) and atomic(3) man page links.

Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)

This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)

This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.

s/ifdef _ARM_ARCH_6/if defined(_ARM_ARCH_6)/ for consistency. NFCI.

Improve the membar_ops barriers - no need to use dsb and wait for
completion. Also, we only to act on the inner shareability domain.

branches: 1.6.26; 1.6.28;
Ensure SBZ register is zero

Don't export __sync* if _KERNEL || _STANDALONE are defined.
(except if _RUMPKERNEL is defined)

Add support for the gcc __sync builtins.
Note that these need earmv6 or later to get the ldrex/strex instructions

branches: 1.3.2;
Actually use the assembly version of the atomic function if compiling
for armv6 or armv7 cpus. Use atomic_cas_ptr instead of _lock_cas so
we pick up the assembly version when it's used.

branches: 1.2.2; 1.2.4; 1.2.24;
Add assembly versions of atomic ops with ldrex/strex

branches: 1.1.2;
file membar_ops.S was initially added on branch thorpej-atomic.

Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)

This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)

This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.

branches: 1.3.16; 1.3.18;
Move arm sync_* changes to Makefile.inc

Do not build arm toolchain symbols in the rump kernel

These symbols will be provided at link time and will be
duplicate symbols in rump kernel and libc if defined.

Many have been fixed previously, but these were missed
as did not have a test.

branches: 1.1.4; 1.1.8;
Add support for the gcc __sync builtins.
Note that these need earmv6 or later to get the ldrex/strex instructions

Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)

This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)

This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.

branches: 1.3.16; 1.3.18;
Move arm sync_* changes to Makefile.inc

Do not build arm toolchain symbols in the rump kernel

These symbols will be provided at link time and will be
duplicate symbols in rump kernel and libc if defined.

Many have been fixed previously, but these were missed
as did not have a test.

branches: 1.1.4; 1.1.8;
Add support for the gcc __sync builtins.
Note that these need earmv6 or later to get the ldrex/strex instructions

Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)

This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)

This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.

branches: 1.3.16; 1.3.18;
Move arm sync_* changes to Makefile.inc

Do not build arm toolchain symbols in the rump kernel

These symbols will be provided at link time and will be
duplicate symbols in rump kernel and libc if defined.

Many have been fixed previously, but these were missed
as did not have a test.

branches: 1.1.4; 1.1.8;
Add support for the gcc __sync builtins.
Note that these need earmv6 or later to get the ldrex/strex instructions

Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)

This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)

This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.

branches: 1.4.16; 1.4.18;
Move arm sync_* changes to Makefile.inc

Do not build arm toolchain symbols in the rump kernel

These symbols will be provided at link time and will be
duplicate symbols in rump kernel and libc if defined.

Many have been fixed previously, but these were missed
as did not have a test.

branches: 1.2.4; 1.2.8;
Use explicit form of register pair operations by specifying both.

Add support for the gcc __sync builtins.
Note that these need earmv6 or later to get the ldrex/strex instructions

Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)

This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)

This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.

branches: 1.5.16; 1.5.18;
Move arm sync_* changes to Makefile.inc

Do not build arm toolchain symbols in the rump kernel

These symbols will be provided at link time and will be
duplicate symbols in rump kernel and libc if defined.

Many have been fixed previously, but these were missed
as did not have a test.

branches: 1.3.4;
Add aliases for the builtins used to implement C11/C++11 atomics.

branches: 1.2.2; 1.2.4;
Use explicit form of register pair operations by specifying both.

Add support for the gcc __sync builtins.
Note that these need earmv6 or later to get the ldrex/strex instructions

Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)

This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)

This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.

branches: 1.5.16; 1.5.18;
Move arm sync_* changes to Makefile.inc

Do not build arm toolchain symbols in the rump kernel

These symbols will be provided at link time and will be
duplicate symbols in rump kernel and libc if defined.

Many have been fixed previously, but these were missed
as did not have a test.

branches: 1.3.4;
Add aliases for the builtins used to implement C11/C++11 atomics.

branches: 1.2.2; 1.2.4;
Use explicit form of register pair operations by specifying both.

Add support for the gcc __sync builtins.
Note that these need earmv6 or later to get the ldrex/strex instructions

Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)

This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)

This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.

branches: 1.5.16; 1.5.18;
Use gcc 4.4 and later operation for nand, i.e.
*ptr = ~(tmp & value) instead of *ptr = ~tmp & value

There was also another bug in sync_fetch_and_nand_8 which I've also fixed.

PR port-arm32/50513: Incorrect logic for atomic_nand_xx.S

Move arm sync_* changes to Makefile.inc

Do not build arm toolchain symbols in the rump kernel

These symbols will be provided at link time and will be
duplicate symbols in rump kernel and libc if defined.

Many have been fixed previously, but these were missed
as did not have a test.

branches: 1.2.4; 1.2.6; 1.2.8;
Use explicit form of register pair operations by specifying both.

Add support for the gcc __sync builtins.
Note that these need earmv6 or later to get the ldrex/strex instructions

Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)

This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)

This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.

branches: 1.5.16; 1.5.18;
Move arm sync_* changes to Makefile.inc

Do not build arm toolchain symbols in the rump kernel

These symbols will be provided at link time and will be
duplicate symbols in rump kernel and libc if defined.

Many have been fixed previously, but these were missed
as did not have a test.

branches: 1.3.4;
Add aliases for the builtins used to implement C11/C++11 atomics.

branches: 1.2.2; 1.2.4;
Use explicit form of register pair operations by specifying both.

Add support for the gcc __sync builtins.
Note that these need earmv6 or later to get the ldrex/strex instructions

Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)

This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)

This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.

branches: 1.5.16; 1.5.18;
Move arm sync_* changes to Makefile.inc

Do not build arm toolchain symbols in the rump kernel

These symbols will be provided at link time and will be
duplicate symbols in rump kernel and libc if defined.

Many have been fixed previously, but these were missed
as did not have a test.

branches: 1.3.4;
Add aliases for the builtins used to implement C11/C++11 atomics.

branches: 1.2.2; 1.2.4;
Use explicit form of register pair operations by specifying both.

Add support for the gcc __sync builtins.
Note that these need earmv6 or later to get the ldrex/strex instructions

Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)

This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)

This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.

branches: 1.5.16; 1.5.18;
Move arm sync_* changes to Makefile.inc

Do not build arm toolchain symbols in the rump kernel

These symbols will be provided at link time and will be
duplicate symbols in rump kernel and libc if defined.

Many have been fixed previously, but these were missed
as did not have a test.

branches: 1.3.4;
Add aliases for the builtins used to implement C11/C++11 atomics.

branches: 1.2.2; 1.2.4;
Use explicit form of register pair operations by specifying both.

Add support for the gcc __sync builtins.
Note that these need earmv6 or later to get the ldrex/strex instructions

branches: 1.3.4; 1.3.28;
Kernel and standalone code uses panic() for broken code.

When using EABI, call __aeabi_{i,l}div0 when diving by 0.

branches: 1.1.4;
Add EABI support routines to raise SIGFPE on integer divide by 0.

branches: 1.3.4; 1.3.28;
Kernel and standalone code uses panic() for broken code.

When using EABI, call __aeabi_{i,l}div0 when diving by 0.

branches: 1.1.4;
Add EABI support routines to raise SIGFPE on integer divide by 0.

branches: 1.6.26;
Use STRONG_ALIAS
Add thumb variation

Use the armv6 rev/rev16 if armv6 or later

branches: 1.4.4; 1.4.24; 1.4.26;
Remove clause 3 and 4 from TNF licenses

branches: 1.3.18; 1.3.26;
Define htonl &co as alternative names to bswap only if little endian.
I'm not sure if idempotent versions need to be defined for big endian,
as the man page says in that case they are defined idempotent macros.

PR port-arm/32734

libc wants __bswapNN, kernel wants bswapNN. That was not accounted
for during the merge of kernel and libc versions. Fix to match
e.g. i386 code.

Merge libkern + libc common files. As requested by core.

arm bswap32: fix fatal typo in thumb code (PR 55854)

arm bswap32: Improve the comments showing the byte flow.

It's confusing to use 1-4 for bytes 1-4 and then 0 for literal zero,
so use a-d for bytes 1-4.

branches: 1.7.26;
Add thumb version
Use STRONG_ALIAS

Add some comments to illustrate what is actually happening.

Use the armv6 rev/rev16 if armv6 or later

branches: 1.4.4; 1.4.24; 1.4.26;
Remove clause 3 and 4 from TNF licenses

branches: 1.3.18; 1.3.26;
Define htonl &co as alternative names to bswap only if little endian.
I'm not sure if idempotent versions need to be defined for big endian,
as the man page says in that case they are defined idempotent macros.

PR port-arm/32734

libc wants __bswapNN, kernel wants bswapNN. That was not accounted
for during the merge of kernel and libc versions. Fix to match
e.g. i386 code.

Merge libkern + libc common files. As requested by core.

Include compiler-rt in libc, libm and libkern.

This is ARM only

Add support for __ARM_ARCH_EXT_IDIV__ which is set for those cores
with hardware divide instructions. Note that gcc 4.5.x doesn't support
this so this is just latent. gcc 4.7.x does.

When using EABI, call __aeabi_{i,l}div0 when diving by 0.

branches: 1.1.2; 1.1.4; 1.1.6; 1.1.8;
split udivsi3 and divsi3 to fix static linking. This could be done better.

Include compiler-rt in libc, libm and libkern.

Pass PICFLAGS down to cc-as-as and use __PIC__ to decide if it is small
vs big PIC mode. Retire -DPIC and -DBIGPIC.

Remove movw/movt due to linker problems.
Check for 0 divisor and __aeabi_idiv0 if needed (EABI && _LIBC only).

s/__libc/_libc/

Support using hwdiv instructions if those are available.
But only for EABI.

Use __ARM_EABI__ and new __UNWIND_TABLES__ to decide when to use .cfi ops

Don't include .cfi info if _KERNEL || _STANDALONE

Thumbify (and use .cfi ops)

Add support for __ARM_ARCH_EXT_IDIV__ which is set for those cores
with hardware divide instructions. Note that gcc 4.5.x doesn't support
this so this is just latent. gcc 4.7.x does.

When using EABI, call __aeabi_{i,l}div0 when diving by 0.

split udivsi3 and divsi3 to fix static linking. This could be done better.

Split out modsi3 and umodsi3 from the divsi3 file. This is so that
we don't get re-defined symbols in libc from libgcc in static linking.
Example: cc -pthread -static main-calls-pthread-create.c

branches: 1.2.2;
For __udivsi3 and __divsi3, add their EABI aliases as alternate entry
points.

branches: 1.1.18; 1.1.50; 1.1.52; 1.1.54;
Merge libkern + libc common files. As requested by core.

Include compiler-rt in libc, libm and libkern.

Only assemble if !__ARM_EABI__

Add support for __ARM_ARCH_EXT_IDIV__ which is set for those cores
with hardware divide instructions. Note that gcc 4.5.x doesn't support
this so this is just latent. gcc 4.7.x does.

branches: 1.2.2;
Optimize.

branches: 1.1.2; 1.1.4; 1.1.6;
Split out modsi3 and umodsi3 from the divsi3 file. This is so that
we don't get re-defined symbols in libc from libgcc in static linking.
Example: cc -pthread -static main-calls-pthread-create.c

branches: 1.2.2; 1.2.6; 1.2.36;
Don't need to include assym.h
Add a missing comma.

Add a routine to create an up to an 128 bitmask returned
in VFP/NEON q0 starting at the rightmost bit (bit 0).

Include compiler-rt in libc, libm and libkern.

Pass PICFLAGS down to cc-as-as and use __PIC__ to decide if it is small
vs big PIC mode. Retire -DPIC and -DBIGPIC.

Remove movw/movt due to linker problems.
Check for 0 divisor and __aeabi_idiv0 if needed (EABI && _LIBC only).

s/__libc/_libc/

Support using hwdiv instructions if those are available.
But only for EABI.

Use __ARM_EABI__ and new __UNWIND_TABLES__ to decide when to use .cfi ops

Don't include .cfi info if _KERNEL || _STANDALONE

thumbify
add .cfi ops (for thumb)

Add support for __ARM_ARCH_EXT_IDIV__ which is set for those cores
with hardware divide instructions. Note that gcc 4.5.x doesn't support
this so this is just latent. gcc 4.7.x does.

branches: 1.1.2; 1.1.4; 1.1.6; 1.1.8;
split udivsi3 and divsi3 to fix static linking. This could be done better.

Include compiler-rt in libc, libm and libkern.

Only assemble if !__ARM_EABI__

Add support for __ARM_ARCH_EXT_IDIV__ which is set for those cores
with hardware divide instructions. Note that gcc 4.5.x doesn't support
this so this is just latent. gcc 4.7.x does.

branches: 1.2.2;
Optimize.

branches: 1.1.2; 1.1.4; 1.1.6;
Split out modsi3 and umodsi3 from the divsi3 file. This is so that
we don't get re-defined symbols in libc from libgcc in static linking.
Example: cc -pthread -static main-calls-pthread-create.c

branches: 1.2.4; 1.2.36;
Change include to <arm/aeabi.h>

Add long long / unsigned long long comparision routines. These differ
from cmpdi2 in they return strcmp like values (-1, 0, 1).

branches: 1.13.24;
Make EHABI optional.

branches: 1.12.2;
Add cfi ops.
Thumbify

Use mvnge AHI, #0x80000000 instead of mvnge AHI, ALO, lsr #1

0x800000000 -> 0x80000000 (one too many zeroes)

Fix movlt

andlt -> movlt

Don't worry about divide by 0 for kernel or standalone

Don't use old syntax - it breaks the build. hi matt.

When using EABI, call __aeabi_{i,l}div0 when diving by 0.

Whitespace.

Fix logic inversion.

Fix calling of __qdivrem which the 3rd arg needs to passed on the stack.

branches: 1.1.2; 1.1.4;
Add another ARM EABI runtime routine for
combined quotient / remainder for signed long long.

branches: 1.2.4; 1.2.36;
Change include to <arm/aeabi.h>

Add long long / unsigned long long comparision routines. These differ
from cmpdi2 in they return strcmp like values (-1, 0, 1).

branches: 1.9.24;
Make EHABI optional.

branches: 1.8.2;
Fix a bug in the divby0 case.

Add .cfi ops
Thumbify

Don't worry about divide by 0 for kernel or standalone

When using EABI, call __aeabi_{i,l}div0 when diving by 0.

Pretty sure we don't want to change instruction set here. hi matt.

Fix calling of __qdivrem which the 3rd arg needs to passed on the stack.

branches: 1.2.2; 1.2.4;
Add RCSID.
Simplify.

Add a routine for __aeabi_uldivmod which is just a wrapper around __qdivrem
but returns the 64-bit dividend and remainder in r0-r3.

branches: 1.9.16;
It is just the __ffssi2 alias we do not want in rump kernel

Do not build arm toolchain symbols in the rump kernel

These symbols will be provided at link time and will be
duplicate symbols in rump kernel and libc if defined.

Many have been fixed previously, but these were missed
as did not have a test.

PR port-arm/48543: do provide __ffssi2 as strong alias (in case libgcc.a gets
not linked in) for now.
OK: skrll@

More (stylistic) whitespace.

Appease new gas

Add END() and clarify thumb/arm

Convert some more conditional instructions to unified syntax

branches: 1.2.2; 1.2.8;
Add a weak symbol definition for __ffssi2 so that __builtin_ffs will use
this if no strong defintion of __ffssi2 is available.

branches: 1.1.18;
Merge libkern + libc common files. As requested by core.

branches: 1.5.16;
Don't build the __eabi_ aliases in the arm rump kernel.

These are provided by libc in userspace, so leave out of kernel

Improve EABI handling of string functions. Most importantly, fix
__aeabi_memset, which has the arguments in the wrong order.

Add END()

Switch to unified syntax
use RET/RETc
use push/pop

branches: 1.1.18; 1.1.50; 1.1.56;
Merge libkern + libc common files. As requested by core.

branches: 1.8.16;
Add ARM EABI aliases for aligned arguments.

Don't build the __eabi_ aliases in the arm rump kernel.

These are provided by libc in userspace, so leave out of kernel

branches: 1.6.4;
Improve EABI handling of string functions. Most importantly, fix
__aeabi_memset, which has the arguments in the wrong order.

Unless we are using an XSCALE, default to the normal arm version of memcpy.

#include <arm/cdefs.h> to get _ARM_ARCH_DWORD_OK

Fix typo.

Change __XSCALE__ to _ARM_ARCH_DWORD_OK so that any cpu with dword load/store
can use it.

branches: 1.1.50; 1.1.54; 1.1.56;
Merge libkern + libc common files. As requested by core.

Fix two signed comparisons that were missed in the last patch.
Found be rillig@

Use unsigned comparisons for pointers and size_t values.

branches: 1.5.26; 1.5.40;
Improve EABI handling of string functions. Most importantly, fix
__aeabi_memset, which has the arguments in the wrong order.

Switch to unified syntax
use RET/RETc
use push/pop

Add aeabi strong aliases.

branches: 1.2.4; 1.2.24; 1.2.26;
Remove clause 3 and 4 from TNF licenses

branches: 1.1.18; 1.1.26;
Merge libkern + libc common files. As requested by core.

Fix various typos in comments.

branches: 1.2.6;
Use unsigned comparisons for pointers and size_t values.

branches: 1.1.2; 1.1.4; 1.1.8; 1.1.38; 1.1.52;
This is a working version of memcpy implemented using NEON instructions.
Still needs tuning as it is still about 15% than the non-NEON version.

Use unsigned comparisons for pointers and size_t values.

branches: 1.5.26; 1.5.40;
Write out register pairs for strd.

Add END(memcpy)

Switch to unified syntax
use RET/RETc
use push/pop

branches: 1.2.4; 1.2.34; 1.2.40;
Apply the patch, with some minor tweaks, supplied in PR/36513.
This prevents a possible prefetch past the end of the source buffer.

Note that the semantics of the pld instruction mean that it is unlikely
that this would have caused any problems except in very specific
circumstances in some types of device drivers.

branches: 1.1.6; 1.1.14;
Merge libkern + libc common files. As requested by core.

Use unsigned comparisons for pointers and size_t values.

branches: 1.10.12; 1.10.26;
Comment alignment. No functional change.

branches: 1.9.2; 1.9.4;
Don't build the __eabi_ aliases in the arm rump kernel.

These are provided by libc in userspace, so leave out of kernel