Home | History | Annotate | only in /src/common/lib/libc
History log of /src/common/lib/libc
RevisionDateAuthorComments
 1.21 27-Apr-2021  skrll Provide all the LSE operation fuctions. The use of LSE instructions is
currently disabled.
 1.20 30-Apr-2020  riastradh Rewrite entropy subsystem.

Primary goals:

1. Use cryptography primitives designed and vetted by cryptographers.
2. Be honest about entropy estimation.
3. Propagate full entropy as soon as possible.
4. Simplify the APIs.
5. Reduce overhead of rnd_add_data and cprng_strong.
6. Reduce side channels of HWRNG data and human input sources.
7. Improve visibility of operation with sysctl and event counters.

Caveat: rngtest is no longer used generically for RND_TYPE_RNG
rndsources. Hardware RNG devices should have hardware-specific
health tests. For example, checking for two repeated 256-bit outputs
works to detect AMD's 2019 RDRAND bug. Not all hardware RNGs are
necessarily designed to produce exactly uniform output.

ENTROPY POOL

- A Keccak sponge, with test vectors, replaces the old LFSR/SHA-1
kludge as the cryptographic primitive.

- `Entropy depletion' is available for testing purposes with a sysctl
knob kern.entropy.depletion; otherwise it is disabled, and once the
system reaches full entropy it is assumed to stay there as far as
modern cryptography is concerned.

- No `entropy estimation' based on sample values. Such `entropy
estimation' is a contradiction in terms, dishonest to users, and a
potential source of side channels. It is the responsibility of the
driver author to study the entropy of the process that generates
the samples.

- Per-CPU gathering pools avoid contention on a global queue.

- Entropy is occasionally consolidated into global pool -- as soon as
it's ready, if we've never reached full entropy, and with a rate
limit afterward. Operators can force consolidation now by running
sysctl -w kern.entropy.consolidate=1.

- rndsink(9) API has been replaced by an epoch counter which changes
whenever entropy is consolidated into the global pool.
. Usage: Cache entropy_epoch() when you seed. If entropy_epoch()
has changed when you're about to use whatever you seeded, reseed.
. Epoch is never zero, so initialize cache to 0 if you want to reseed
on first use.
. Epoch is -1 iff we have never reached full entropy -- in other
words, the old rnd_initial_entropy is (entropy_epoch() != -1) --
but it is better if you check for changes rather than for -1, so
that if the system estimated its own entropy incorrectly, entropy
consolidation has the opportunity to prevent future compromise.

- Sysctls and event counters provide operator visibility into what's
happening:
. kern.entropy.needed - bits of entropy short of full entropy
. kern.entropy.pending - bits known to be pending in per-CPU pools,
can be consolidated with sysctl -w kern.entropy.consolidate=1
. kern.entropy.epoch - number of times consolidation has happened,
never 0, and -1 iff we have never reached full entropy

CPRNG_STRONG

- A cprng_strong instance is now a collection of per-CPU NIST
Hash_DRBGs. There are only two in the system: user_cprng for
/dev/urandom and sysctl kern.?random, and kern_cprng for kernel
users which may need to operate in interrupt context up to IPL_VM.

(Calling cprng_strong in interrupt context does not strike me as a
particularly good idea, so I added an event counter to see whether
anything actually does.)

- Event counters provide operator visibility into when reseeding
happens.

INTEL RDRAND/RDSEED, VIA C3 RNG (CPU_RNG)

- Unwired for now; will be rewired in a subsequent commit.
 1.19 04-Jun-2019  hannken Move the basic part of XDR to common/include/rpc and common/lib/libc/rpc.

No functional change intended.
 1.18 03-Aug-2018  kamil Register a new directory in common/lib/libc/misc

Registe misc/ with ubsan.c.
 1.17 30-Nov-2017  riastradh branches: 1.17.2; 1.17.4;
Import SHA-3 code into libc and libkern.

No new public symbols in libc, but publishing the symbols is a simple
matter if/when we decide to do so.

Proposed on tech-kern and tech-userlevel with no objections:

https://mail-index.NetBSD.org/tech-kern/2017/11/11/msg022581.html
https://mail-index.NetBSD.org/tech-userlevel/2017/11/11/msg010968.html
 1.16 10-Aug-2014  matt MKCOMPAT fixes for when compat MACHINE_CPU != normal MACHINE_CPU
 1.15 29-Jan-2014  joerg Include compiler-rt in libc, libm and libkern.
 1.14 16-Jan-2014  joerg Include bsd.own.mk in case it hasn't been yet.
 1.13 15-Jan-2014  joerg Use the quad support from compiler-rt for MKLIBGCC=no.
 1.12 08-Jul-2012  rmind branches: 1.12.2;
Add MurmurHash2 -- a non-cryptographic hash function by Austin Appleby.
The code is taken from the upstream and is in the public domain.

OK christos@
 1.11 16-Jun-2011  joerg branches: 1.11.2;
Add support for size optimised versions of the assembler functions in
the small subdirectory. Provide more compact versions of the functions
used by bootxx in x86.
 1.10 26-Oct-2008  mrg branches: 1.10.14;
look for COMMON_MACHINE_ARCH before MACHINE_ARCH or MACHINE_CPU.
 1.9 15-Jun-2008  christos Hook into sys
 1.8 22-Jan-2008  nakayama branches: 1.8.4; 1.8.6;
Search files in arch/sparc64/atomic before arch/sparc/atomic in case of
sparc64 32-bit kernel.

Make sparc64 32-bit kernel works again.
 1.7 28-Nov-2007  ad Hook the atomic ops in. Right now they are only built into libkern, and
only if available.
 1.6 27-Oct-2006  christos branches: 1.6.2; 1.6.10; 1.6.16;
this is shared with the kernel now.
 1.5 27-Dec-2005  yamt lack of COMMON_ARCHDIR is not fatal.
 1.4 27-Dec-2005  yamt print a meaningful error rather than "Need an operator".
 1.3 21-Dec-2005  christos x86_64 uses i386 assembly files in standalone mode.
 1.2 20-Dec-2005  christos Include string in the cpp flags.
 1.1 20-Dec-2005  christos Merge libkern + libc common files. As requested by core.
 1.6.16.2 23-Mar-2008  matt sync with HEAD
 1.6.16.1 09-Jan-2008  matt sync with HEAD
 1.6.10.1 17-Apr-2007  thorpej Add build glue for i386 atomic ops.
 1.6.2.2 27-Oct-2006  christos this is shared with the kernel now.
 1.6.2.1 27-Oct-2006  christos file Makefile.inc was added on branch newlock2 on 2006-10-27 21:23:16 +0000
 1.8.6.1 23-Jun-2008  wrstuden Sync w/ -current. 34 merge conflicts to follow.
 1.8.4.1 17-Jun-2008  yamt sync with head.
 1.10.14.1 23-Jun-2011  cherry Catchup with rmind-uvmplock merge.
 1.11.2.2 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.11.2.1 30-Oct-2012  yamt sync with head
 1.12.2.1 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.17.4.3 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.17.4.2 21-Apr-2020  martin Sync with HEAD
 1.17.4.1 10-Jun-2019  christos Sync with HEAD
 1.17.2.1 06-Sep-2018  pgoyette Sync with HEAD

Resolve a couple of conflicts (result of the uimin/uimax changes)
 1.5 06-Aug-2022  riastradh aarch64: Implement __aarch64_casN_sync.

gcc generates calls to this symbol in programs that use
__sync_*_compare_and_swap, which require full sequential consistency
barriers, including store-before-load ordering on both sides of the
atomic; none of the release/acquire operations guarantee that, so we
have to insert explicit DMB instructions.

Note: gcc's own definition omits some of the DMB instructions, but I
can't prove that it's correct that way -- stores preceding the CAS
must complete before the load part of the CAS, and the store part of
the CAS must complete before loads following the CAS. Maybe there's
some way to prove that one of these orderings is guaranteed some
other way than a DMB but I'm not seeing it, and store-before-load
ordering is hard to understand.

Patch by skrll@ based on a patch by mrg@, soliloquy in commit message
by me.
 1.4 27-Apr-2021  skrll Provide all the LSE operation fuctions. The use of LSE instructions is
currently disabled.
 1.3 21-Apr-2021  skrll Do previous differently as the API is different.
 1.2 21-Apr-2021  skrll Provide some more operations that are part of compiler lse.S. This is
incomplete, but at least covers all the atomic_swap ops and allows the
aa64 kernel to link with gcc 10.
 1.1 10-Aug-2014  matt branches: 1.1.4; 1.1.26;
Preliminary files for AARCH64 (64-bit ARM) support.
Enough for a distribution build.
 1.1.26.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.1.26.1 21-Apr-2020  martin Sync with HEAD
 1.1.4.2 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.1.4.1 10-Aug-2014  tls file Makefile.inc was added on branch tls-maxphys on 2014-08-19 23:45:11 +0000
 1.7 06-Aug-2022  riastradh aarch64: Implement __aarch64_casN_sync.

gcc generates calls to this symbol in programs that use
__sync_*_compare_and_swap, which require full sequential consistency
barriers, including store-before-load ordering on both sides of the
atomic; none of the release/acquire operations guarantee that, so we
have to insert explicit DMB instructions.

Note: gcc's own definition omits some of the DMB instructions, but I
can't prove that it's correct that way -- stores preceding the CAS
must complete before the load part of the CAS, and the store part of
the CAS must complete before loads following the CAS. Maybe there's
some way to prove that one of these orderings is guaranteed some
other way than a DMB but I'm not seeing it, and store-before-load
ordering is hard to understand.

Patch by skrll@ based on a patch by mrg@, soliloquy in commit message
by me.
 1.6 23-Jul-2022  skrll whitespace
 1.5 18-Jun-2022  skrll be consistent about comparing loaded value against expected old value
register ordering
 1.4 18-Jun-2022  skrll Fix some register usage
 1.3 16-Jun-2022  skrll remove stray 'w'
 1.2 08-Aug-2021  skrll Whitespace
 1.1 27-Apr-2021  skrll Provide all the LSE operation fuctions. The use of LSE instructions is
currently disabled.
 1.2 27-Apr-2021  skrll Provide all the LSE operation fuctions. The use of LSE instructions is
currently disabled.
 1.1 21-Apr-2021  skrll Do previous differently as the API is different.
 1.2 27-Apr-2021  skrll Provide all the LSE operation fuctions. The use of LSE instructions is
currently disabled.
 1.1 21-Apr-2021  skrll Do previous differently as the API is different.
 1.2 27-Apr-2021  skrll Provide all the LSE operation fuctions. The use of LSE instructions is
currently disabled.
 1.1 21-Apr-2021  skrll Do previous differently as the API is different.
 1.2 27-Apr-2021  skrll Provide all the LSE operation fuctions. The use of LSE instructions is
currently disabled.
 1.1 21-Apr-2021  skrll Do previous differently as the API is different.
 1.2 27-Apr-2021  skrll Provide all the LSE operation fuctions. The use of LSE instructions is
currently disabled.
 1.1 21-Apr-2021  skrll Do previous differently as the API is different.
 1.2 27-Apr-2021  skrll Provide all the LSE operation fuctions. The use of LSE instructions is
currently disabled.
 1.1 21-Apr-2021  skrll Do previous differently as the API is different.
 1.2 27-Apr-2021  skrll Provide all the LSE operation fuctions. The use of LSE instructions is
currently disabled.
 1.1 21-Apr-2021  skrll Do previous differently as the API is different.
 1.2 27-Apr-2021  skrll Provide all the LSE operation fuctions. The use of LSE instructions is
currently disabled.
 1.1 21-Apr-2021  skrll Do previous differently as the API is different.
 1.2 27-Apr-2021  skrll Provide all the LSE operation fuctions. The use of LSE instructions is
currently disabled.
 1.1 21-Apr-2021  skrll Do previous differently as the API is different.
 1.2 27-Apr-2021  skrll Provide all the LSE operation fuctions. The use of LSE instructions is
currently disabled.
 1.1 21-Apr-2021  skrll Do previous differently as the API is different.
 1.2 27-Apr-2021  skrll Provide all the LSE operation fuctions. The use of LSE instructions is
currently disabled.
 1.1 21-Apr-2021  skrll Do previous differently as the API is different.
 1.2 27-Apr-2021  skrll Provide all the LSE operation fuctions. The use of LSE instructions is
currently disabled.
 1.1 21-Apr-2021  skrll Do previous differently as the API is different.
 1.2 27-Apr-2021  skrll Provide all the LSE operation fuctions. The use of LSE instructions is
currently disabled.
 1.1 21-Apr-2021  skrll Do previous differently as the API is different.
 1.2 27-Apr-2021  skrll Provide all the LSE operation fuctions. The use of LSE instructions is
currently disabled.
 1.1 21-Apr-2021  skrll Do previous differently as the API is different.
 1.2 27-Apr-2021  skrll Provide all the LSE operation fuctions. The use of LSE instructions is
currently disabled.
 1.1 21-Apr-2021  skrll Do previous differently as the API is different.
 1.2 27-Apr-2021  skrll Provide all the LSE operation fuctions. The use of LSE instructions is
currently disabled.
 1.1 21-Apr-2021  skrll Do previous differently as the API is different.
 1.2 29-Jul-2021  skrll As we're providing the legacy gcc __sync built-in functions for atomic
memory access we might as well get the memory barriers right...
From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)

This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)

This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.1 10-Aug-2014  matt branches: 1.1.4; 1.1.26; 1.1.28;
Preliminary files for AARCH64 (64-bit ARM) support.
Enough for a distribution build.
 1.1.28.1 11-Aug-2021  martin Pull up following revision(s) (requested by skrll in ticket #1331):

common/lib/libc/arch/aarch64/atomic/atomic_add_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_sub_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_sub_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_64.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_inc_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_or_64.S: revision 1.3
common/lib/libc/arch/aarch64/atomic/atomic_and_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_xor_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_and_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_add_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_op_asm.h: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_swap_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_op_asm.h: revision 1.6
common/lib/libc/arch/aarch64/atomic/atomic_cas_64.S: revision 1.4
common/lib/libc/arch/aarch64/atomic/atomic_nand_8.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_sub_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_64.S: revision 1.6
common/lib/libc/arch/aarch64/atomic/atomic_swap_8.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_sub_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_xor_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_32.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_or_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_64.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_swap_16.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_or_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_or_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_xor_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_16.S: revision 1.4
common/lib/libc/arch/aarch64/atomic/atomic_add_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_xor_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_dec_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_32.S: revision 1.4
common/lib/libc/arch/aarch64/atomic/atomic_add_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_8.S: revision 1.4
common/lib/libc/arch/aarch64/atomic/membar_ops.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_16.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_nand_32.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_inc_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_dec_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_and_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_and_64.S: revision 1.2

Part I of ad@'s performance improvements for aarch64
- Remove memory barriers from the atomic ops. I don't understand why thos=
e
are there. Is it some architectural thing, or for a CPU bug, or just
over-caution maybe? They're not needed for correctness.
- Have unlikely conditional branches go forwards to help the static branch
predictor.

Remove memory barriers from the atomic ops macros in the same way as was
done for the other atomic ops earlier.

Use the correct barriers - all of membar_{sync,producer,consumer} have
less scope than before.

LGTM from riastradh

As we're providing the legacy gcc __sync built-in functions for atomic
memory access we might as well get the memory barriers right...

From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)
This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)
This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.1.26.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.1.26.1 21-Apr-2020  martin Sync with HEAD
 1.1.4.2 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.1.4.1 10-Aug-2014  tls file atomic_add_16.S was added on branch tls-maxphys on 2014-08-19 23:45:11 +0000
 1.2 29-Jul-2021  skrll As we're providing the legacy gcc __sync built-in functions for atomic
memory access we might as well get the memory barriers right...
From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)

This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)

This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.1 10-Aug-2014  matt branches: 1.1.4; 1.1.26; 1.1.28;
Preliminary files for AARCH64 (64-bit ARM) support.
Enough for a distribution build.
 1.1.28.1 11-Aug-2021  martin Pull up following revision(s) (requested by skrll in ticket #1331):

common/lib/libc/arch/aarch64/atomic/atomic_add_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_sub_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_sub_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_64.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_inc_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_or_64.S: revision 1.3
common/lib/libc/arch/aarch64/atomic/atomic_and_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_xor_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_and_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_add_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_op_asm.h: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_swap_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_op_asm.h: revision 1.6
common/lib/libc/arch/aarch64/atomic/atomic_cas_64.S: revision 1.4
common/lib/libc/arch/aarch64/atomic/atomic_nand_8.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_sub_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_64.S: revision 1.6
common/lib/libc/arch/aarch64/atomic/atomic_swap_8.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_sub_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_xor_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_32.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_or_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_64.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_swap_16.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_or_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_or_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_xor_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_16.S: revision 1.4
common/lib/libc/arch/aarch64/atomic/atomic_add_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_xor_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_dec_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_32.S: revision 1.4
common/lib/libc/arch/aarch64/atomic/atomic_add_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_8.S: revision 1.4
common/lib/libc/arch/aarch64/atomic/membar_ops.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_16.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_nand_32.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_inc_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_dec_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_and_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_and_64.S: revision 1.2

Part I of ad@'s performance improvements for aarch64
- Remove memory barriers from the atomic ops. I don't understand why thos=
e
are there. Is it some architectural thing, or for a CPU bug, or just
over-caution maybe? They're not needed for correctness.
- Have unlikely conditional branches go forwards to help the static branch
predictor.

Remove memory barriers from the atomic ops macros in the same way as was
done for the other atomic ops earlier.

Use the correct barriers - all of membar_{sync,producer,consumer} have
less scope than before.

LGTM from riastradh

As we're providing the legacy gcc __sync built-in functions for atomic
memory access we might as well get the memory barriers right...

From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)
This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)
This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.1.26.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.1.26.1 21-Apr-2020  martin Sync with HEAD
 1.1.4.2 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.1.4.1 10-Aug-2014  tls file atomic_add_32.S was added on branch tls-maxphys on 2014-08-19 23:45:11 +0000
 1.2 29-Jul-2021  skrll As we're providing the legacy gcc __sync built-in functions for atomic
memory access we might as well get the memory barriers right...
From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)

This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)

This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.1 10-Aug-2014  matt branches: 1.1.4; 1.1.26; 1.1.28;
Preliminary files for AARCH64 (64-bit ARM) support.
Enough for a distribution build.
 1.1.28.1 11-Aug-2021  martin Pull up following revision(s) (requested by skrll in ticket #1331):

common/lib/libc/arch/aarch64/atomic/atomic_add_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_sub_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_sub_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_64.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_inc_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_or_64.S: revision 1.3
common/lib/libc/arch/aarch64/atomic/atomic_and_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_xor_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_and_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_add_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_op_asm.h: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_swap_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_op_asm.h: revision 1.6
common/lib/libc/arch/aarch64/atomic/atomic_cas_64.S: revision 1.4
common/lib/libc/arch/aarch64/atomic/atomic_nand_8.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_sub_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_64.S: revision 1.6
common/lib/libc/arch/aarch64/atomic/atomic_swap_8.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_sub_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_xor_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_32.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_or_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_64.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_swap_16.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_or_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_or_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_xor_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_16.S: revision 1.4
common/lib/libc/arch/aarch64/atomic/atomic_add_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_xor_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_dec_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_32.S: revision 1.4
common/lib/libc/arch/aarch64/atomic/atomic_add_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_8.S: revision 1.4
common/lib/libc/arch/aarch64/atomic/membar_ops.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_16.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_nand_32.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_inc_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_dec_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_and_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_and_64.S: revision 1.2

Part I of ad@'s performance improvements for aarch64
- Remove memory barriers from the atomic ops. I don't understand why thos=
e
are there. Is it some architectural thing, or for a CPU bug, or just
over-caution maybe? They're not needed for correctness.
- Have unlikely conditional branches go forwards to help the static branch
predictor.

Remove memory barriers from the atomic ops macros in the same way as was
done for the other atomic ops earlier.

Use the correct barriers - all of membar_{sync,producer,consumer} have
less scope than before.

LGTM from riastradh

As we're providing the legacy gcc __sync built-in functions for atomic
memory access we might as well get the memory barriers right...

From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)
This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)
This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.1.26.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.1.26.1 21-Apr-2020  martin Sync with HEAD
 1.1.4.2 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.1.4.1 10-Aug-2014  tls file atomic_add_64.S was added on branch tls-maxphys on 2014-08-19 23:45:11 +0000
 1.2 29-Jul-2021  skrll As we're providing the legacy gcc __sync built-in functions for atomic
memory access we might as well get the memory barriers right...
From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)

This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)

This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.1 10-Aug-2014  matt branches: 1.1.4; 1.1.26; 1.1.28;
Preliminary files for AARCH64 (64-bit ARM) support.
Enough for a distribution build.
 1.1.28.1 11-Aug-2021  martin Pull up following revision(s) (requested by skrll in ticket #1331):

common/lib/libc/arch/aarch64/atomic/atomic_add_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_sub_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_sub_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_64.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_inc_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_or_64.S: revision 1.3
common/lib/libc/arch/aarch64/atomic/atomic_and_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_xor_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_and_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_add_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_op_asm.h: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_swap_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_op_asm.h: revision 1.6
common/lib/libc/arch/aarch64/atomic/atomic_cas_64.S: revision 1.4
common/lib/libc/arch/aarch64/atomic/atomic_nand_8.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_sub_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_64.S: revision 1.6
common/lib/libc/arch/aarch64/atomic/atomic_swap_8.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_sub_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_xor_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_32.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_or_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_64.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_swap_16.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_or_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_or_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_xor_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_16.S: revision 1.4
common/lib/libc/arch/aarch64/atomic/atomic_add_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_xor_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_dec_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_32.S: revision 1.4
common/lib/libc/arch/aarch64/atomic/atomic_add_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_8.S: revision 1.4
common/lib/libc/arch/aarch64/atomic/membar_ops.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_16.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_nand_32.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_inc_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_dec_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_and_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_and_64.S: revision 1.2

Part I of ad@'s performance improvements for aarch64
- Remove memory barriers from the atomic ops. I don't understand why thos=
e
are there. Is it some architectural thing, or for a CPU bug, or just
over-caution maybe? They're not needed for correctness.
- Have unlikely conditional branches go forwards to help the static branch
predictor.

Remove memory barriers from the atomic ops macros in the same way as was
done for the other atomic ops earlier.

Use the correct barriers - all of membar_{sync,producer,consumer} have
less scope than before.

LGTM from riastradh

As we're providing the legacy gcc __sync built-in functions for atomic
memory access we might as well get the memory barriers right...

From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)
This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)
This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.1.26.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.1.26.1 21-Apr-2020  martin Sync with HEAD
 1.1.4.2 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.1.4.1 10-Aug-2014  tls file atomic_add_8.S was added on branch tls-maxphys on 2014-08-19 23:45:11 +0000
 1.2 29-Jul-2021  skrll As we're providing the legacy gcc __sync built-in functions for atomic
memory access we might as well get the memory barriers right...
From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)

This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)

This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.1 10-Aug-2014  matt branches: 1.1.4; 1.1.26; 1.1.28;
Preliminary files for AARCH64 (64-bit ARM) support.
Enough for a distribution build.
 1.1.28.1 11-Aug-2021  martin Pull up following revision(s) (requested by skrll in ticket #1331):

common/lib/libc/arch/aarch64/atomic/atomic_add_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_sub_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_sub_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_64.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_inc_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_or_64.S: revision 1.3
common/lib/libc/arch/aarch64/atomic/atomic_and_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_xor_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_and_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_add_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_op_asm.h: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_swap_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_op_asm.h: revision 1.6
common/lib/libc/arch/aarch64/atomic/atomic_cas_64.S: revision 1.4
common/lib/libc/arch/aarch64/atomic/atomic_nand_8.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_sub_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_64.S: revision 1.6
common/lib/libc/arch/aarch64/atomic/atomic_swap_8.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_sub_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_xor_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_32.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_or_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_64.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_swap_16.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_or_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_or_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_xor_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_16.S: revision 1.4
common/lib/libc/arch/aarch64/atomic/atomic_add_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_xor_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_dec_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_32.S: revision 1.4
common/lib/libc/arch/aarch64/atomic/atomic_add_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_8.S: revision 1.4
common/lib/libc/arch/aarch64/atomic/membar_ops.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_16.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_nand_32.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_inc_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_dec_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_and_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_and_64.S: revision 1.2

Part I of ad@'s performance improvements for aarch64
- Remove memory barriers from the atomic ops. I don't understand why thos=
e
are there. Is it some architectural thing, or for a CPU bug, or just
over-caution maybe? They're not needed for correctness.
- Have unlikely conditional branches go forwards to help the static branch
predictor.

Remove memory barriers from the atomic ops macros in the same way as was
done for the other atomic ops earlier.

Use the correct barriers - all of membar_{sync,producer,consumer} have
less scope than before.

LGTM from riastradh

As we're providing the legacy gcc __sync built-in functions for atomic
memory access we might as well get the memory barriers right...

From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)
This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)
This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.1.26.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.1.26.1 21-Apr-2020  martin Sync with HEAD
 1.1.4.2 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.1.4.1 10-Aug-2014  tls file atomic_and_16.S was added on branch tls-maxphys on 2014-08-19 23:45:11 +0000
 1.2 29-Jul-2021  skrll As we're providing the legacy gcc __sync built-in functions for atomic
memory access we might as well get the memory barriers right...
From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)

This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)

This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.1 10-Aug-2014  matt branches: 1.1.4; 1.1.26; 1.1.28;
Preliminary files for AARCH64 (64-bit ARM) support.
Enough for a distribution build.
 1.1.28.1 11-Aug-2021  martin Pull up following revision(s) (requested by skrll in ticket #1331):

common/lib/libc/arch/aarch64/atomic/atomic_add_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_sub_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_sub_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_64.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_inc_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_or_64.S: revision 1.3
common/lib/libc/arch/aarch64/atomic/atomic_and_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_xor_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_and_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_add_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_op_asm.h: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_swap_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_op_asm.h: revision 1.6
common/lib/libc/arch/aarch64/atomic/atomic_cas_64.S: revision 1.4
common/lib/libc/arch/aarch64/atomic/atomic_nand_8.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_sub_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_64.S: revision 1.6
common/lib/libc/arch/aarch64/atomic/atomic_swap_8.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_sub_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_xor_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_32.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_or_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_64.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_swap_16.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_or_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_or_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_xor_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_16.S: revision 1.4
common/lib/libc/arch/aarch64/atomic/atomic_add_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_xor_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_dec_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_32.S: revision 1.4
common/lib/libc/arch/aarch64/atomic/atomic_add_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_8.S: revision 1.4
common/lib/libc/arch/aarch64/atomic/membar_ops.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_16.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_nand_32.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_inc_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_dec_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_and_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_and_64.S: revision 1.2

Part I of ad@'s performance improvements for aarch64
- Remove memory barriers from the atomic ops. I don't understand why thos=
e
are there. Is it some architectural thing, or for a CPU bug, or just
over-caution maybe? They're not needed for correctness.
- Have unlikely conditional branches go forwards to help the static branch
predictor.

Remove memory barriers from the atomic ops macros in the same way as was
done for the other atomic ops earlier.

Use the correct barriers - all of membar_{sync,producer,consumer} have
less scope than before.

LGTM from riastradh

As we're providing the legacy gcc __sync built-in functions for atomic
memory access we might as well get the memory barriers right...

From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)
This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)
This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.1.26.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.1.26.1 21-Apr-2020  martin Sync with HEAD
 1.1.4.2 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.1.4.1 10-Aug-2014  tls file atomic_and_32.S was added on branch tls-maxphys on 2014-08-19 23:45:11 +0000
 1.2 29-Jul-2021  skrll As we're providing the legacy gcc __sync built-in functions for atomic
memory access we might as well get the memory barriers right...
From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)

This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)

This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.1 10-Aug-2014  matt branches: 1.1.4; 1.1.26; 1.1.28;
Preliminary files for AARCH64 (64-bit ARM) support.
Enough for a distribution build.
 1.1.28.1 11-Aug-2021  martin Pull up following revision(s) (requested by skrll in ticket #1331):

common/lib/libc/arch/aarch64/atomic/atomic_add_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_sub_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_sub_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_64.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_inc_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_or_64.S: revision 1.3
common/lib/libc/arch/aarch64/atomic/atomic_and_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_xor_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_and_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_add_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_op_asm.h: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_swap_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_op_asm.h: revision 1.6
common/lib/libc/arch/aarch64/atomic/atomic_cas_64.S: revision 1.4
common/lib/libc/arch/aarch64/atomic/atomic_nand_8.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_sub_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_64.S: revision 1.6
common/lib/libc/arch/aarch64/atomic/atomic_swap_8.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_sub_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_xor_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_32.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_or_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_64.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_swap_16.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_or_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_or_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_xor_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_16.S: revision 1.4
common/lib/libc/arch/aarch64/atomic/atomic_add_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_xor_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_dec_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_32.S: revision 1.4
common/lib/libc/arch/aarch64/atomic/atomic_add_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_8.S: revision 1.4
common/lib/libc/arch/aarch64/atomic/membar_ops.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_16.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_nand_32.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_inc_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_dec_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_and_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_and_64.S: revision 1.2

Part I of ad@'s performance improvements for aarch64
- Remove memory barriers from the atomic ops. I don't understand why thos=
e
are there. Is it some architectural thing, or for a CPU bug, or just
over-caution maybe? They're not needed for correctness.
- Have unlikely conditional branches go forwards to help the static branch
predictor.

Remove memory barriers from the atomic ops macros in the same way as was
done for the other atomic ops earlier.

Use the correct barriers - all of membar_{sync,producer,consumer} have
less scope than before.

LGTM from riastradh

As we're providing the legacy gcc __sync built-in functions for atomic
memory access we might as well get the memory barriers right...

From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)
This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)
This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.1.26.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.1.26.1 21-Apr-2020  martin Sync with HEAD
 1.1.4.2 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.1.4.1 10-Aug-2014  tls file atomic_and_64.S was added on branch tls-maxphys on 2014-08-19 23:45:11 +0000
 1.2 29-Jul-2021  skrll As we're providing the legacy gcc __sync built-in functions for atomic
memory access we might as well get the memory barriers right...
From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)

This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)

This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.1 10-Aug-2014  matt branches: 1.1.4; 1.1.26; 1.1.28;
Preliminary files for AARCH64 (64-bit ARM) support.
Enough for a distribution build.
 1.1.28.1 11-Aug-2021  martin Pull up following revision(s) (requested by skrll in ticket #1331):

common/lib/libc/arch/aarch64/atomic/atomic_add_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_sub_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_sub_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_64.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_inc_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_or_64.S: revision 1.3
common/lib/libc/arch/aarch64/atomic/atomic_and_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_xor_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_and_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_add_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_op_asm.h: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_swap_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_op_asm.h: revision 1.6
common/lib/libc/arch/aarch64/atomic/atomic_cas_64.S: revision 1.4
common/lib/libc/arch/aarch64/atomic/atomic_nand_8.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_sub_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_64.S: revision 1.6
common/lib/libc/arch/aarch64/atomic/atomic_swap_8.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_sub_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_xor_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_32.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_or_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_64.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_swap_16.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_or_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_or_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_xor_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_16.S: revision 1.4
common/lib/libc/arch/aarch64/atomic/atomic_add_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_xor_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_dec_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_32.S: revision 1.4
common/lib/libc/arch/aarch64/atomic/atomic_add_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_8.S: revision 1.4
common/lib/libc/arch/aarch64/atomic/membar_ops.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_16.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_nand_32.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_inc_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_dec_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_and_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_and_64.S: revision 1.2

Part I of ad@'s performance improvements for aarch64
- Remove memory barriers from the atomic ops. I don't understand why thos=
e
are there. Is it some architectural thing, or for a CPU bug, or just
over-caution maybe? They're not needed for correctness.
- Have unlikely conditional branches go forwards to help the static branch
predictor.

Remove memory barriers from the atomic ops macros in the same way as was
done for the other atomic ops earlier.

Use the correct barriers - all of membar_{sync,producer,consumer} have
less scope than before.

LGTM from riastradh

As we're providing the legacy gcc __sync built-in functions for atomic
memory access we might as well get the memory barriers right...

From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)
This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)
This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.1.26.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.1.26.1 21-Apr-2020  martin Sync with HEAD
 1.1.4.2 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.1.4.1 10-Aug-2014  tls file atomic_and_8.S was added on branch tls-maxphys on 2014-08-19 23:45:11 +0000
 1.4 29-Jul-2021  skrll As we're providing the legacy gcc __sync built-in functions for atomic
memory access we might as well get the memory barriers right...
From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)

This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)

This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.3 07-Oct-2020  skrll Comment nit
 1.2 12-Aug-2020  skrll Part I of ad@'s performance improvements for aarch64

- Remove memory barriers from the atomic ops. I don't understand why those
are there. Is it some architectural thing, or for a CPU bug, or just
over-caution maybe? They're not needed for correctness.

- Have unlikely conditional branches go forwards to help the static branch
predictor.
 1.1 10-Aug-2014  matt branches: 1.1.4; 1.1.26; 1.1.28;
Preliminary files for AARCH64 (64-bit ARM) support.
Enough for a distribution build.
 1.1.28.1 11-Aug-2021  martin Pull up following revision(s) (requested by skrll in ticket #1331):

common/lib/libc/arch/aarch64/atomic/atomic_add_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_sub_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_sub_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_64.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_inc_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_or_64.S: revision 1.3
common/lib/libc/arch/aarch64/atomic/atomic_and_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_xor_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_and_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_add_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_op_asm.h: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_swap_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_op_asm.h: revision 1.6
common/lib/libc/arch/aarch64/atomic/atomic_cas_64.S: revision 1.4
common/lib/libc/arch/aarch64/atomic/atomic_nand_8.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_sub_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_64.S: revision 1.6
common/lib/libc/arch/aarch64/atomic/atomic_swap_8.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_sub_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_xor_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_32.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_or_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_64.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_swap_16.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_or_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_or_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_xor_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_16.S: revision 1.4
common/lib/libc/arch/aarch64/atomic/atomic_add_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_xor_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_dec_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_32.S: revision 1.4
common/lib/libc/arch/aarch64/atomic/atomic_add_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_8.S: revision 1.4
common/lib/libc/arch/aarch64/atomic/membar_ops.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_16.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_nand_32.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_inc_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_dec_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_and_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_and_64.S: revision 1.2

Part I of ad@'s performance improvements for aarch64
- Remove memory barriers from the atomic ops. I don't understand why thos=
e
are there. Is it some architectural thing, or for a CPU bug, or just
over-caution maybe? They're not needed for correctness.
- Have unlikely conditional branches go forwards to help the static branch
predictor.

Remove memory barriers from the atomic ops macros in the same way as was
done for the other atomic ops earlier.

Use the correct barriers - all of membar_{sync,producer,consumer} have
less scope than before.

LGTM from riastradh

As we're providing the legacy gcc __sync built-in functions for atomic
memory access we might as well get the memory barriers right...

From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)
This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)
This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.1.26.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.1.26.1 21-Apr-2020  martin Sync with HEAD
 1.1.4.2 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.1.4.1 10-Aug-2014  tls file atomic_cas_16.S was added on branch tls-maxphys on 2014-08-19 23:45:12 +0000
 1.4 29-Jul-2021  skrll As we're providing the legacy gcc __sync built-in functions for atomic
memory access we might as well get the memory barriers right...
From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)

This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)

This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.3 07-Oct-2020  skrll Comment nit
 1.2 12-Aug-2020  skrll Part I of ad@'s performance improvements for aarch64

- Remove memory barriers from the atomic ops. I don't understand why those
are there. Is it some architectural thing, or for a CPU bug, or just
over-caution maybe? They're not needed for correctness.

- Have unlikely conditional branches go forwards to help the static branch
predictor.
 1.1 10-Aug-2014  matt branches: 1.1.4; 1.1.26; 1.1.28;
Preliminary files for AARCH64 (64-bit ARM) support.
Enough for a distribution build.
 1.1.28.1 11-Aug-2021  martin Pull up following revision(s) (requested by skrll in ticket #1331):

common/lib/libc/arch/aarch64/atomic/atomic_add_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_sub_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_sub_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_64.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_inc_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_or_64.S: revision 1.3
common/lib/libc/arch/aarch64/atomic/atomic_and_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_xor_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_and_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_add_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_op_asm.h: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_swap_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_op_asm.h: revision 1.6
common/lib/libc/arch/aarch64/atomic/atomic_cas_64.S: revision 1.4
common/lib/libc/arch/aarch64/atomic/atomic_nand_8.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_sub_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_64.S: revision 1.6
common/lib/libc/arch/aarch64/atomic/atomic_swap_8.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_sub_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_xor_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_32.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_or_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_64.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_swap_16.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_or_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_or_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_xor_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_16.S: revision 1.4
common/lib/libc/arch/aarch64/atomic/atomic_add_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_xor_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_dec_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_32.S: revision 1.4
common/lib/libc/arch/aarch64/atomic/atomic_add_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_8.S: revision 1.4
common/lib/libc/arch/aarch64/atomic/membar_ops.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_16.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_nand_32.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_inc_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_dec_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_and_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_and_64.S: revision 1.2

Part I of ad@'s performance improvements for aarch64
- Remove memory barriers from the atomic ops. I don't understand why thos=
e
are there. Is it some architectural thing, or for a CPU bug, or just
over-caution maybe? They're not needed for correctness.
- Have unlikely conditional branches go forwards to help the static branch
predictor.

Remove memory barriers from the atomic ops macros in the same way as was
done for the other atomic ops earlier.

Use the correct barriers - all of membar_{sync,producer,consumer} have
less scope than before.

LGTM from riastradh

As we're providing the legacy gcc __sync built-in functions for atomic
memory access we might as well get the memory barriers right...

From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)
This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)
This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.1.26.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.1.26.1 21-Apr-2020  martin Sync with HEAD
 1.1.4.2 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.1.4.1 10-Aug-2014  tls file atomic_cas_32.S was added on branch tls-maxphys on 2014-08-19 23:45:12 +0000
 1.6 29-Jul-2021  skrll As we're providing the legacy gcc __sync built-in functions for atomic
memory access we might as well get the memory barriers right...
From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)

This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)

This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.5 07-Oct-2020  skrll Comment nit
 1.4 12-Aug-2020  skrll Part I of ad@'s performance improvements for aarch64

- Remove memory barriers from the atomic ops. I don't understand why those
are there. Is it some architectural thing, or for a CPU bug, or just
over-caution maybe? They're not needed for correctness.

- Have unlikely conditional branches go forwards to help the static branch
predictor.
 1.3 19-Feb-2019  rin branches: 1.3.2;
Sort STRONG_ALIAS's in the same manner as ATOMIC_OP_ALIAS's.
No functional changes.
 1.2 19-Feb-2019  rin Export _atomic_cas_64 as atomic_cas_64_ni.

Note that _atomic_cas_64 is already exported as atomic_cas_{ulong,prt}_ni.

Fix build error of test/lib/atomic/t_atomic_cas, which is successfully
passed on RPI3B+ now.
 1.1 10-Aug-2014  matt branches: 1.1.4; 1.1.26;
Preliminary files for AARCH64 (64-bit ARM) support.
Enough for a distribution build.
 1.1.26.2 21-Apr-2020  martin Sync with HEAD
 1.1.26.1 10-Jun-2019  christos Sync with HEAD
 1.1.4.2 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.1.4.1 10-Aug-2014  tls file atomic_cas_64.S was added on branch tls-maxphys on 2014-08-19 23:45:12 +0000
 1.3.2.1 11-Aug-2021  martin Pull up following revision(s) (requested by skrll in ticket #1331):

common/lib/libc/arch/aarch64/atomic/atomic_add_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_sub_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_sub_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_64.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_inc_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_or_64.S: revision 1.3
common/lib/libc/arch/aarch64/atomic/atomic_and_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_xor_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_and_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_add_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_op_asm.h: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_swap_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_op_asm.h: revision 1.6
common/lib/libc/arch/aarch64/atomic/atomic_cas_64.S: revision 1.4
common/lib/libc/arch/aarch64/atomic/atomic_nand_8.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_sub_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_64.S: revision 1.6
common/lib/libc/arch/aarch64/atomic/atomic_swap_8.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_sub_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_xor_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_32.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_or_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_64.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_swap_16.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_or_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_or_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_xor_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_16.S: revision 1.4
common/lib/libc/arch/aarch64/atomic/atomic_add_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_xor_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_dec_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_32.S: revision 1.4
common/lib/libc/arch/aarch64/atomic/atomic_add_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_8.S: revision 1.4
common/lib/libc/arch/aarch64/atomic/membar_ops.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_16.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_nand_32.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_inc_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_dec_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_and_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_and_64.S: revision 1.2

Part I of ad@'s performance improvements for aarch64
- Remove memory barriers from the atomic ops. I don't understand why thos=
e
are there. Is it some architectural thing, or for a CPU bug, or just
over-caution maybe? They're not needed for correctness.
- Have unlikely conditional branches go forwards to help the static branch
predictor.

Remove memory barriers from the atomic ops macros in the same way as was
done for the other atomic ops earlier.

Use the correct barriers - all of membar_{sync,producer,consumer} have
less scope than before.

LGTM from riastradh

As we're providing the legacy gcc __sync built-in functions for atomic
memory access we might as well get the memory barriers right...

From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)
This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)
This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.4 29-Jul-2021  skrll As we're providing the legacy gcc __sync built-in functions for atomic
memory access we might as well get the memory barriers right...
From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)

This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)

This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.3 07-Oct-2020  skrll Comment nit
 1.2 12-Aug-2020  skrll Part I of ad@'s performance improvements for aarch64

- Remove memory barriers from the atomic ops. I don't understand why those
are there. Is it some architectural thing, or for a CPU bug, or just
over-caution maybe? They're not needed for correctness.

- Have unlikely conditional branches go forwards to help the static branch
predictor.
 1.1 10-Aug-2014  matt branches: 1.1.4; 1.1.26; 1.1.28;
Preliminary files for AARCH64 (64-bit ARM) support.
Enough for a distribution build.
 1.1.28.1 11-Aug-2021  martin Pull up following revision(s) (requested by skrll in ticket #1331):

common/lib/libc/arch/aarch64/atomic/atomic_add_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_sub_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_sub_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_64.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_inc_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_or_64.S: revision 1.3
common/lib/libc/arch/aarch64/atomic/atomic_and_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_xor_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_and_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_add_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_op_asm.h: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_swap_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_op_asm.h: revision 1.6
common/lib/libc/arch/aarch64/atomic/atomic_cas_64.S: revision 1.4
common/lib/libc/arch/aarch64/atomic/atomic_nand_8.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_sub_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_64.S: revision 1.6
common/lib/libc/arch/aarch64/atomic/atomic_swap_8.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_sub_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_xor_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_32.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_or_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_64.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_swap_16.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_or_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_or_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_xor_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_16.S: revision 1.4
common/lib/libc/arch/aarch64/atomic/atomic_add_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_xor_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_dec_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_32.S: revision 1.4
common/lib/libc/arch/aarch64/atomic/atomic_add_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_8.S: revision 1.4
common/lib/libc/arch/aarch64/atomic/membar_ops.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_16.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_nand_32.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_inc_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_dec_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_and_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_and_64.S: revision 1.2

Part I of ad@'s performance improvements for aarch64
- Remove memory barriers from the atomic ops. I don't understand why thos=
e
are there. Is it some architectural thing, or for a CPU bug, or just
over-caution maybe? They're not needed for correctness.
- Have unlikely conditional branches go forwards to help the static branch
predictor.

Remove memory barriers from the atomic ops macros in the same way as was
done for the other atomic ops earlier.

Use the correct barriers - all of membar_{sync,producer,consumer} have
less scope than before.

LGTM from riastradh

As we're providing the legacy gcc __sync built-in functions for atomic
memory access we might as well get the memory barriers right...

From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)
This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)
This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.1.26.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.1.26.1 21-Apr-2020  martin Sync with HEAD
 1.1.4.2 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.1.4.1 10-Aug-2014  tls file atomic_cas_8.S was added on branch tls-maxphys on 2014-08-19 23:45:12 +0000
 1.2 12-Aug-2020  skrll Part I of ad@'s performance improvements for aarch64

- Remove memory barriers from the atomic ops. I don't understand why those
are there. Is it some architectural thing, or for a CPU bug, or just
over-caution maybe? They're not needed for correctness.

- Have unlikely conditional branches go forwards to help the static branch
predictor.
 1.1 10-Aug-2014  matt branches: 1.1.4; 1.1.26; 1.1.28;
Preliminary files for AARCH64 (64-bit ARM) support.
Enough for a distribution build.
 1.1.28.1 11-Aug-2021  martin Pull up following revision(s) (requested by skrll in ticket #1331):

common/lib/libc/arch/aarch64/atomic/atomic_add_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_sub_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_sub_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_64.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_inc_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_or_64.S: revision 1.3
common/lib/libc/arch/aarch64/atomic/atomic_and_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_xor_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_and_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_add_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_op_asm.h: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_swap_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_op_asm.h: revision 1.6
common/lib/libc/arch/aarch64/atomic/atomic_cas_64.S: revision 1.4
common/lib/libc/arch/aarch64/atomic/atomic_nand_8.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_sub_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_64.S: revision 1.6
common/lib/libc/arch/aarch64/atomic/atomic_swap_8.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_sub_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_xor_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_32.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_or_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_64.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_swap_16.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_or_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_or_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_xor_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_16.S: revision 1.4
common/lib/libc/arch/aarch64/atomic/atomic_add_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_xor_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_dec_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_32.S: revision 1.4
common/lib/libc/arch/aarch64/atomic/atomic_add_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_8.S: revision 1.4
common/lib/libc/arch/aarch64/atomic/membar_ops.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_16.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_nand_32.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_inc_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_dec_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_and_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_and_64.S: revision 1.2

Part I of ad@'s performance improvements for aarch64
- Remove memory barriers from the atomic ops. I don't understand why thos=
e
are there. Is it some architectural thing, or for a CPU bug, or just
over-caution maybe? They're not needed for correctness.
- Have unlikely conditional branches go forwards to help the static branch
predictor.

Remove memory barriers from the atomic ops macros in the same way as was
done for the other atomic ops earlier.

Use the correct barriers - all of membar_{sync,producer,consumer} have
less scope than before.

LGTM from riastradh

As we're providing the legacy gcc __sync built-in functions for atomic
memory access we might as well get the memory barriers right...

From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)
This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)
This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.1.26.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.1.26.1 21-Apr-2020  martin Sync with HEAD
 1.1.4.2 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.1.4.1 10-Aug-2014  tls file atomic_dec_32.S was added on branch tls-maxphys on 2014-08-19 23:45:12 +0000
 1.2 12-Aug-2020  skrll Part I of ad@'s performance improvements for aarch64

- Remove memory barriers from the atomic ops. I don't understand why those
are there. Is it some architectural thing, or for a CPU bug, or just
over-caution maybe? They're not needed for correctness.

- Have unlikely conditional branches go forwards to help the static branch
predictor.
 1.1 10-Aug-2014  matt branches: 1.1.4; 1.1.26; 1.1.28;
Preliminary files for AARCH64 (64-bit ARM) support.
Enough for a distribution build.
 1.1.28.1 11-Aug-2021  martin Pull up following revision(s) (requested by skrll in ticket #1331):

common/lib/libc/arch/aarch64/atomic/atomic_add_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_sub_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_sub_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_64.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_inc_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_or_64.S: revision 1.3
common/lib/libc/arch/aarch64/atomic/atomic_and_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_xor_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_and_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_add_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_op_asm.h: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_swap_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_op_asm.h: revision 1.6
common/lib/libc/arch/aarch64/atomic/atomic_cas_64.S: revision 1.4
common/lib/libc/arch/aarch64/atomic/atomic_nand_8.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_sub_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_64.S: revision 1.6
common/lib/libc/arch/aarch64/atomic/atomic_swap_8.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_sub_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_xor_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_32.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_or_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_64.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_swap_16.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_or_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_or_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_xor_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_16.S: revision 1.4
common/lib/libc/arch/aarch64/atomic/atomic_add_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_xor_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_dec_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_32.S: revision 1.4
common/lib/libc/arch/aarch64/atomic/atomic_add_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_8.S: revision 1.4
common/lib/libc/arch/aarch64/atomic/membar_ops.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_16.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_nand_32.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_inc_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_dec_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_and_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_and_64.S: revision 1.2

Part I of ad@'s performance improvements for aarch64
- Remove memory barriers from the atomic ops. I don't understand why thos=
e
are there. Is it some architectural thing, or for a CPU bug, or just
over-caution maybe? They're not needed for correctness.
- Have unlikely conditional branches go forwards to help the static branch
predictor.

Remove memory barriers from the atomic ops macros in the same way as was
done for the other atomic ops earlier.

Use the correct barriers - all of membar_{sync,producer,consumer} have
less scope than before.

LGTM from riastradh

As we're providing the legacy gcc __sync built-in functions for atomic
memory access we might as well get the memory barriers right...

From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)
This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)
This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.1.26.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.1.26.1 21-Apr-2020  martin Sync with HEAD
 1.1.4.2 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.1.4.1 10-Aug-2014  tls file atomic_dec_64.S was added on branch tls-maxphys on 2014-08-19 23:45:12 +0000
 1.2 12-Aug-2020  skrll Part I of ad@'s performance improvements for aarch64

- Remove memory barriers from the atomic ops. I don't understand why those
are there. Is it some architectural thing, or for a CPU bug, or just
over-caution maybe? They're not needed for correctness.

- Have unlikely conditional branches go forwards to help the static branch
predictor.
 1.1 10-Aug-2014  matt branches: 1.1.4; 1.1.26; 1.1.28;
Preliminary files for AARCH64 (64-bit ARM) support.
Enough for a distribution build.
 1.1.28.1 11-Aug-2021  martin Pull up following revision(s) (requested by skrll in ticket #1331):

common/lib/libc/arch/aarch64/atomic/atomic_add_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_sub_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_sub_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_64.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_inc_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_or_64.S: revision 1.3
common/lib/libc/arch/aarch64/atomic/atomic_and_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_xor_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_and_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_add_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_op_asm.h: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_swap_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_op_asm.h: revision 1.6
common/lib/libc/arch/aarch64/atomic/atomic_cas_64.S: revision 1.4
common/lib/libc/arch/aarch64/atomic/atomic_nand_8.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_sub_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_64.S: revision 1.6
common/lib/libc/arch/aarch64/atomic/atomic_swap_8.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_sub_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_xor_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_32.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_or_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_64.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_swap_16.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_or_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_or_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_xor_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_16.S: revision 1.4
common/lib/libc/arch/aarch64/atomic/atomic_add_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_xor_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_dec_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_32.S: revision 1.4
common/lib/libc/arch/aarch64/atomic/atomic_add_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_8.S: revision 1.4
common/lib/libc/arch/aarch64/atomic/membar_ops.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_16.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_nand_32.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_inc_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_dec_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_and_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_and_64.S: revision 1.2

Part I of ad@'s performance improvements for aarch64
- Remove memory barriers from the atomic ops. I don't understand why thos=
e
are there. Is it some architectural thing, or for a CPU bug, or just
over-caution maybe? They're not needed for correctness.
- Have unlikely conditional branches go forwards to help the static branch
predictor.

Remove memory barriers from the atomic ops macros in the same way as was
done for the other atomic ops earlier.

Use the correct barriers - all of membar_{sync,producer,consumer} have
less scope than before.

LGTM from riastradh

As we're providing the legacy gcc __sync built-in functions for atomic
memory access we might as well get the memory barriers right...

From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)
This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)
This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.1.26.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.1.26.1 21-Apr-2020  martin Sync with HEAD
 1.1.4.2 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.1.4.1 10-Aug-2014  tls file atomic_inc_32.S was added on branch tls-maxphys on 2014-08-19 23:45:12 +0000
 1.2 12-Aug-2020  skrll Part I of ad@'s performance improvements for aarch64

- Remove memory barriers from the atomic ops. I don't understand why those
are there. Is it some architectural thing, or for a CPU bug, or just
over-caution maybe? They're not needed for correctness.

- Have unlikely conditional branches go forwards to help the static branch
predictor.
 1.1 10-Aug-2014  matt branches: 1.1.4; 1.1.26; 1.1.28;
Preliminary files for AARCH64 (64-bit ARM) support.
Enough for a distribution build.
 1.1.28.1 11-Aug-2021  martin Pull up following revision(s) (requested by skrll in ticket #1331):

common/lib/libc/arch/aarch64/atomic/atomic_add_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_sub_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_sub_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_64.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_inc_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_or_64.S: revision 1.3
common/lib/libc/arch/aarch64/atomic/atomic_and_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_xor_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_and_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_add_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_op_asm.h: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_swap_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_op_asm.h: revision 1.6
common/lib/libc/arch/aarch64/atomic/atomic_cas_64.S: revision 1.4
common/lib/libc/arch/aarch64/atomic/atomic_nand_8.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_sub_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_64.S: revision 1.6
common/lib/libc/arch/aarch64/atomic/atomic_swap_8.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_sub_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_xor_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_32.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_or_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_64.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_swap_16.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_or_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_or_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_xor_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_16.S: revision 1.4
common/lib/libc/arch/aarch64/atomic/atomic_add_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_xor_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_dec_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_32.S: revision 1.4
common/lib/libc/arch/aarch64/atomic/atomic_add_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_8.S: revision 1.4
common/lib/libc/arch/aarch64/atomic/membar_ops.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_16.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_nand_32.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_inc_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_dec_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_and_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_and_64.S: revision 1.2

Part I of ad@'s performance improvements for aarch64
- Remove memory barriers from the atomic ops. I don't understand why thos=
e
are there. Is it some architectural thing, or for a CPU bug, or just
over-caution maybe? They're not needed for correctness.
- Have unlikely conditional branches go forwards to help the static branch
predictor.

Remove memory barriers from the atomic ops macros in the same way as was
done for the other atomic ops earlier.

Use the correct barriers - all of membar_{sync,producer,consumer} have
less scope than before.

LGTM from riastradh

As we're providing the legacy gcc __sync built-in functions for atomic
memory access we might as well get the memory barriers right...

From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)
This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)
This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.1.26.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.1.26.1 21-Apr-2020  martin Sync with HEAD
 1.1.4.2 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.1.4.1 10-Aug-2014  tls file atomic_inc_64.S was added on branch tls-maxphys on 2014-08-19 23:45:12 +0000
 1.5 29-Jul-2021  skrll As we're providing the legacy gcc __sync built-in functions for atomic
memory access we might as well get the memory barriers right...
From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)

This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)

This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.4 06-Jul-2021  skrll One more s/pte/ptr/
 1.3 04-Jul-2021  skrll Fix the logic operation for atomic_nand_{8,16,32,64}

From the gcc docs the operations are as follows

{ tmp = *ptr; *ptr = ~(tmp & value); return tmp; } // nand
{ tmp = ~(*ptr & value); *ptr = tmp; return *ptr; } // nand

yes, this is really rather strange.
 1.2 12-Aug-2020  skrll Part I of ad@'s performance improvements for aarch64

- Remove memory barriers from the atomic ops. I don't understand why those
are there. Is it some architectural thing, or for a CPU bug, or just
over-caution maybe? They're not needed for correctness.

- Have unlikely conditional branches go forwards to help the static branch
predictor.
 1.1 10-Aug-2014  matt branches: 1.1.4; 1.1.26; 1.1.28;
Preliminary files for AARCH64 (64-bit ARM) support.
Enough for a distribution build.
 1.1.28.3 11-Aug-2021  martin Pull up following revision(s) (requested by skrll in ticket #1331):

common/lib/libc/arch/aarch64/atomic/atomic_add_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_sub_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_sub_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_64.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_inc_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_or_64.S: revision 1.3
common/lib/libc/arch/aarch64/atomic/atomic_and_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_xor_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_and_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_add_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_op_asm.h: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_swap_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_op_asm.h: revision 1.6
common/lib/libc/arch/aarch64/atomic/atomic_cas_64.S: revision 1.4
common/lib/libc/arch/aarch64/atomic/atomic_nand_8.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_sub_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_64.S: revision 1.6
common/lib/libc/arch/aarch64/atomic/atomic_swap_8.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_sub_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_xor_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_32.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_or_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_64.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_swap_16.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_or_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_or_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_xor_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_16.S: revision 1.4
common/lib/libc/arch/aarch64/atomic/atomic_add_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_xor_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_dec_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_32.S: revision 1.4
common/lib/libc/arch/aarch64/atomic/atomic_add_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_8.S: revision 1.4
common/lib/libc/arch/aarch64/atomic/membar_ops.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_16.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_nand_32.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_inc_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_dec_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_and_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_and_64.S: revision 1.2

Part I of ad@'s performance improvements for aarch64
- Remove memory barriers from the atomic ops. I don't understand why thos=
e
are there. Is it some architectural thing, or for a CPU bug, or just
over-caution maybe? They're not needed for correctness.
- Have unlikely conditional branches go forwards to help the static branch
predictor.

Remove memory barriers from the atomic ops macros in the same way as was
done for the other atomic ops earlier.

Use the correct barriers - all of membar_{sync,producer,consumer} have
less scope than before.

LGTM from riastradh

As we're providing the legacy gcc __sync built-in functions for atomic
memory access we might as well get the memory barriers right...

From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)
This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)
This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.1.28.2 08-Jul-2021  martin Pull up following revision(s) (requested by skrll in ticket #1319):

common/lib/libc/arch/aarch64/atomic/atomic_nand_16.S: revision 1.4

One more s/pte/ptr/
 1.1.28.1 06-Jul-2021  martin Pull up following revision(s) (requested by skrll in ticket #1314):

common/lib/libc/arch/aarch64/atomic/atomic_nand_8.S: revision 1.3
common/lib/libc/arch/aarch64/atomic/atomic_nand_8.S: revision 1.4
common/lib/libc/arch/aarch64/atomic/atomic_nand_64.S: revision 1.3
common/lib/libc/arch/aarch64/atomic/atomic_nand_64.S: revision 1.4
common/lib/libc/arch/aarch64/atomic/atomic_nand_16.S: revision 1.3
common/lib/libc/arch/aarch64/atomic/atomic_nand_32.S: revision 1.3
common/lib/libc/arch/aarch64/atomic/atomic_nand_32.S: revision 1.4

Fix the logic operation for atomic_nand_{8,16,32,64}

From the gcc docs the operations are as follows
{ tmp = *ptr; *ptr = ~(tmp & value); return tmp; } // nand
{ tmp = ~(*ptr & value); *ptr = tmp; return *ptr; } // nand

yes, this is really rather strange.

typo in comment s/pte/ptr/
 1.1.26.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.1.26.1 21-Apr-2020  martin Sync with HEAD
 1.1.4.2 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.1.4.1 10-Aug-2014  tls file atomic_nand_16.S was added on branch tls-maxphys on 2014-08-19 23:45:12 +0000
 1.5 29-Jul-2021  skrll As we're providing the legacy gcc __sync built-in functions for atomic
memory access we might as well get the memory barriers right...
From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)

This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)

This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.4 05-Jul-2021  skrll typo in comment s/pte/ptr/
 1.3 04-Jul-2021  skrll Fix the logic operation for atomic_nand_{8,16,32,64}

From the gcc docs the operations are as follows

{ tmp = *ptr; *ptr = ~(tmp & value); return tmp; } // nand
{ tmp = ~(*ptr & value); *ptr = tmp; return *ptr; } // nand

yes, this is really rather strange.
 1.2 12-Aug-2020  skrll Part I of ad@'s performance improvements for aarch64

- Remove memory barriers from the atomic ops. I don't understand why those
are there. Is it some architectural thing, or for a CPU bug, or just
over-caution maybe? They're not needed for correctness.

- Have unlikely conditional branches go forwards to help the static branch
predictor.
 1.1 10-Aug-2014  matt branches: 1.1.4; 1.1.26; 1.1.28;
Preliminary files for AARCH64 (64-bit ARM) support.
Enough for a distribution build.
 1.1.28.2 11-Aug-2021  martin Pull up following revision(s) (requested by skrll in ticket #1331):

common/lib/libc/arch/aarch64/atomic/atomic_add_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_sub_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_sub_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_64.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_inc_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_or_64.S: revision 1.3
common/lib/libc/arch/aarch64/atomic/atomic_and_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_xor_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_and_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_add_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_op_asm.h: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_swap_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_op_asm.h: revision 1.6
common/lib/libc/arch/aarch64/atomic/atomic_cas_64.S: revision 1.4
common/lib/libc/arch/aarch64/atomic/atomic_nand_8.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_sub_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_64.S: revision 1.6
common/lib/libc/arch/aarch64/atomic/atomic_swap_8.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_sub_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_xor_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_32.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_or_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_64.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_swap_16.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_or_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_or_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_xor_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_16.S: revision 1.4
common/lib/libc/arch/aarch64/atomic/atomic_add_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_xor_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_dec_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_32.S: revision 1.4
common/lib/libc/arch/aarch64/atomic/atomic_add_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_8.S: revision 1.4
common/lib/libc/arch/aarch64/atomic/membar_ops.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_16.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_nand_32.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_inc_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_dec_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_and_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_and_64.S: revision 1.2

Part I of ad@'s performance improvements for aarch64
- Remove memory barriers from the atomic ops. I don't understand why thos=
e
are there. Is it some architectural thing, or for a CPU bug, or just
over-caution maybe? They're not needed for correctness.
- Have unlikely conditional branches go forwards to help the static branch
predictor.

Remove memory barriers from the atomic ops macros in the same way as was
done for the other atomic ops earlier.

Use the correct barriers - all of membar_{sync,producer,consumer} have
less scope than before.

LGTM from riastradh

As we're providing the legacy gcc __sync built-in functions for atomic
memory access we might as well get the memory barriers right...

From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)
This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)
This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.1.28.1 06-Jul-2021  martin Pull up following revision(s) (requested by skrll in ticket #1314):

common/lib/libc/arch/aarch64/atomic/atomic_nand_8.S: revision 1.3
common/lib/libc/arch/aarch64/atomic/atomic_nand_8.S: revision 1.4
common/lib/libc/arch/aarch64/atomic/atomic_nand_64.S: revision 1.3
common/lib/libc/arch/aarch64/atomic/atomic_nand_64.S: revision 1.4
common/lib/libc/arch/aarch64/atomic/atomic_nand_16.S: revision 1.3
common/lib/libc/arch/aarch64/atomic/atomic_nand_32.S: revision 1.3
common/lib/libc/arch/aarch64/atomic/atomic_nand_32.S: revision 1.4

Fix the logic operation for atomic_nand_{8,16,32,64}

From the gcc docs the operations are as follows
{ tmp = *ptr; *ptr = ~(tmp & value); return tmp; } // nand
{ tmp = ~(*ptr & value); *ptr = tmp; return *ptr; } // nand

yes, this is really rather strange.

typo in comment s/pte/ptr/
 1.1.26.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.1.26.1 21-Apr-2020  martin Sync with HEAD
 1.1.4.2 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.1.4.1 10-Aug-2014  tls file atomic_nand_32.S was added on branch tls-maxphys on 2014-08-19 23:45:12 +0000
 1.5 29-Jul-2021  skrll As we're providing the legacy gcc __sync built-in functions for atomic
memory access we might as well get the memory barriers right...
From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)

This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)

This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.4 05-Jul-2021  skrll typo in comment s/pte/ptr/
 1.3 04-Jul-2021  skrll Fix the logic operation for atomic_nand_{8,16,32,64}

From the gcc docs the operations are as follows

{ tmp = *ptr; *ptr = ~(tmp & value); return tmp; } // nand
{ tmp = ~(*ptr & value); *ptr = tmp; return *ptr; } // nand

yes, this is really rather strange.
 1.2 12-Aug-2020  skrll Part I of ad@'s performance improvements for aarch64

- Remove memory barriers from the atomic ops. I don't understand why those
are there. Is it some architectural thing, or for a CPU bug, or just
over-caution maybe? They're not needed for correctness.

- Have unlikely conditional branches go forwards to help the static branch
predictor.
 1.1 10-Aug-2014  matt branches: 1.1.4; 1.1.26; 1.1.28;
Preliminary files for AARCH64 (64-bit ARM) support.
Enough for a distribution build.
 1.1.28.2 11-Aug-2021  martin Pull up following revision(s) (requested by skrll in ticket #1331):

common/lib/libc/arch/aarch64/atomic/atomic_add_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_sub_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_sub_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_64.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_inc_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_or_64.S: revision 1.3
common/lib/libc/arch/aarch64/atomic/atomic_and_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_xor_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_and_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_add_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_op_asm.h: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_swap_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_op_asm.h: revision 1.6
common/lib/libc/arch/aarch64/atomic/atomic_cas_64.S: revision 1.4
common/lib/libc/arch/aarch64/atomic/atomic_nand_8.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_sub_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_64.S: revision 1.6
common/lib/libc/arch/aarch64/atomic/atomic_swap_8.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_sub_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_xor_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_32.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_or_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_64.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_swap_16.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_or_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_or_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_xor_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_16.S: revision 1.4
common/lib/libc/arch/aarch64/atomic/atomic_add_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_xor_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_dec_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_32.S: revision 1.4
common/lib/libc/arch/aarch64/atomic/atomic_add_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_8.S: revision 1.4
common/lib/libc/arch/aarch64/atomic/membar_ops.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_16.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_nand_32.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_inc_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_dec_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_and_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_and_64.S: revision 1.2

Part I of ad@'s performance improvements for aarch64
- Remove memory barriers from the atomic ops. I don't understand why thos=
e
are there. Is it some architectural thing, or for a CPU bug, or just
over-caution maybe? They're not needed for correctness.
- Have unlikely conditional branches go forwards to help the static branch
predictor.

Remove memory barriers from the atomic ops macros in the same way as was
done for the other atomic ops earlier.

Use the correct barriers - all of membar_{sync,producer,consumer} have
less scope than before.

LGTM from riastradh

As we're providing the legacy gcc __sync built-in functions for atomic
memory access we might as well get the memory barriers right...

From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)
This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)
This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.1.28.1 06-Jul-2021  martin Pull up following revision(s) (requested by skrll in ticket #1314):

common/lib/libc/arch/aarch64/atomic/atomic_nand_8.S: revision 1.3
common/lib/libc/arch/aarch64/atomic/atomic_nand_8.S: revision 1.4
common/lib/libc/arch/aarch64/atomic/atomic_nand_64.S: revision 1.3
common/lib/libc/arch/aarch64/atomic/atomic_nand_64.S: revision 1.4
common/lib/libc/arch/aarch64/atomic/atomic_nand_16.S: revision 1.3
common/lib/libc/arch/aarch64/atomic/atomic_nand_32.S: revision 1.3
common/lib/libc/arch/aarch64/atomic/atomic_nand_32.S: revision 1.4

Fix the logic operation for atomic_nand_{8,16,32,64}

From the gcc docs the operations are as follows
{ tmp = *ptr; *ptr = ~(tmp & value); return tmp; } // nand
{ tmp = ~(*ptr & value); *ptr = tmp; return *ptr; } // nand

yes, this is really rather strange.

typo in comment s/pte/ptr/
 1.1.26.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.1.26.1 21-Apr-2020  martin Sync with HEAD
 1.1.4.2 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.1.4.1 10-Aug-2014  tls file atomic_nand_64.S was added on branch tls-maxphys on 2014-08-19 23:45:12 +0000
 1.5 29-Jul-2021  skrll As we're providing the legacy gcc __sync built-in functions for atomic
memory access we might as well get the memory barriers right...
From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)

This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)

This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.4 05-Jul-2021  skrll typo in comment s/pte/ptr/
 1.3 04-Jul-2021  skrll Fix the logic operation for atomic_nand_{8,16,32,64}

From the gcc docs the operations are as follows

{ tmp = *ptr; *ptr = ~(tmp & value); return tmp; } // nand
{ tmp = ~(*ptr & value); *ptr = tmp; return *ptr; } // nand

yes, this is really rather strange.
 1.2 12-Aug-2020  skrll Part I of ad@'s performance improvements for aarch64

- Remove memory barriers from the atomic ops. I don't understand why those
are there. Is it some architectural thing, or for a CPU bug, or just
over-caution maybe? They're not needed for correctness.

- Have unlikely conditional branches go forwards to help the static branch
predictor.
 1.1 10-Aug-2014  matt branches: 1.1.4; 1.1.26; 1.1.28;
Preliminary files for AARCH64 (64-bit ARM) support.
Enough for a distribution build.
 1.1.28.2 11-Aug-2021  martin Pull up following revision(s) (requested by skrll in ticket #1331):

common/lib/libc/arch/aarch64/atomic/atomic_add_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_sub_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_sub_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_64.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_inc_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_or_64.S: revision 1.3
common/lib/libc/arch/aarch64/atomic/atomic_and_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_xor_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_and_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_add_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_op_asm.h: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_swap_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_op_asm.h: revision 1.6
common/lib/libc/arch/aarch64/atomic/atomic_cas_64.S: revision 1.4
common/lib/libc/arch/aarch64/atomic/atomic_nand_8.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_sub_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_64.S: revision 1.6
common/lib/libc/arch/aarch64/atomic/atomic_swap_8.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_sub_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_xor_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_32.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_or_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_64.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_swap_16.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_or_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_or_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_xor_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_16.S: revision 1.4
common/lib/libc/arch/aarch64/atomic/atomic_add_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_xor_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_dec_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_32.S: revision 1.4
common/lib/libc/arch/aarch64/atomic/atomic_add_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_8.S: revision 1.4
common/lib/libc/arch/aarch64/atomic/membar_ops.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_16.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_nand_32.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_inc_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_dec_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_and_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_and_64.S: revision 1.2

Part I of ad@'s performance improvements for aarch64
- Remove memory barriers from the atomic ops. I don't understand why thos=
e
are there. Is it some architectural thing, or for a CPU bug, or just
over-caution maybe? They're not needed for correctness.
- Have unlikely conditional branches go forwards to help the static branch
predictor.

Remove memory barriers from the atomic ops macros in the same way as was
done for the other atomic ops earlier.

Use the correct barriers - all of membar_{sync,producer,consumer} have
less scope than before.

LGTM from riastradh

As we're providing the legacy gcc __sync built-in functions for atomic
memory access we might as well get the memory barriers right...

From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)
This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)
This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.1.28.1 06-Jul-2021  martin Pull up following revision(s) (requested by skrll in ticket #1314):

common/lib/libc/arch/aarch64/atomic/atomic_nand_8.S: revision 1.3
common/lib/libc/arch/aarch64/atomic/atomic_nand_8.S: revision 1.4
common/lib/libc/arch/aarch64/atomic/atomic_nand_64.S: revision 1.3
common/lib/libc/arch/aarch64/atomic/atomic_nand_64.S: revision 1.4
common/lib/libc/arch/aarch64/atomic/atomic_nand_16.S: revision 1.3
common/lib/libc/arch/aarch64/atomic/atomic_nand_32.S: revision 1.3
common/lib/libc/arch/aarch64/atomic/atomic_nand_32.S: revision 1.4

Fix the logic operation for atomic_nand_{8,16,32,64}

From the gcc docs the operations are as follows
{ tmp = *ptr; *ptr = ~(tmp & value); return tmp; } // nand
{ tmp = ~(*ptr & value); *ptr = tmp; return *ptr; } // nand

yes, this is really rather strange.

typo in comment s/pte/ptr/
 1.1.26.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.1.26.1 21-Apr-2020  martin Sync with HEAD
 1.1.4.2 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.1.4.1 10-Aug-2014  tls file atomic_nand_8.S was added on branch tls-maxphys on 2014-08-19 23:45:12 +0000
 1.6 29-Jul-2021  skrll As we're providing the legacy gcc __sync built-in functions for atomic
memory access we might as well get the memory barriers right...
From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)

This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)

This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.5 13-Oct-2020  skrll Remove memory barriers from the atomic ops macros in the same way as was
done for the other atomic ops earlier.
 1.4 07-Oct-2020  skrll Comment nit
 1.3 08-Feb-2019  ryo branches: 1.3.2;
- atomic_*_{8,16}_nv() must return a new value, not an old value.
- use "dmb sy" for atomic_*{8,16}_nv() in the same way as atomic_*{32,64}_nv().
 1.2 06-Feb-2019  ryo fix atomic_sub_*(). it was (delta - *ptr), should be (*ptr - delta).
changing shared macro doesn't effect other atomic_ops because
(*ptr [+|^] delta) and (delta [+|^] *ptr) have same result.

atomic_sub_*() haven't used because non standard API?
 1.1 10-Aug-2014  matt branches: 1.1.4; 1.1.26;
Preliminary files for AARCH64 (64-bit ARM) support.
Enough for a distribution build.
 1.1.26.2 21-Apr-2020  martin Sync with HEAD
 1.1.26.1 10-Jun-2019  christos Sync with HEAD
 1.1.4.2 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.1.4.1 10-Aug-2014  tls file atomic_op_asm.h was added on branch tls-maxphys on 2014-08-19 23:45:12 +0000
 1.3.2.1 11-Aug-2021  martin Pull up following revision(s) (requested by skrll in ticket #1331):

common/lib/libc/arch/aarch64/atomic/atomic_add_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_sub_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_sub_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_64.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_inc_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_or_64.S: revision 1.3
common/lib/libc/arch/aarch64/atomic/atomic_and_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_xor_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_and_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_add_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_op_asm.h: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_swap_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_op_asm.h: revision 1.6
common/lib/libc/arch/aarch64/atomic/atomic_cas_64.S: revision 1.4
common/lib/libc/arch/aarch64/atomic/atomic_nand_8.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_sub_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_64.S: revision 1.6
common/lib/libc/arch/aarch64/atomic/atomic_swap_8.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_sub_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_xor_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_32.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_or_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_64.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_swap_16.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_or_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_or_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_xor_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_16.S: revision 1.4
common/lib/libc/arch/aarch64/atomic/atomic_add_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_xor_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_dec_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_32.S: revision 1.4
common/lib/libc/arch/aarch64/atomic/atomic_add_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_8.S: revision 1.4
common/lib/libc/arch/aarch64/atomic/membar_ops.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_16.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_nand_32.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_inc_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_dec_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_and_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_and_64.S: revision 1.2

Part I of ad@'s performance improvements for aarch64
- Remove memory barriers from the atomic ops. I don't understand why thos=
e
are there. Is it some architectural thing, or for a CPU bug, or just
over-caution maybe? They're not needed for correctness.
- Have unlikely conditional branches go forwards to help the static branch
predictor.

Remove memory barriers from the atomic ops macros in the same way as was
done for the other atomic ops earlier.

Use the correct barriers - all of membar_{sync,producer,consumer} have
less scope than before.

LGTM from riastradh

As we're providing the legacy gcc __sync built-in functions for atomic
memory access we might as well get the memory barriers right...

From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)
This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)
This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.2 29-Jul-2021  skrll As we're providing the legacy gcc __sync built-in functions for atomic
memory access we might as well get the memory barriers right...
From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)

This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)

This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.1 10-Aug-2014  matt branches: 1.1.4; 1.1.26; 1.1.28;
Preliminary files for AARCH64 (64-bit ARM) support.
Enough for a distribution build.
 1.1.28.1 11-Aug-2021  martin Pull up following revision(s) (requested by skrll in ticket #1331):

common/lib/libc/arch/aarch64/atomic/atomic_add_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_sub_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_sub_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_64.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_inc_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_or_64.S: revision 1.3
common/lib/libc/arch/aarch64/atomic/atomic_and_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_xor_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_and_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_add_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_op_asm.h: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_swap_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_op_asm.h: revision 1.6
common/lib/libc/arch/aarch64/atomic/atomic_cas_64.S: revision 1.4
common/lib/libc/arch/aarch64/atomic/atomic_nand_8.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_sub_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_64.S: revision 1.6
common/lib/libc/arch/aarch64/atomic/atomic_swap_8.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_sub_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_xor_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_32.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_or_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_64.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_swap_16.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_or_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_or_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_xor_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_16.S: revision 1.4
common/lib/libc/arch/aarch64/atomic/atomic_add_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_xor_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_dec_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_32.S: revision 1.4
common/lib/libc/arch/aarch64/atomic/atomic_add_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_8.S: revision 1.4
common/lib/libc/arch/aarch64/atomic/membar_ops.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_16.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_nand_32.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_inc_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_dec_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_and_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_and_64.S: revision 1.2

Part I of ad@'s performance improvements for aarch64
- Remove memory barriers from the atomic ops. I don't understand why thos=
e
are there. Is it some architectural thing, or for a CPU bug, or just
over-caution maybe? They're not needed for correctness.
- Have unlikely conditional branches go forwards to help the static branch
predictor.

Remove memory barriers from the atomic ops macros in the same way as was
done for the other atomic ops earlier.

Use the correct barriers - all of membar_{sync,producer,consumer} have
less scope than before.

LGTM from riastradh

As we're providing the legacy gcc __sync built-in functions for atomic
memory access we might as well get the memory barriers right...

From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)
This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)
This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.1.26.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.1.26.1 21-Apr-2020  martin Sync with HEAD
 1.1.4.2 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.1.4.1 10-Aug-2014  tls file atomic_or_16.S was added on branch tls-maxphys on 2014-08-19 23:45:12 +0000
 1.2 29-Jul-2021  skrll As we're providing the legacy gcc __sync built-in functions for atomic
memory access we might as well get the memory barriers right...
From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)

This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)

This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.1 10-Aug-2014  matt branches: 1.1.4; 1.1.26; 1.1.28;
Preliminary files for AARCH64 (64-bit ARM) support.
Enough for a distribution build.
 1.1.28.1 11-Aug-2021  martin Pull up following revision(s) (requested by skrll in ticket #1331):

common/lib/libc/arch/aarch64/atomic/atomic_add_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_sub_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_sub_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_64.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_inc_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_or_64.S: revision 1.3
common/lib/libc/arch/aarch64/atomic/atomic_and_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_xor_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_and_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_add_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_op_asm.h: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_swap_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_op_asm.h: revision 1.6
common/lib/libc/arch/aarch64/atomic/atomic_cas_64.S: revision 1.4
common/lib/libc/arch/aarch64/atomic/atomic_nand_8.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_sub_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_64.S: revision 1.6
common/lib/libc/arch/aarch64/atomic/atomic_swap_8.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_sub_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_xor_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_32.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_or_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_64.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_swap_16.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_or_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_or_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_xor_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_16.S: revision 1.4
common/lib/libc/arch/aarch64/atomic/atomic_add_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_xor_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_dec_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_32.S: revision 1.4
common/lib/libc/arch/aarch64/atomic/atomic_add_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_8.S: revision 1.4
common/lib/libc/arch/aarch64/atomic/membar_ops.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_16.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_nand_32.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_inc_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_dec_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_and_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_and_64.S: revision 1.2

Part I of ad@'s performance improvements for aarch64
- Remove memory barriers from the atomic ops. I don't understand why thos=
e
are there. Is it some architectural thing, or for a CPU bug, or just
over-caution maybe? They're not needed for correctness.
- Have unlikely conditional branches go forwards to help the static branch
predictor.

Remove memory barriers from the atomic ops macros in the same way as was
done for the other atomic ops earlier.

Use the correct barriers - all of membar_{sync,producer,consumer} have
less scope than before.

LGTM from riastradh

As we're providing the legacy gcc __sync built-in functions for atomic
memory access we might as well get the memory barriers right...

From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)
This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)
This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.1.26.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.1.26.1 21-Apr-2020  martin Sync with HEAD
 1.1.4.2 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.1.4.1 10-Aug-2014  tls file atomic_or_32.S was added on branch tls-maxphys on 2014-08-19 23:45:12 +0000
 1.3 29-Jul-2021  skrll As we're providing the legacy gcc __sync built-in functions for atomic
memory access we might as well get the memory barriers right...
From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)

This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)

This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.2 15-Sep-2019  skrll __sync_or_and_fetch_8 should return new value... make it do that.
 1.1 10-Aug-2014  matt branches: 1.1.4; 1.1.26; 1.1.28;
Preliminary files for AARCH64 (64-bit ARM) support.
Enough for a distribution build.
 1.1.28.2 11-Aug-2021  martin Pull up following revision(s) (requested by skrll in ticket #1331):

common/lib/libc/arch/aarch64/atomic/atomic_add_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_sub_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_sub_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_64.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_inc_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_or_64.S: revision 1.3
common/lib/libc/arch/aarch64/atomic/atomic_and_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_xor_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_and_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_add_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_op_asm.h: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_swap_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_op_asm.h: revision 1.6
common/lib/libc/arch/aarch64/atomic/atomic_cas_64.S: revision 1.4
common/lib/libc/arch/aarch64/atomic/atomic_nand_8.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_sub_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_64.S: revision 1.6
common/lib/libc/arch/aarch64/atomic/atomic_swap_8.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_sub_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_xor_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_32.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_or_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_64.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_swap_16.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_or_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_or_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_xor_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_16.S: revision 1.4
common/lib/libc/arch/aarch64/atomic/atomic_add_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_xor_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_dec_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_32.S: revision 1.4
common/lib/libc/arch/aarch64/atomic/atomic_add_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_8.S: revision 1.4
common/lib/libc/arch/aarch64/atomic/membar_ops.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_16.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_nand_32.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_inc_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_dec_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_and_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_and_64.S: revision 1.2

Part I of ad@'s performance improvements for aarch64
- Remove memory barriers from the atomic ops. I don't understand why thos=
e
are there. Is it some architectural thing, or for a CPU bug, or just
over-caution maybe? They're not needed for correctness.
- Have unlikely conditional branches go forwards to help the static branch
predictor.

Remove memory barriers from the atomic ops macros in the same way as was
done for the other atomic ops earlier.

Use the correct barriers - all of membar_{sync,producer,consumer} have
less scope than before.

LGTM from riastradh

As we're providing the legacy gcc __sync built-in functions for atomic
memory access we might as well get the memory barriers right...

From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)
This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)
This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.1.28.1 17-Sep-2019  martin Pull up following revision(s) (requested by skrll in ticket #201):

common/lib/libc/arch/aarch64/atomic/atomic_or_64.S: revision 1.2

__sync_or_and_fetch_8 should return new value... make it do that.
 1.1.26.2 21-Apr-2020  martin Sync with HEAD
 1.1.26.1 13-Apr-2020  martin Mostly merge changes from HEAD upto 20200411
 1.1.4.2 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.1.4.1 10-Aug-2014  tls file atomic_or_64.S was added on branch tls-maxphys on 2014-08-19 23:45:12 +0000
 1.2 29-Jul-2021  skrll As we're providing the legacy gcc __sync built-in functions for atomic
memory access we might as well get the memory barriers right...
From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)

This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)

This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.1 10-Aug-2014  matt branches: 1.1.4; 1.1.26; 1.1.28;
Preliminary files for AARCH64 (64-bit ARM) support.
Enough for a distribution build.
 1.1.28.1 11-Aug-2021  martin Pull up following revision(s) (requested by skrll in ticket #1331):

common/lib/libc/arch/aarch64/atomic/atomic_add_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_sub_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_sub_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_64.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_inc_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_or_64.S: revision 1.3
common/lib/libc/arch/aarch64/atomic/atomic_and_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_xor_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_and_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_add_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_op_asm.h: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_swap_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_op_asm.h: revision 1.6
common/lib/libc/arch/aarch64/atomic/atomic_cas_64.S: revision 1.4
common/lib/libc/arch/aarch64/atomic/atomic_nand_8.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_sub_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_64.S: revision 1.6
common/lib/libc/arch/aarch64/atomic/atomic_swap_8.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_sub_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_xor_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_32.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_or_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_64.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_swap_16.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_or_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_or_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_xor_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_16.S: revision 1.4
common/lib/libc/arch/aarch64/atomic/atomic_add_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_xor_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_dec_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_32.S: revision 1.4
common/lib/libc/arch/aarch64/atomic/atomic_add_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_8.S: revision 1.4
common/lib/libc/arch/aarch64/atomic/membar_ops.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_16.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_nand_32.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_inc_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_dec_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_and_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_and_64.S: revision 1.2

Part I of ad@'s performance improvements for aarch64
- Remove memory barriers from the atomic ops. I don't understand why thos=
e
are there. Is it some architectural thing, or for a CPU bug, or just
over-caution maybe? They're not needed for correctness.
- Have unlikely conditional branches go forwards to help the static branch
predictor.

Remove memory barriers from the atomic ops macros in the same way as was
done for the other atomic ops earlier.

Use the correct barriers - all of membar_{sync,producer,consumer} have
less scope than before.

LGTM from riastradh

As we're providing the legacy gcc __sync built-in functions for atomic
memory access we might as well get the memory barriers right...

From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)
This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)
This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.1.26.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.1.26.1 21-Apr-2020  martin Sync with HEAD
 1.1.4.2 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.1.4.1 10-Aug-2014  tls file atomic_or_8.S was added on branch tls-maxphys on 2014-08-19 23:45:12 +0000
 1.2 29-Jul-2021  skrll As we're providing the legacy gcc __sync built-in functions for atomic
memory access we might as well get the memory barriers right...
From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)

This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)

This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.1 10-Aug-2014  matt branches: 1.1.4; 1.1.26; 1.1.28;
Preliminary files for AARCH64 (64-bit ARM) support.
Enough for a distribution build.
 1.1.28.1 11-Aug-2021  martin Pull up following revision(s) (requested by skrll in ticket #1331):

common/lib/libc/arch/aarch64/atomic/atomic_add_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_sub_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_sub_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_64.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_inc_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_or_64.S: revision 1.3
common/lib/libc/arch/aarch64/atomic/atomic_and_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_xor_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_and_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_add_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_op_asm.h: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_swap_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_op_asm.h: revision 1.6
common/lib/libc/arch/aarch64/atomic/atomic_cas_64.S: revision 1.4
common/lib/libc/arch/aarch64/atomic/atomic_nand_8.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_sub_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_64.S: revision 1.6
common/lib/libc/arch/aarch64/atomic/atomic_swap_8.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_sub_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_xor_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_32.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_or_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_64.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_swap_16.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_or_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_or_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_xor_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_16.S: revision 1.4
common/lib/libc/arch/aarch64/atomic/atomic_add_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_xor_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_dec_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_32.S: revision 1.4
common/lib/libc/arch/aarch64/atomic/atomic_add_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_8.S: revision 1.4
common/lib/libc/arch/aarch64/atomic/membar_ops.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_16.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_nand_32.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_inc_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_dec_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_and_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_and_64.S: revision 1.2

Part I of ad@'s performance improvements for aarch64
- Remove memory barriers from the atomic ops. I don't understand why thos=
e
are there. Is it some architectural thing, or for a CPU bug, or just
over-caution maybe? They're not needed for correctness.
- Have unlikely conditional branches go forwards to help the static branch
predictor.

Remove memory barriers from the atomic ops macros in the same way as was
done for the other atomic ops earlier.

Use the correct barriers - all of membar_{sync,producer,consumer} have
less scope than before.

LGTM from riastradh

As we're providing the legacy gcc __sync built-in functions for atomic
memory access we might as well get the memory barriers right...

From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)
This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)
This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.1.26.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.1.26.1 21-Apr-2020  martin Sync with HEAD
 1.1.4.2 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.1.4.1 10-Aug-2014  tls file atomic_sub_16.S was added on branch tls-maxphys on 2014-08-19 23:45:12 +0000
 1.2 29-Jul-2021  skrll As we're providing the legacy gcc __sync built-in functions for atomic
memory access we might as well get the memory barriers right...
From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)

This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)

This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.1 10-Aug-2014  matt branches: 1.1.4; 1.1.26; 1.1.28;
Preliminary files for AARCH64 (64-bit ARM) support.
Enough for a distribution build.
 1.1.28.1 11-Aug-2021  martin Pull up following revision(s) (requested by skrll in ticket #1331):

common/lib/libc/arch/aarch64/atomic/atomic_add_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_sub_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_sub_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_64.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_inc_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_or_64.S: revision 1.3
common/lib/libc/arch/aarch64/atomic/atomic_and_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_xor_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_and_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_add_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_op_asm.h: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_swap_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_op_asm.h: revision 1.6
common/lib/libc/arch/aarch64/atomic/atomic_cas_64.S: revision 1.4
common/lib/libc/arch/aarch64/atomic/atomic_nand_8.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_sub_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_64.S: revision 1.6
common/lib/libc/arch/aarch64/atomic/atomic_swap_8.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_sub_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_xor_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_32.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_or_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_64.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_swap_16.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_or_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_or_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_xor_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_16.S: revision 1.4
common/lib/libc/arch/aarch64/atomic/atomic_add_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_xor_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_dec_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_32.S: revision 1.4
common/lib/libc/arch/aarch64/atomic/atomic_add_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_8.S: revision 1.4
common/lib/libc/arch/aarch64/atomic/membar_ops.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_16.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_nand_32.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_inc_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_dec_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_and_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_and_64.S: revision 1.2

Part I of ad@'s performance improvements for aarch64
- Remove memory barriers from the atomic ops. I don't understand why thos=
e
are there. Is it some architectural thing, or for a CPU bug, or just
over-caution maybe? They're not needed for correctness.
- Have unlikely conditional branches go forwards to help the static branch
predictor.

Remove memory barriers from the atomic ops macros in the same way as was
done for the other atomic ops earlier.

Use the correct barriers - all of membar_{sync,producer,consumer} have
less scope than before.

LGTM from riastradh

As we're providing the legacy gcc __sync built-in functions for atomic
memory access we might as well get the memory barriers right...

From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)
This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)
This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.1.26.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.1.26.1 21-Apr-2020  martin Sync with HEAD
 1.1.4.2 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.1.4.1 10-Aug-2014  tls file atomic_sub_32.S was added on branch tls-maxphys on 2014-08-19 23:45:12 +0000
 1.2 29-Jul-2021  skrll As we're providing the legacy gcc __sync built-in functions for atomic
memory access we might as well get the memory barriers right...
From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)

This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)

This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.1 10-Aug-2014  matt branches: 1.1.4; 1.1.26; 1.1.28;
Preliminary files for AARCH64 (64-bit ARM) support.
Enough for a distribution build.
 1.1.28.1 11-Aug-2021  martin Pull up following revision(s) (requested by skrll in ticket #1331):

common/lib/libc/arch/aarch64/atomic/atomic_add_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_sub_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_sub_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_64.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_inc_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_or_64.S: revision 1.3
common/lib/libc/arch/aarch64/atomic/atomic_and_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_xor_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_and_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_add_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_op_asm.h: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_swap_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_op_asm.h: revision 1.6
common/lib/libc/arch/aarch64/atomic/atomic_cas_64.S: revision 1.4
common/lib/libc/arch/aarch64/atomic/atomic_nand_8.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_sub_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_64.S: revision 1.6
common/lib/libc/arch/aarch64/atomic/atomic_swap_8.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_sub_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_xor_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_32.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_or_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_64.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_swap_16.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_or_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_or_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_xor_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_16.S: revision 1.4
common/lib/libc/arch/aarch64/atomic/atomic_add_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_xor_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_dec_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_32.S: revision 1.4
common/lib/libc/arch/aarch64/atomic/atomic_add_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_8.S: revision 1.4
common/lib/libc/arch/aarch64/atomic/membar_ops.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_16.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_nand_32.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_inc_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_dec_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_and_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_and_64.S: revision 1.2

Part I of ad@'s performance improvements for aarch64
- Remove memory barriers from the atomic ops. I don't understand why thos=
e
are there. Is it some architectural thing, or for a CPU bug, or just
over-caution maybe? They're not needed for correctness.
- Have unlikely conditional branches go forwards to help the static branch
predictor.

Remove memory barriers from the atomic ops macros in the same way as was
done for the other atomic ops earlier.

Use the correct barriers - all of membar_{sync,producer,consumer} have
less scope than before.

LGTM from riastradh

As we're providing the legacy gcc __sync built-in functions for atomic
memory access we might as well get the memory barriers right...

From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)
This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)
This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.1.26.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.1.26.1 21-Apr-2020  martin Sync with HEAD
 1.1.4.2 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.1.4.1 10-Aug-2014  tls file atomic_sub_64.S was added on branch tls-maxphys on 2014-08-19 23:45:12 +0000
 1.2 29-Jul-2021  skrll As we're providing the legacy gcc __sync built-in functions for atomic
memory access we might as well get the memory barriers right...
From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)

This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)

This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.1 10-Aug-2014  matt branches: 1.1.4; 1.1.26; 1.1.28;
Preliminary files for AARCH64 (64-bit ARM) support.
Enough for a distribution build.
 1.1.28.1 11-Aug-2021  martin Pull up following revision(s) (requested by skrll in ticket #1331):

common/lib/libc/arch/aarch64/atomic/atomic_add_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_sub_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_sub_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_64.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_inc_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_or_64.S: revision 1.3
common/lib/libc/arch/aarch64/atomic/atomic_and_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_xor_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_and_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_add_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_op_asm.h: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_swap_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_op_asm.h: revision 1.6
common/lib/libc/arch/aarch64/atomic/atomic_cas_64.S: revision 1.4
common/lib/libc/arch/aarch64/atomic/atomic_nand_8.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_sub_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_64.S: revision 1.6
common/lib/libc/arch/aarch64/atomic/atomic_swap_8.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_sub_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_xor_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_32.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_or_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_64.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_swap_16.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_or_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_or_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_xor_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_16.S: revision 1.4
common/lib/libc/arch/aarch64/atomic/atomic_add_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_xor_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_dec_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_32.S: revision 1.4
common/lib/libc/arch/aarch64/atomic/atomic_add_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_8.S: revision 1.4
common/lib/libc/arch/aarch64/atomic/membar_ops.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_16.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_nand_32.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_inc_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_dec_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_and_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_and_64.S: revision 1.2

Part I of ad@'s performance improvements for aarch64
- Remove memory barriers from the atomic ops. I don't understand why thos=
e
are there. Is it some architectural thing, or for a CPU bug, or just
over-caution maybe? They're not needed for correctness.
- Have unlikely conditional branches go forwards to help the static branch
predictor.

Remove memory barriers from the atomic ops macros in the same way as was
done for the other atomic ops earlier.

Use the correct barriers - all of membar_{sync,producer,consumer} have
less scope than before.

LGTM from riastradh

As we're providing the legacy gcc __sync built-in functions for atomic
memory access we might as well get the memory barriers right...

From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)
This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)
This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.1.26.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.1.26.1 21-Apr-2020  martin Sync with HEAD
 1.1.4.2 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.1.4.1 10-Aug-2014  tls file atomic_sub_8.S was added on branch tls-maxphys on 2014-08-19 23:45:12 +0000
 1.5 29-Jul-2021  skrll As we're providing the legacy gcc __sync built-in functions for atomic
memory access we might as well get the memory barriers right...
From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)

This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)

This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.4 21-Apr-2021  skrll Do previous differently as the API is different.
 1.3 21-Apr-2021  skrll Provide some more operations that are part of compiler lse.S. This is
incomplete, but at least covers all the atomic_swap ops and allows the
aa64 kernel to link with gcc 10.
 1.2 12-Aug-2020  skrll Part I of ad@'s performance improvements for aarch64

- Remove memory barriers from the atomic ops. I don't understand why those
are there. Is it some architectural thing, or for a CPU bug, or just
over-caution maybe? They're not needed for correctness.

- Have unlikely conditional branches go forwards to help the static branch
predictor.
 1.1 10-Aug-2014  matt branches: 1.1.4; 1.1.26; 1.1.28;
Preliminary files for AARCH64 (64-bit ARM) support.
Enough for a distribution build.
 1.1.28.1 11-Aug-2021  martin Pull up following revision(s) (requested by skrll in ticket #1331):

common/lib/libc/arch/aarch64/atomic/atomic_add_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_sub_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_sub_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_64.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_inc_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_or_64.S: revision 1.3
common/lib/libc/arch/aarch64/atomic/atomic_and_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_xor_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_and_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_add_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_op_asm.h: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_swap_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_op_asm.h: revision 1.6
common/lib/libc/arch/aarch64/atomic/atomic_cas_64.S: revision 1.4
common/lib/libc/arch/aarch64/atomic/atomic_nand_8.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_sub_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_64.S: revision 1.6
common/lib/libc/arch/aarch64/atomic/atomic_swap_8.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_sub_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_xor_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_32.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_or_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_64.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_swap_16.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_or_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_or_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_xor_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_16.S: revision 1.4
common/lib/libc/arch/aarch64/atomic/atomic_add_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_xor_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_dec_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_32.S: revision 1.4
common/lib/libc/arch/aarch64/atomic/atomic_add_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_8.S: revision 1.4
common/lib/libc/arch/aarch64/atomic/membar_ops.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_16.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_nand_32.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_inc_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_dec_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_and_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_and_64.S: revision 1.2

Part I of ad@'s performance improvements for aarch64
- Remove memory barriers from the atomic ops. I don't understand why thos=
e
are there. Is it some architectural thing, or for a CPU bug, or just
over-caution maybe? They're not needed for correctness.
- Have unlikely conditional branches go forwards to help the static branch
predictor.

Remove memory barriers from the atomic ops macros in the same way as was
done for the other atomic ops earlier.

Use the correct barriers - all of membar_{sync,producer,consumer} have
less scope than before.

LGTM from riastradh

As we're providing the legacy gcc __sync built-in functions for atomic
memory access we might as well get the memory barriers right...

From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)
This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)
This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.1.26.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.1.26.1 21-Apr-2020  martin Sync with HEAD
 1.1.4.2 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.1.4.1 10-Aug-2014  tls file atomic_swap_16.S was added on branch tls-maxphys on 2014-08-19 23:45:12 +0000
 1.5 29-Jul-2021  skrll As we're providing the legacy gcc __sync built-in functions for atomic
memory access we might as well get the memory barriers right...
From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)

This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)

This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.4 21-Apr-2021  skrll Do previous differently as the API is different.
 1.3 21-Apr-2021  skrll Provide some more operations that are part of compiler lse.S. This is
incomplete, but at least covers all the atomic_swap ops and allows the
aa64 kernel to link with gcc 10.
 1.2 12-Aug-2020  skrll Part I of ad@'s performance improvements for aarch64

- Remove memory barriers from the atomic ops. I don't understand why those
are there. Is it some architectural thing, or for a CPU bug, or just
over-caution maybe? They're not needed for correctness.

- Have unlikely conditional branches go forwards to help the static branch
predictor.
 1.1 10-Aug-2014  matt branches: 1.1.4; 1.1.26; 1.1.28;
Preliminary files for AARCH64 (64-bit ARM) support.
Enough for a distribution build.
 1.1.28.1 11-Aug-2021  martin Pull up following revision(s) (requested by skrll in ticket #1331):

common/lib/libc/arch/aarch64/atomic/atomic_add_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_sub_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_sub_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_64.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_inc_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_or_64.S: revision 1.3
common/lib/libc/arch/aarch64/atomic/atomic_and_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_xor_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_and_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_add_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_op_asm.h: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_swap_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_op_asm.h: revision 1.6
common/lib/libc/arch/aarch64/atomic/atomic_cas_64.S: revision 1.4
common/lib/libc/arch/aarch64/atomic/atomic_nand_8.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_sub_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_64.S: revision 1.6
common/lib/libc/arch/aarch64/atomic/atomic_swap_8.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_sub_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_xor_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_32.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_or_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_64.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_swap_16.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_or_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_or_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_xor_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_16.S: revision 1.4
common/lib/libc/arch/aarch64/atomic/atomic_add_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_xor_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_dec_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_32.S: revision 1.4
common/lib/libc/arch/aarch64/atomic/atomic_add_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_8.S: revision 1.4
common/lib/libc/arch/aarch64/atomic/membar_ops.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_16.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_nand_32.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_inc_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_dec_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_and_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_and_64.S: revision 1.2

Part I of ad@'s performance improvements for aarch64
- Remove memory barriers from the atomic ops. I don't understand why thos=
e
are there. Is it some architectural thing, or for a CPU bug, or just
over-caution maybe? They're not needed for correctness.
- Have unlikely conditional branches go forwards to help the static branch
predictor.

Remove memory barriers from the atomic ops macros in the same way as was
done for the other atomic ops earlier.

Use the correct barriers - all of membar_{sync,producer,consumer} have
less scope than before.

LGTM from riastradh

As we're providing the legacy gcc __sync built-in functions for atomic
memory access we might as well get the memory barriers right...

From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)
This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)
This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.1.26.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.1.26.1 21-Apr-2020  martin Sync with HEAD
 1.1.4.2 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.1.4.1 10-Aug-2014  tls file atomic_swap_32.S was added on branch tls-maxphys on 2014-08-19 23:45:12 +0000
 1.5 29-Jul-2021  skrll As we're providing the legacy gcc __sync built-in functions for atomic
memory access we might as well get the memory barriers right...
From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)

This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)

This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.4 21-Apr-2021  skrll Do previous differently as the API is different.
 1.3 21-Apr-2021  skrll Provide some more operations that are part of compiler lse.S. This is
incomplete, but at least covers all the atomic_swap ops and allows the
aa64 kernel to link with gcc 10.
 1.2 12-Aug-2020  skrll Part I of ad@'s performance improvements for aarch64

- Remove memory barriers from the atomic ops. I don't understand why those
are there. Is it some architectural thing, or for a CPU bug, or just
over-caution maybe? They're not needed for correctness.

- Have unlikely conditional branches go forwards to help the static branch
predictor.
 1.1 10-Aug-2014  matt branches: 1.1.4; 1.1.26; 1.1.28;
Preliminary files for AARCH64 (64-bit ARM) support.
Enough for a distribution build.
 1.1.28.1 11-Aug-2021  martin Pull up following revision(s) (requested by skrll in ticket #1331):

common/lib/libc/arch/aarch64/atomic/atomic_add_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_sub_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_sub_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_64.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_inc_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_or_64.S: revision 1.3
common/lib/libc/arch/aarch64/atomic/atomic_and_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_xor_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_and_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_add_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_op_asm.h: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_swap_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_op_asm.h: revision 1.6
common/lib/libc/arch/aarch64/atomic/atomic_cas_64.S: revision 1.4
common/lib/libc/arch/aarch64/atomic/atomic_nand_8.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_sub_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_64.S: revision 1.6
common/lib/libc/arch/aarch64/atomic/atomic_swap_8.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_sub_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_xor_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_32.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_or_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_64.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_swap_16.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_or_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_or_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_xor_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_16.S: revision 1.4
common/lib/libc/arch/aarch64/atomic/atomic_add_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_xor_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_dec_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_32.S: revision 1.4
common/lib/libc/arch/aarch64/atomic/atomic_add_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_8.S: revision 1.4
common/lib/libc/arch/aarch64/atomic/membar_ops.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_16.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_nand_32.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_inc_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_dec_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_and_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_and_64.S: revision 1.2

Part I of ad@'s performance improvements for aarch64
- Remove memory barriers from the atomic ops. I don't understand why thos=
e
are there. Is it some architectural thing, or for a CPU bug, or just
over-caution maybe? They're not needed for correctness.
- Have unlikely conditional branches go forwards to help the static branch
predictor.

Remove memory barriers from the atomic ops macros in the same way as was
done for the other atomic ops earlier.

Use the correct barriers - all of membar_{sync,producer,consumer} have
less scope than before.

LGTM from riastradh

As we're providing the legacy gcc __sync built-in functions for atomic
memory access we might as well get the memory barriers right...

From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)
This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)
This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.1.26.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.1.26.1 21-Apr-2020  martin Sync with HEAD
 1.1.4.2 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.1.4.1 10-Aug-2014  tls file atomic_swap_64.S was added on branch tls-maxphys on 2014-08-19 23:45:12 +0000
 1.5 29-Jul-2021  skrll As we're providing the legacy gcc __sync built-in functions for atomic
memory access we might as well get the memory barriers right...
From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)

This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)

This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.4 21-Apr-2021  skrll Do previous differently as the API is different.
 1.3 21-Apr-2021  skrll Provide some more operations that are part of compiler lse.S. This is
incomplete, but at least covers all the atomic_swap ops and allows the
aa64 kernel to link with gcc 10.
 1.2 12-Aug-2020  skrll Part I of ad@'s performance improvements for aarch64

- Remove memory barriers from the atomic ops. I don't understand why those
are there. Is it some architectural thing, or for a CPU bug, or just
over-caution maybe? They're not needed for correctness.

- Have unlikely conditional branches go forwards to help the static branch
predictor.
 1.1 10-Aug-2014  matt branches: 1.1.4; 1.1.26; 1.1.28;
Preliminary files for AARCH64 (64-bit ARM) support.
Enough for a distribution build.
 1.1.28.1 11-Aug-2021  martin Pull up following revision(s) (requested by skrll in ticket #1331):

common/lib/libc/arch/aarch64/atomic/atomic_add_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_sub_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_sub_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_64.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_inc_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_or_64.S: revision 1.3
common/lib/libc/arch/aarch64/atomic/atomic_and_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_xor_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_and_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_add_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_op_asm.h: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_swap_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_op_asm.h: revision 1.6
common/lib/libc/arch/aarch64/atomic/atomic_cas_64.S: revision 1.4
common/lib/libc/arch/aarch64/atomic/atomic_nand_8.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_sub_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_64.S: revision 1.6
common/lib/libc/arch/aarch64/atomic/atomic_swap_8.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_sub_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_xor_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_32.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_or_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_64.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_swap_16.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_or_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_or_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_xor_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_16.S: revision 1.4
common/lib/libc/arch/aarch64/atomic/atomic_add_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_xor_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_dec_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_32.S: revision 1.4
common/lib/libc/arch/aarch64/atomic/atomic_add_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_8.S: revision 1.4
common/lib/libc/arch/aarch64/atomic/membar_ops.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_16.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_nand_32.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_inc_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_dec_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_and_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_and_64.S: revision 1.2

Part I of ad@'s performance improvements for aarch64
- Remove memory barriers from the atomic ops. I don't understand why thos=
e
are there. Is it some architectural thing, or for a CPU bug, or just
over-caution maybe? They're not needed for correctness.
- Have unlikely conditional branches go forwards to help the static branch
predictor.

Remove memory barriers from the atomic ops macros in the same way as was
done for the other atomic ops earlier.

Use the correct barriers - all of membar_{sync,producer,consumer} have
less scope than before.

LGTM from riastradh

As we're providing the legacy gcc __sync built-in functions for atomic
memory access we might as well get the memory barriers right...

From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)
This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)
This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.1.26.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.1.26.1 21-Apr-2020  martin Sync with HEAD
 1.1.4.2 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.1.4.1 10-Aug-2014  tls file atomic_swap_8.S was added on branch tls-maxphys on 2014-08-19 23:45:12 +0000
 1.2 21-Apr-2021  skrll Do previous differently as the API is different.
 1.1 21-Apr-2021  skrll Provide some more operations that are part of compiler lse.S. This is
incomplete, but at least covers all the atomic_swap ops and allows the
aa64 kernel to link with gcc 10.
 1.2 21-Apr-2021  skrll Do previous differently as the API is different.
 1.1 21-Apr-2021  skrll Provide some more operations that are part of compiler lse.S. This is
incomplete, but at least covers all the atomic_swap ops and allows the
aa64 kernel to link with gcc 10.
 1.2 21-Apr-2021  skrll Do previous differently as the API is different.
 1.1 21-Apr-2021  skrll Provide some more operations that are part of compiler lse.S. This is
incomplete, but at least covers all the atomic_swap ops and allows the
aa64 kernel to link with gcc 10.
 1.2 21-Apr-2021  skrll Do previous differently as the API is different.
 1.1 21-Apr-2021  skrll Provide some more operations that are part of compiler lse.S. This is
incomplete, but at least covers all the atomic_swap ops and allows the
aa64 kernel to link with gcc 10.
 1.2 21-Apr-2021  skrll Do previous differently as the API is different.
 1.1 21-Apr-2021  skrll Provide some more operations that are part of compiler lse.S. This is
incomplete, but at least covers all the atomic_swap ops and allows the
aa64 kernel to link with gcc 10.
 1.2 21-Apr-2021  skrll Do previous differently as the API is different.
 1.1 21-Apr-2021  skrll Provide some more operations that are part of compiler lse.S. This is
incomplete, but at least covers all the atomic_swap ops and allows the
aa64 kernel to link with gcc 10.
 1.2 21-Apr-2021  skrll Do previous differently as the API is different.
 1.1 21-Apr-2021  skrll Provide some more operations that are part of compiler lse.S. This is
incomplete, but at least covers all the atomic_swap ops and allows the
aa64 kernel to link with gcc 10.
 1.2 21-Apr-2021  skrll Do previous differently as the API is different.
 1.1 21-Apr-2021  skrll Provide some more operations that are part of compiler lse.S. This is
incomplete, but at least covers all the atomic_swap ops and allows the
aa64 kernel to link with gcc 10.
 1.2 21-Apr-2021  skrll Do previous differently as the API is different.
 1.1 21-Apr-2021  skrll Provide some more operations that are part of compiler lse.S. This is
incomplete, but at least covers all the atomic_swap ops and allows the
aa64 kernel to link with gcc 10.
 1.2 21-Apr-2021  skrll Do previous differently as the API is different.
 1.1 21-Apr-2021  skrll Provide some more operations that are part of compiler lse.S. This is
incomplete, but at least covers all the atomic_swap ops and allows the
aa64 kernel to link with gcc 10.
 1.2 21-Apr-2021  skrll Do previous differently as the API is different.
 1.1 21-Apr-2021  skrll Provide some more operations that are part of compiler lse.S. This is
incomplete, but at least covers all the atomic_swap ops and allows the
aa64 kernel to link with gcc 10.
 1.2 21-Apr-2021  skrll Do previous differently as the API is different.
 1.1 21-Apr-2021  skrll Provide some more operations that are part of compiler lse.S. This is
incomplete, but at least covers all the atomic_swap ops and allows the
aa64 kernel to link with gcc 10.
 1.2 29-Jul-2021  skrll As we're providing the legacy gcc __sync built-in functions for atomic
memory access we might as well get the memory barriers right...
From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)

This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)

This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.1 10-Aug-2014  matt branches: 1.1.4; 1.1.26; 1.1.28;
Preliminary files for AARCH64 (64-bit ARM) support.
Enough for a distribution build.
 1.1.28.1 11-Aug-2021  martin Pull up following revision(s) (requested by skrll in ticket #1331):

common/lib/libc/arch/aarch64/atomic/atomic_add_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_sub_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_sub_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_64.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_inc_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_or_64.S: revision 1.3
common/lib/libc/arch/aarch64/atomic/atomic_and_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_xor_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_and_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_add_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_op_asm.h: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_swap_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_op_asm.h: revision 1.6
common/lib/libc/arch/aarch64/atomic/atomic_cas_64.S: revision 1.4
common/lib/libc/arch/aarch64/atomic/atomic_nand_8.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_sub_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_64.S: revision 1.6
common/lib/libc/arch/aarch64/atomic/atomic_swap_8.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_sub_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_xor_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_32.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_or_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_64.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_swap_16.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_or_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_or_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_xor_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_16.S: revision 1.4
common/lib/libc/arch/aarch64/atomic/atomic_add_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_xor_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_dec_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_32.S: revision 1.4
common/lib/libc/arch/aarch64/atomic/atomic_add_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_8.S: revision 1.4
common/lib/libc/arch/aarch64/atomic/membar_ops.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_16.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_nand_32.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_inc_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_dec_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_and_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_and_64.S: revision 1.2

Part I of ad@'s performance improvements for aarch64
- Remove memory barriers from the atomic ops. I don't understand why thos=
e
are there. Is it some architectural thing, or for a CPU bug, or just
over-caution maybe? They're not needed for correctness.
- Have unlikely conditional branches go forwards to help the static branch
predictor.

Remove memory barriers from the atomic ops macros in the same way as was
done for the other atomic ops earlier.

Use the correct barriers - all of membar_{sync,producer,consumer} have
less scope than before.

LGTM from riastradh

As we're providing the legacy gcc __sync built-in functions for atomic
memory access we might as well get the memory barriers right...

From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)
This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)
This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.1.26.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.1.26.1 21-Apr-2020  martin Sync with HEAD
 1.1.4.2 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.1.4.1 10-Aug-2014  tls file atomic_xor_16.S was added on branch tls-maxphys on 2014-08-19 23:45:12 +0000
 1.2 29-Jul-2021  skrll As we're providing the legacy gcc __sync built-in functions for atomic
memory access we might as well get the memory barriers right...
From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)

This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)

This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.1 10-Aug-2014  matt branches: 1.1.4; 1.1.26; 1.1.28;
Preliminary files for AARCH64 (64-bit ARM) support.
Enough for a distribution build.
 1.1.28.1 11-Aug-2021  martin Pull up following revision(s) (requested by skrll in ticket #1331):

common/lib/libc/arch/aarch64/atomic/atomic_add_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_sub_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_sub_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_64.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_inc_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_or_64.S: revision 1.3
common/lib/libc/arch/aarch64/atomic/atomic_and_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_xor_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_and_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_add_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_op_asm.h: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_swap_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_op_asm.h: revision 1.6
common/lib/libc/arch/aarch64/atomic/atomic_cas_64.S: revision 1.4
common/lib/libc/arch/aarch64/atomic/atomic_nand_8.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_sub_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_64.S: revision 1.6
common/lib/libc/arch/aarch64/atomic/atomic_swap_8.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_sub_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_xor_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_32.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_or_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_64.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_swap_16.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_or_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_or_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_xor_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_16.S: revision 1.4
common/lib/libc/arch/aarch64/atomic/atomic_add_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_xor_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_dec_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_32.S: revision 1.4
common/lib/libc/arch/aarch64/atomic/atomic_add_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_8.S: revision 1.4
common/lib/libc/arch/aarch64/atomic/membar_ops.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_16.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_nand_32.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_inc_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_dec_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_and_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_and_64.S: revision 1.2

Part I of ad@'s performance improvements for aarch64
- Remove memory barriers from the atomic ops. I don't understand why thos=
e
are there. Is it some architectural thing, or for a CPU bug, or just
over-caution maybe? They're not needed for correctness.
- Have unlikely conditional branches go forwards to help the static branch
predictor.

Remove memory barriers from the atomic ops macros in the same way as was
done for the other atomic ops earlier.

Use the correct barriers - all of membar_{sync,producer,consumer} have
less scope than before.

LGTM from riastradh

As we're providing the legacy gcc __sync built-in functions for atomic
memory access we might as well get the memory barriers right...

From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)
This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)
This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.1.26.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.1.26.1 21-Apr-2020  martin Sync with HEAD
 1.1.4.2 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.1.4.1 10-Aug-2014  tls file atomic_xor_32.S was added on branch tls-maxphys on 2014-08-19 23:45:12 +0000
 1.2 29-Jul-2021  skrll As we're providing the legacy gcc __sync built-in functions for atomic
memory access we might as well get the memory barriers right...
From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)

This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)

This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.1 10-Aug-2014  matt branches: 1.1.4; 1.1.26; 1.1.28;
Preliminary files for AARCH64 (64-bit ARM) support.
Enough for a distribution build.
 1.1.28.1 11-Aug-2021  martin Pull up following revision(s) (requested by skrll in ticket #1331):

common/lib/libc/arch/aarch64/atomic/atomic_add_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_sub_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_sub_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_64.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_inc_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_or_64.S: revision 1.3
common/lib/libc/arch/aarch64/atomic/atomic_and_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_xor_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_and_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_add_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_op_asm.h: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_swap_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_op_asm.h: revision 1.6
common/lib/libc/arch/aarch64/atomic/atomic_cas_64.S: revision 1.4
common/lib/libc/arch/aarch64/atomic/atomic_nand_8.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_sub_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_64.S: revision 1.6
common/lib/libc/arch/aarch64/atomic/atomic_swap_8.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_sub_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_xor_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_32.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_or_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_64.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_swap_16.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_or_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_or_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_xor_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_16.S: revision 1.4
common/lib/libc/arch/aarch64/atomic/atomic_add_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_xor_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_dec_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_32.S: revision 1.4
common/lib/libc/arch/aarch64/atomic/atomic_add_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_8.S: revision 1.4
common/lib/libc/arch/aarch64/atomic/membar_ops.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_16.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_nand_32.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_inc_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_dec_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_and_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_and_64.S: revision 1.2

Part I of ad@'s performance improvements for aarch64
- Remove memory barriers from the atomic ops. I don't understand why thos=
e
are there. Is it some architectural thing, or for a CPU bug, or just
over-caution maybe? They're not needed for correctness.
- Have unlikely conditional branches go forwards to help the static branch
predictor.

Remove memory barriers from the atomic ops macros in the same way as was
done for the other atomic ops earlier.

Use the correct barriers - all of membar_{sync,producer,consumer} have
less scope than before.

LGTM from riastradh

As we're providing the legacy gcc __sync built-in functions for atomic
memory access we might as well get the memory barriers right...

From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)
This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)
This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.1.26.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.1.26.1 21-Apr-2020  martin Sync with HEAD
 1.1.4.2 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.1.4.1 10-Aug-2014  tls file atomic_xor_64.S was added on branch tls-maxphys on 2014-08-19 23:45:12 +0000
 1.2 29-Jul-2021  skrll As we're providing the legacy gcc __sync built-in functions for atomic
memory access we might as well get the memory barriers right...
From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)

This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)

This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.1 10-Aug-2014  matt branches: 1.1.4; 1.1.26; 1.1.28;
Preliminary files for AARCH64 (64-bit ARM) support.
Enough for a distribution build.
 1.1.28.1 11-Aug-2021  martin Pull up following revision(s) (requested by skrll in ticket #1331):

common/lib/libc/arch/aarch64/atomic/atomic_add_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_sub_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_sub_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_64.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_inc_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_or_64.S: revision 1.3
common/lib/libc/arch/aarch64/atomic/atomic_and_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_xor_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_and_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_add_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_op_asm.h: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_swap_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_op_asm.h: revision 1.6
common/lib/libc/arch/aarch64/atomic/atomic_cas_64.S: revision 1.4
common/lib/libc/arch/aarch64/atomic/atomic_nand_8.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_sub_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_64.S: revision 1.6
common/lib/libc/arch/aarch64/atomic/atomic_swap_8.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_sub_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_xor_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_32.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_or_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_64.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_swap_16.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_or_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_or_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_xor_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_16.S: revision 1.4
common/lib/libc/arch/aarch64/atomic/atomic_add_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_xor_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_dec_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_32.S: revision 1.4
common/lib/libc/arch/aarch64/atomic/atomic_add_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_8.S: revision 1.4
common/lib/libc/arch/aarch64/atomic/membar_ops.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_16.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_nand_32.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_inc_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_dec_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_and_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_and_64.S: revision 1.2

Part I of ad@'s performance improvements for aarch64
- Remove memory barriers from the atomic ops. I don't understand why thos=
e
are there. Is it some architectural thing, or for a CPU bug, or just
over-caution maybe? They're not needed for correctness.
- Have unlikely conditional branches go forwards to help the static branch
predictor.

Remove memory barriers from the atomic ops macros in the same way as was
done for the other atomic ops earlier.

Use the correct barriers - all of membar_{sync,producer,consumer} have
less scope than before.

LGTM from riastradh

As we're providing the legacy gcc __sync built-in functions for atomic
memory access we might as well get the memory barriers right...

From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)
This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)
This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.1.26.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.1.26.1 21-Apr-2020  martin Sync with HEAD
 1.1.4.2 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.1.4.1 10-Aug-2014  tls file atomic_xor_8.S was added on branch tls-maxphys on 2014-08-19 23:45:12 +0000
 1.4 09-Apr-2022  riastradh Introduce membar_acquire/release. Deprecate membar_enter/exit.

The names membar_enter/exit were unclear, and the documentation of
membar_enter has disagreed with the implementations on sparc,
powerpc, and even x86(!) for the entire time it has been in NetBSD.

The terms `acquire' and `release' are ubiquitous in the literature
today, and have been adopted in the C and C++ standards to mean
load-before-load/store and load/store-before-store, respectively,
which are exactly the orderings required by acquiring and releasing a
mutex, as well as other useful applications like decrementing a
reference count and then freeing the underlying object if it went to
zero.

Originally I proposed changing one word in the documentation for
membar_enter to make it load-before-load/store instead of
store-before-load/store, i.e., to make it an acquire barrier. I
proposed this on the grounds that

(a) all implementations guarantee load-before-load/store,
(b) some implementations fail to guarantee store-before-load/store,
and
(c) all uses in-tree assume load-before-load/store.

I verified parts (a) and (b) (except, for (a), powerpc didn't even
guarantee load-before-load/store -- isync isn't necessarily enough;
need lwsync in general -- but it _almost_ did, and it certainly didn't
guarantee store-before-load/store).

Part (c) might not be correct, however: under the mistaken assumption
that atomic-r/m/w then membar-w/rw is equivalent to atomic-r/m/w then
membar-r/rw, I only audited the cases of membar_enter that _aren't_
immediately after an atomic-r/m/w. All of those cases assume
load-before-load/store. But my assumption was wrong -- there are
cases of atomic-r/m/w then membar-w/rw that would be broken by
changing to atomic-r/m/w then membar-r/rw:

https://mail-index.netbsd.org/tech-kern/2022/03/29/msg028044.html

Furthermore, the name membar_enter has been adopted in other places
like OpenBSD where it actually does follow the documentation and
guarantee store-before-load/store, even if that order is not useful.
So the name membar_enter currently lives in a bad place where it
means either of two things -- r/rw or w/rw.

With this change, we deprecate membar_enter/exit, introduce
membar_acquire/release as better names for the useful pair (r/rw and
rw/w), and make sure the implementation of membar_enter guarantees
both what was documented _and_ what was implemented, making it an
alias for membar_sync.

While here, rework all of the membar_* definitions and aliases. The
new logic follows a rule to make it easier to audit:

membar_X is defined as an alias for membar_Y iff membar_X is
guaranteed by membar_Y.

The `no stronger than' relation is (the transitive closure of):

- membar_consumer (r/r) is guaranteed by membar_acquire (r/rw)
- membar_producer (w/w) is guaranteed by membar_release (rw/w)
- membar_acquire (r/rw) is guaranteed by membar_sync (rw/rw)
- membar_release (rw/w) is guaranteed by membar_sync (rw/rw)

And, for the deprecated membars:

- membar_enter (whether r/rw, w/rw, or rw/rw) is guaranteed by
membar_sync (rw/rw)
- membar_exit (rw/w) is guaranteed by membar_release (rw/w)

(membar_exit is identical to membar_release, but the name is
deprecated.)

Finally, while here, annotate some of the instructions with their
semantics. For powerpc, leave an essay with citations on the
unfortunate but -- as far as I can tell -- necessary decision to use
lwsync, not isync, for membar_acquire and membar_consumer.

Also add membar(3) and atomic(3) man page links.
 1.3 09-Apr-2022  riastradh aarch64/membar_ops: Fix wrong symbol end.
 1.2 13-Oct-2020  skrll Use the correct barriers - all of membar_{sync,producer,consumer} have
less scope than before.

LGTM from riastradh
 1.1 10-Aug-2014  matt branches: 1.1.4; 1.1.26; 1.1.28;
Preliminary files for AARCH64 (64-bit ARM) support.
Enough for a distribution build.
 1.1.28.1 11-Aug-2021  martin Pull up following revision(s) (requested by skrll in ticket #1331):

common/lib/libc/arch/aarch64/atomic/atomic_add_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_sub_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_sub_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_64.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_inc_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_or_64.S: revision 1.3
common/lib/libc/arch/aarch64/atomic/atomic_and_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_xor_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_and_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_add_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_op_asm.h: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_swap_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_op_asm.h: revision 1.6
common/lib/libc/arch/aarch64/atomic/atomic_cas_64.S: revision 1.4
common/lib/libc/arch/aarch64/atomic/atomic_nand_8.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_sub_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_64.S: revision 1.6
common/lib/libc/arch/aarch64/atomic/atomic_swap_8.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_sub_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_xor_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_swap_32.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_or_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_64.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_swap_16.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_or_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_or_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_xor_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_16.S: revision 1.4
common/lib/libc/arch/aarch64/atomic/atomic_add_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_xor_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_dec_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_32.S: revision 1.4
common/lib/libc/arch/aarch64/atomic/atomic_add_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_cas_8.S: revision 1.4
common/lib/libc/arch/aarch64/atomic/membar_ops.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_16.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_nand_16.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_nand_32.S: revision 1.5
common/lib/libc/arch/aarch64/atomic/atomic_inc_64.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_dec_32.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_and_8.S: revision 1.2
common/lib/libc/arch/aarch64/atomic/atomic_and_64.S: revision 1.2

Part I of ad@'s performance improvements for aarch64
- Remove memory barriers from the atomic ops. I don't understand why thos=
e
are there. Is it some architectural thing, or for a CPU bug, or just
over-caution maybe? They're not needed for correctness.
- Have unlikely conditional branches go forwards to help the static branch
predictor.

Remove memory barriers from the atomic ops macros in the same way as was
done for the other atomic ops earlier.

Use the correct barriers - all of membar_{sync,producer,consumer} have
less scope than before.

LGTM from riastradh

As we're providing the legacy gcc __sync built-in functions for atomic
memory access we might as well get the memory barriers right...

From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)
This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)
This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.1.26.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.1.26.1 21-Apr-2020  martin Sync with HEAD
 1.1.4.2 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.1.4.1 10-Aug-2014  tls file membar_ops.S was added on branch tls-maxphys on 2014-08-19 23:45:12 +0000
 1.1 10-Aug-2014  matt branches: 1.1.4; 1.1.26;
Preliminary files for AARCH64 (64-bit ARM) support.
Enough for a distribution build.
 1.1.26.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.1.26.1 21-Apr-2020  martin Sync with HEAD
 1.1.4.2 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.1.4.1 10-Aug-2014  tls file byte_swap_2.S was added on branch tls-maxphys on 2014-08-19 23:45:12 +0000
 1.1 10-Aug-2014  matt branches: 1.1.4; 1.1.26;
Preliminary files for AARCH64 (64-bit ARM) support.
Enough for a distribution build.
 1.1.26.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.1.26.1 21-Apr-2020  martin Sync with HEAD
 1.1.4.2 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.1.4.1 10-Aug-2014  tls file byte_swap_4.S was added on branch tls-maxphys on 2014-08-19 23:45:12 +0000
 1.2 12-Oct-2019  maya Remove htonll and ntohll as symbols from aarch64 libc.

Other architectures do not define them, and so we don't provide a
function declaration in any header.

This means a package may detect it with a link-test and then fail
due to the missing declaration, like sysutils/collectd currently does.

Done this way as aarch64 has not had a release yet. Discussed with releng.
 1.1 10-Aug-2014  matt branches: 1.1.4; 1.1.26; 1.1.28;
Preliminary files for AARCH64 (64-bit ARM) support.
Enough for a distribution build.
 1.1.28.1 12-Oct-2019  martin Pull up following revision(s) (requested by maya in ticket #304):

common/lib/libc/arch/aarch64/gen/byte_swap_8.S: revision 1.2

Remove htonll and ntohll as symbols from aarch64 libc.

Other architectures do not define them, and so we don't provide a
function declaration in any header.

This means a package may detect it with a link-test and then fail
due to the missing declaration, like sysutils/collectd currently does.

Done this way as aarch64 has not had a release yet. Discussed with releng.
 1.1.26.2 21-Apr-2020  martin Sync with HEAD
 1.1.26.1 13-Apr-2020  martin Mostly merge changes from HEAD upto 20200411
 1.1.4.2 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.1.4.1 10-Aug-2014  tls file byte_swap_8.S was added on branch tls-maxphys on 2014-08-19 23:45:12 +0000
 1.3 03-Sep-2020  jakllsch Remove unused assembly source files
 1.2 02-Sep-2020  jakllsch Fix typo/pasteo in aarch64 clzdi2() END()
 1.1 10-Aug-2014  matt branches: 1.1.4; 1.1.26;
Preliminary files for AARCH64 (64-bit ARM) support.
Enough for a distribution build.
 1.1.26.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.1.26.1 21-Apr-2020  martin Sync with HEAD
 1.1.4.2 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.1.4.1 10-Aug-2014  tls file clzdi2.S was added on branch tls-maxphys on 2014-08-19 23:45:12 +0000
 1.2 03-Sep-2020  jakllsch Remove unused assembly source files
 1.1 10-Aug-2014  matt branches: 1.1.4; 1.1.26;
Preliminary files for AARCH64 (64-bit ARM) support.
Enough for a distribution build.
 1.1.26.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.1.26.1 21-Apr-2020  martin Sync with HEAD
 1.1.4.2 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.1.4.1 10-Aug-2014  tls file ctzdi2.S was added on branch tls-maxphys on 2014-08-19 23:45:12 +0000
 1.2 03-Sep-2020  jakllsch Remove unused assembly source files
 1.1 10-Aug-2014  matt branches: 1.1.4; 1.1.26;
Preliminary files for AARCH64 (64-bit ARM) support.
Enough for a distribution build.
 1.1.26.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.1.26.1 21-Apr-2020  martin Sync with HEAD
 1.1.4.2 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.1.4.1 10-Aug-2014  tls file ffsdi2.S was added on branch tls-maxphys on 2014-08-19 23:45:12 +0000
 1.4 07-Feb-2024  msaitoh Remove ryo@'s mail addresses.
 1.3 23-Jul-2023  skrll port-arm/57388: Minor bug fix in bcopy.S

Use correct register to check alignment of destination in backwards copy.

Patch from Antoni Pokusinski. Thanks.
 1.2 11-Apr-2020  ryo branches: 1.2.8;
Fixed to not use the "br" instruction. Branch Target Identification (BTI) doesn't like "br".

requested by maxv@
 1.1 04-Feb-2018  skrll branches: 1.1.4;

Working / new versions from Ryo Shimizu
 1.1.4.3 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.1.4.2 21-Apr-2020  martin Sync with HEAD
 1.1.4.1 13-Apr-2020  martin Mostly merge changes from HEAD upto 20200411
 1.2.8.1 20-Jul-2024  martin Pull up following revision(s) (requested by rin in ticket #744):

common/lib/libc/arch/aarch64/string/bcopy.S: revision 1.3

port-arm/57388: Minor bug fix in bcopy.S

Use correct register to check alignment of destination in backwards copy.
Patch from Antoni Pokusinski. Thanks.
 1.3 09-Jul-2018  ryo avoid reading from out of range that may cause access fault.
 1.2 04-Feb-2018  skrll branches: 1.2.2; 1.2.4;

Working / new versions from Ryo Shimizu
 1.1 10-Aug-2014  matt branches: 1.1.4;
Preliminary files for AARCH64 (64-bit ARM) support.
Enough for a distribution build.
 1.1.4.2 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.1.4.1 10-Aug-2014  tls file memcmp.S was added on branch tls-maxphys on 2014-08-19 23:45:12 +0000
 1.2.4.3 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.2.4.2 21-Apr-2020  martin Sync with HEAD
 1.2.4.1 10-Jun-2019  christos Sync with HEAD
 1.2.2.1 28-Jul-2018  pgoyette Sync with HEAD
 1.2 04-Feb-2018  skrll branches: 1.2.4;

Working / new versions from Ryo Shimizu
 1.1 10-Aug-2014  matt branches: 1.1.4;
Preliminary files for AARCH64 (64-bit ARM) support.
Enough for a distribution build.
 1.1.4.2 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.1.4.1 10-Aug-2014  tls file memcpy.S was added on branch tls-maxphys on 2014-08-19 23:45:12 +0000
 1.2.4.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.2.4.1 21-Apr-2020  martin Sync with HEAD
 1.1 04-Feb-2018  skrll branches: 1.1.4;

Working / new versions from Ryo Shimizu
 1.1.4.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.1.4.1 21-Apr-2020  martin Sync with HEAD
 1.3 11-Apr-2020  ryo Fixed to not use the "br" instruction. Branch Target Identification (BTI) doesn't like "br".

requested by maxv@
 1.2 29-Aug-2017  ryo branches: 1.2.4;
* aarch64/memset.S didn't work! fixed some bugs.
* maximum size of DCZID_EL0:BS (2048) supported.
 1.1 10-Aug-2014  matt branches: 1.1.4;
Preliminary files for AARCH64 (64-bit ARM) support.
Enough for a distribution build.
 1.1.4.2 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.1.4.1 10-Aug-2014  tls file memset.S was added on branch tls-maxphys on 2014-08-19 23:45:12 +0000
 1.2.4.3 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.2.4.2 21-Apr-2020  martin Sync with HEAD
 1.2.4.1 13-Apr-2020  martin Mostly merge changes from HEAD upto 20200411
 1.1 10-Aug-2014  matt branches: 1.1.4; 1.1.26;
Preliminary files for AARCH64 (64-bit ARM) support.
Enough for a distribution build.
 1.1.26.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.1.26.1 21-Apr-2020  martin Sync with HEAD
 1.1.4.2 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.1.4.1 10-Aug-2014  tls file strcat.S was added on branch tls-maxphys on 2014-08-19 23:45:12 +0000
 1.5 09-Sep-2020  jakllsch Re-do previous aarch64eb strlen fix more simply and correctly.
 1.4 05-Sep-2020  jakllsch Fix a broken corner case of strlen()/strnlen() on aarch64eb

Previously a string such as "\x1\x1\x1\x1\x1\x1\x1" would count as
0 instead of 7 on BE.
 1.3 01-Aug-2018  ryo strnlen(s, (size_t)-1) returned -1. it must return the length of s.
 1.2 22-Aug-2017  ryo branches: 1.2.2; 1.2.4;
aarch64/strlen.S didn't work. fixed some bugs.
 1.1 10-Aug-2014  matt branches: 1.1.4;
Preliminary files for AARCH64 (64-bit ARM) support.
Enough for a distribution build.
 1.1.4.2 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.1.4.1 10-Aug-2014  tls file strlen.S was added on branch tls-maxphys on 2014-08-19 23:45:12 +0000
 1.2.4.3 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.2.4.2 21-Apr-2020  martin Sync with HEAD
 1.2.4.1 10-Jun-2019  christos Sync with HEAD
 1.2.2.1 06-Sep-2018  pgoyette Sync with HEAD

Resolve a couple of conflicts (result of the uimin/uimax changes)
 1.1 10-Aug-2014  matt branches: 1.1.4; 1.1.26;
Preliminary files for AARCH64 (64-bit ARM) support.
Enough for a distribution build.
 1.1.26.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.1.26.1 21-Apr-2020  martin Sync with HEAD
 1.1.4.2 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.1.4.1 10-Aug-2014  tls file strnlen.S was added on branch tls-maxphys on 2014-08-19 23:45:12 +0000
 1.5 04-Jan-2009  pooka branches: 1.5.50;
allow inclusion of atomic ops in librump
 1.4 29-Sep-2008  ad Allow atomic ops to be built as part of libpthread.
 1.3 11-Feb-2008  ad Only build atomic ops for libkern/libc.
 1.2 28-Nov-2007  ad branches: 1.2.4;
Pull in alpha atomic ops from the thorpej-atomic branch.
 1.1 18-Apr-2007  thorpej branches: 1.1.2;
file Makefile.inc was initially added on branch thorpej-atomic.
 1.1.2.1 18-Apr-2007  thorpej Atomic op implementation for Alpha.
 1.2.4.3 23-Mar-2008  matt sync with HEAD
 1.2.4.2 09-Jan-2008  matt sync with HEAD
 1.2.4.1 28-Nov-2007  matt file Makefile.inc was added on branch matt-armv6 on 2008-01-09 01:20:47 +0000
 1.5.50.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.5.50.1 21-Apr-2020  martin Sync with HEAD
 1.5 06-Apr-2022  riastradh Nix trailing whitespace in files of membars, atomics, and lock stubs.

Will be touching many of these files soon for functional changes.

No functional change intended.
 1.4 25-May-2008  chs branches: 1.4.62;
enable profiling of assembly functions.
 1.3 28-Apr-2008  martin branches: 1.3.2;
Remove clause 3 and 4 from TNF licenses
 1.2 28-Nov-2007  ad branches: 1.2.4; 1.2.8;
Pull in alpha atomic ops from the thorpej-atomic branch.
 1.1 18-Apr-2007  thorpej branches: 1.1.2;
file atomic_add.S was initially added on branch thorpej-atomic.
 1.1.2.2 22-Apr-2007  thorpej Make sure namespace-cleansed aliases are avaialble for all atomic ops.
 1.1.2.1 18-Apr-2007  thorpej Atomic op implementation for Alpha.
 1.2.8.2 04-Jun-2008  yamt sync with head
 1.2.8.1 18-May-2008  yamt sync with head.
 1.2.4.2 09-Jan-2008  matt sync with HEAD
 1.2.4.1 28-Nov-2007  matt file atomic_add.S was added on branch matt-armv6 on 2008-01-09 01:20:47 +0000
 1.3.2.1 23-Jun-2008  wrstuden Sync w/ -current. 34 merge conflicts to follow.
 1.4.62.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.4.62.1 21-Apr-2020  martin Sync with HEAD
 1.5 06-Apr-2022  riastradh Nix trailing whitespace in files of membars, atomics, and lock stubs.

Will be touching many of these files soon for functional changes.

No functional change intended.
 1.4 25-May-2008  chs branches: 1.4.62;
enable profiling of assembly functions.
 1.3 28-Apr-2008  martin branches: 1.3.2;
Remove clause 3 and 4 from TNF licenses
 1.2 28-Nov-2007  ad branches: 1.2.4; 1.2.8;
Pull in alpha atomic ops from the thorpej-atomic branch.
 1.1 18-Apr-2007  thorpej branches: 1.1.2;
file atomic_and.S was initially added on branch thorpej-atomic.
 1.1.2.2 22-Apr-2007  thorpej Make sure namespace-cleansed aliases are avaialble for all atomic ops.
 1.1.2.1 18-Apr-2007  thorpej Atomic op implementation for Alpha.
 1.2.8.2 04-Jun-2008  yamt sync with head
 1.2.8.1 18-May-2008  yamt sync with head.
 1.2.4.2 09-Jan-2008  matt sync with HEAD
 1.2.4.1 28-Nov-2007  matt file atomic_and.S was added on branch matt-armv6 on 2008-01-09 01:20:48 +0000
 1.3.2.1 23-Jun-2008  wrstuden Sync w/ -current. 34 merge conflicts to follow.
 1.4.62.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.4.62.1 21-Apr-2020  martin Sync with HEAD
 1.6 06-Apr-2022  riastradh Nix trailing whitespace in files of membars, atomics, and lock stubs.

Will be touching many of these files soon for functional changes.

No functional change intended.
 1.5 25-May-2008  chs branches: 1.5.62;
enable profiling of assembly functions.
 1.4 28-Apr-2008  martin branches: 1.4.2;
Remove clause 3 and 4 from TNF licenses
 1.3 10-Feb-2008  ad branches: 1.3.4;
Add atomic_cas_foo_ni().
 1.2 28-Nov-2007  ad branches: 1.2.4;
Pull in alpha atomic ops from the thorpej-atomic branch.
 1.1 18-Apr-2007  thorpej branches: 1.1.2;
file atomic_cas.S was initially added on branch thorpej-atomic.
 1.1.2.2 22-Apr-2007  thorpej Make sure namespace-cleansed aliases are avaialble for all atomic ops.
 1.1.2.1 18-Apr-2007  thorpej Atomic op implementation for Alpha.
 1.2.4.3 23-Mar-2008  matt sync with HEAD
 1.2.4.2 09-Jan-2008  matt sync with HEAD
 1.2.4.1 28-Nov-2007  matt file atomic_cas.S was added on branch matt-armv6 on 2008-01-09 01:20:48 +0000
 1.3.4.2 04-Jun-2008  yamt sync with head
 1.3.4.1 18-May-2008  yamt sync with head.
 1.4.2.1 23-Jun-2008  wrstuden Sync w/ -current. 34 merge conflicts to follow.
 1.5.62.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.5.62.1 21-Apr-2020  martin Sync with HEAD
 1.5 06-Apr-2022  riastradh Nix trailing whitespace in files of membars, atomics, and lock stubs.

Will be touching many of these files soon for functional changes.

No functional change intended.
 1.4 25-May-2008  chs branches: 1.4.62;
enable profiling of assembly functions.
 1.3 28-Apr-2008  martin branches: 1.3.2;
Remove clause 3 and 4 from TNF licenses
 1.2 28-Nov-2007  ad branches: 1.2.4; 1.2.8;
Pull in alpha atomic ops from the thorpej-atomic branch.
 1.1 18-Apr-2007  thorpej branches: 1.1.2;
file atomic_dec.S was initially added on branch thorpej-atomic.
 1.1.2.2 22-Apr-2007  thorpej Make sure namespace-cleansed aliases are avaialble for all atomic ops.
 1.1.2.1 18-Apr-2007  thorpej Atomic op implementation for Alpha.
 1.2.8.2 04-Jun-2008  yamt sync with head
 1.2.8.1 18-May-2008  yamt sync with head.
 1.2.4.2 09-Jan-2008  matt sync with HEAD
 1.2.4.1 28-Nov-2007  matt file atomic_dec.S was added on branch matt-armv6 on 2008-01-09 01:20:48 +0000
 1.3.2.1 23-Jun-2008  wrstuden Sync w/ -current. 34 merge conflicts to follow.
 1.4.62.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.4.62.1 21-Apr-2020  martin Sync with HEAD
 1.5 06-Apr-2022  riastradh Nix trailing whitespace in files of membars, atomics, and lock stubs.

Will be touching many of these files soon for functional changes.

No functional change intended.
 1.4 25-May-2008  chs branches: 1.4.62;
enable profiling of assembly functions.
 1.3 28-Apr-2008  martin branches: 1.3.2;
Remove clause 3 and 4 from TNF licenses
 1.2 28-Nov-2007  ad branches: 1.2.4; 1.2.8;
Pull in alpha atomic ops from the thorpej-atomic branch.
 1.1 18-Apr-2007  thorpej branches: 1.1.2;
file atomic_inc.S was initially added on branch thorpej-atomic.
 1.1.2.2 22-Apr-2007  thorpej Make sure namespace-cleansed aliases are avaialble for all atomic ops.
 1.1.2.1 18-Apr-2007  thorpej Atomic op implementation for Alpha.
 1.2.8.2 04-Jun-2008  yamt sync with head
 1.2.8.1 18-May-2008  yamt sync with head.
 1.2.4.2 09-Jan-2008  matt sync with HEAD
 1.2.4.1 28-Nov-2007  matt file atomic_inc.S was added on branch matt-armv6 on 2008-01-09 01:20:49 +0000
 1.3.2.1 23-Jun-2008  wrstuden Sync w/ -current. 34 merge conflicts to follow.
 1.4.62.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.4.62.1 21-Apr-2020  martin Sync with HEAD
 1.4 06-Apr-2022  riastradh Nix trailing whitespace in files of membars, atomics, and lock stubs.

Will be touching many of these files soon for functional changes.

No functional change intended.
 1.3 28-Apr-2008  martin branches: 1.3.64;
Remove clause 3 and 4 from TNF licenses
 1.2 28-Nov-2007  ad branches: 1.2.4; 1.2.8;
Pull in alpha atomic ops from the thorpej-atomic branch.
 1.1 18-Apr-2007  thorpej branches: 1.1.2;
file atomic_op_asm.h was initially added on branch thorpej-atomic.
 1.1.2.1 18-Apr-2007  thorpej Atomic op implementation for Alpha.
 1.2.8.1 18-May-2008  yamt sync with head.
 1.2.4.2 09-Jan-2008  matt sync with HEAD
 1.2.4.1 28-Nov-2007  matt file atomic_op_asm.h was added on branch matt-armv6 on 2008-01-09 01:20:49 +0000
 1.3.64.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.3.64.1 21-Apr-2020  martin Sync with HEAD
 1.5 06-Apr-2022  riastradh Nix trailing whitespace in files of membars, atomics, and lock stubs.

Will be touching many of these files soon for functional changes.

No functional change intended.
 1.4 25-May-2008  chs branches: 1.4.62;
enable profiling of assembly functions.
 1.3 28-Apr-2008  martin branches: 1.3.2;
Remove clause 3 and 4 from TNF licenses
 1.2 28-Nov-2007  ad branches: 1.2.4; 1.2.8;
Pull in alpha atomic ops from the thorpej-atomic branch.
 1.1 18-Apr-2007  thorpej branches: 1.1.2;
file atomic_or.S was initially added on branch thorpej-atomic.
 1.1.2.2 22-Apr-2007  thorpej Make sure namespace-cleansed aliases are avaialble for all atomic ops.
 1.1.2.1 18-Apr-2007  thorpej Atomic op implementation for Alpha.
 1.2.8.2 04-Jun-2008  yamt sync with head
 1.2.8.1 18-May-2008  yamt sync with head.
 1.2.4.2 09-Jan-2008  matt sync with HEAD
 1.2.4.1 28-Nov-2007  matt file atomic_or.S was added on branch matt-armv6 on 2008-01-09 01:20:50 +0000
 1.3.2.1 23-Jun-2008  wrstuden Sync w/ -current. 34 merge conflicts to follow.
 1.4.62.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.4.62.1 21-Apr-2020  martin Sync with HEAD
 1.5 06-Apr-2022  riastradh Nix trailing whitespace in files of membars, atomics, and lock stubs.

Will be touching many of these files soon for functional changes.

No functional change intended.
 1.4 25-May-2008  chs branches: 1.4.62;
enable profiling of assembly functions.
 1.3 28-Apr-2008  martin branches: 1.3.2;
Remove clause 3 and 4 from TNF licenses
 1.2 28-Nov-2007  ad branches: 1.2.4; 1.2.8;
Pull in alpha atomic ops from the thorpej-atomic branch.
 1.1 18-Apr-2007  thorpej branches: 1.1.2;
file atomic_swap.S was initially added on branch thorpej-atomic.
 1.1.2.2 22-Apr-2007  thorpej Make sure namespace-cleansed aliases are avaialble for all atomic ops.
 1.1.2.1 18-Apr-2007  thorpej Atomic op implementation for Alpha.
 1.2.8.2 04-Jun-2008  yamt sync with head
 1.2.8.1 18-May-2008  yamt sync with head.
 1.2.4.2 09-Jan-2008  matt sync with HEAD
 1.2.4.1 28-Nov-2007  matt file atomic_swap.S was added on branch matt-armv6 on 2008-01-09 01:20:50 +0000
 1.3.2.1 23-Jun-2008  wrstuden Sync w/ -current. 34 merge conflicts to follow.
 1.4.62.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.4.62.1 21-Apr-2020  martin Sync with HEAD
 1.9 09-Apr-2022  riastradh Introduce membar_acquire/release. Deprecate membar_enter/exit.

The names membar_enter/exit were unclear, and the documentation of
membar_enter has disagreed with the implementations on sparc,
powerpc, and even x86(!) for the entire time it has been in NetBSD.

The terms `acquire' and `release' are ubiquitous in the literature
today, and have been adopted in the C and C++ standards to mean
load-before-load/store and load/store-before-store, respectively,
which are exactly the orderings required by acquiring and releasing a
mutex, as well as other useful applications like decrementing a
reference count and then freeing the underlying object if it went to
zero.

Originally I proposed changing one word in the documentation for
membar_enter to make it load-before-load/store instead of
store-before-load/store, i.e., to make it an acquire barrier. I
proposed this on the grounds that

(a) all implementations guarantee load-before-load/store,
(b) some implementations fail to guarantee store-before-load/store,
and
(c) all uses in-tree assume load-before-load/store.

I verified parts (a) and (b) (except, for (a), powerpc didn't even
guarantee load-before-load/store -- isync isn't necessarily enough;
need lwsync in general -- but it _almost_ did, and it certainly didn't
guarantee store-before-load/store).

Part (c) might not be correct, however: under the mistaken assumption
that atomic-r/m/w then membar-w/rw is equivalent to atomic-r/m/w then
membar-r/rw, I only audited the cases of membar_enter that _aren't_
immediately after an atomic-r/m/w. All of those cases assume
load-before-load/store. But my assumption was wrong -- there are
cases of atomic-r/m/w then membar-w/rw that would be broken by
changing to atomic-r/m/w then membar-r/rw:

https://mail-index.netbsd.org/tech-kern/2022/03/29/msg028044.html

Furthermore, the name membar_enter has been adopted in other places
like OpenBSD where it actually does follow the documentation and
guarantee store-before-load/store, even if that order is not useful.
So the name membar_enter currently lives in a bad place where it
means either of two things -- r/rw or w/rw.

With this change, we deprecate membar_enter/exit, introduce
membar_acquire/release as better names for the useful pair (r/rw and
rw/w), and make sure the implementation of membar_enter guarantees
both what was documented _and_ what was implemented, making it an
alias for membar_sync.

While here, rework all of the membar_* definitions and aliases. The
new logic follows a rule to make it easier to audit:

membar_X is defined as an alias for membar_Y iff membar_X is
guaranteed by membar_Y.

The `no stronger than' relation is (the transitive closure of):

- membar_consumer (r/r) is guaranteed by membar_acquire (r/rw)
- membar_producer (w/w) is guaranteed by membar_release (rw/w)
- membar_acquire (r/rw) is guaranteed by membar_sync (rw/rw)
- membar_release (rw/w) is guaranteed by membar_sync (rw/rw)

And, for the deprecated membars:

- membar_enter (whether r/rw, w/rw, or rw/rw) is guaranteed by
membar_sync (rw/rw)
- membar_exit (rw/w) is guaranteed by membar_release (rw/w)

(membar_exit is identical to membar_release, but the name is
deprecated.)

Finally, while here, annotate some of the instructions with their
semantics. For powerpc, leave an essay with citations on the
unfortunate but -- as far as I can tell -- necessary decision to use
lwsync, not isync, for membar_acquire and membar_consumer.

Also add membar(3) and atomic(3) man page links.
 1.8 06-Apr-2022  riastradh Nix trailing whitespace in files of membars, atomics, and lock stubs.

Will be touching many of these files soon for functional changes.

No functional change intended.
 1.7 08-Jan-2015  riastradh branches: 1.7.16;
Introduce membar_datadep_consumer.

Discussed briefly on tech-kern without objection:

https://mail-index.netbsd.org/tech-kern/2014/11/20/msg018054.html
https://mail-index.netbsd.org/tech-kern/2015/01/07/msg018326.html
 1.6 25-May-2008  chs branches: 1.6.40;
enable profiling of assembly functions.
 1.5 28-Apr-2008  martin branches: 1.5.2;
Remove clause 3 and 4 from TNF licenses
 1.4 12-Feb-2008  ad branches: 1.4.4;
Always enable memory barriers in userspace, otherwise libpthread won't
work correctly on MP systems.
 1.3 12-Feb-2008  matt Fix pthread problem. (export membar* to userland)
 1.2 28-Nov-2007  ad branches: 1.2.4;
Pull in alpha atomic ops from the thorpej-atomic branch.
 1.1 18-Apr-2007  thorpej branches: 1.1.2;
file membar_ops.S was initially added on branch thorpej-atomic.
 1.1.2.2 22-Apr-2007  thorpej Make sure namespace-cleansed aliases are avaialble for all atomic ops.
 1.1.2.1 18-Apr-2007  thorpej Atomic op implementation for Alpha.
 1.2.4.3 23-Mar-2008  matt sync with HEAD
 1.2.4.2 09-Jan-2008  matt sync with HEAD
 1.2.4.1 28-Nov-2007  matt file membar_ops.S was added on branch matt-armv6 on 2008-01-09 01:20:51 +0000
 1.4.4.2 04-Jun-2008  yamt sync with head
 1.4.4.1 18-May-2008  yamt sync with head.
 1.5.2.1 23-Jun-2008  wrstuden Sync w/ -current. 34 merge conflicts to follow.
 1.6.40.1 18-Mar-2015  snj Pull up following revision(s) (requested by riastradh in ticket #600):
common/lib/libc/arch/alpha/atomic/membar_ops.S: revision 1.7
lib/libc/atomic/membar_ops.3: revision 1.4
sys/arch/alpha/include/types.h: revision 1.50
sys/sys/atomic.h: revision 1.13
Introduce membar_datadep_consumer.
Discussed briefly on tech-kern without objection:
https://mail-index.netbsd.org/tech-kern/2014/11/20/msg018054.html
https://mail-index.netbsd.org/tech-kern/2015/01/07/msg018326.html
 1.7.16.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.7.16.1 21-Apr-2020  martin Sync with HEAD
 1.3 16-Feb-2008  apb branches: 1.3.70;
For all files under src/common, as well as related man pages under
src/lib/libc, or related headers under src/sys/sys: change u_intNN_t to
uintNN_t.
 1.2 27-Dec-2005  yamt branches: 1.2.6; 1.2.18;
make alpha kernel buildable again.
don't add __ prefix to bswap{16,32}
if defined(_KERNEL) || defined(_STANDALONE).
 1.1 20-Dec-2005  christos Merge libkern + libc common files. As requested by core.
 1.2.18.1 23-Mar-2008  matt sync with HEAD
 1.2.6.1 18-Nov-2008  snj Pull up following revision(s) (requested by tsutsui in ticket #1224):
common/lib/libc/gen/bswap64.c: revision 1.2
common/lib/libprop/prop_data.c: revision 1.10
common/lib/libc/inet/inet_addr.c: revision 1.2
common/lib/libc/hash/rmd160/rmd160.c: revision 1.4
common/lib/libc/gen/bswap32.c: revision 1.2
common/lib/libc/gen/bswap16.c: revision 1.2
common/lib/libc/arch/alpha/gen/byte_swap_4.S: revision 1.3
common/lib/libc/arch/sh3/string/memset.S: revision 1.2
common/lib/libc/arch/alpha/gen/byte_swap_2.S: revision 1.3
common/lib/libc/md/md5c.c: revision 1.3
common/lib/libc/hash/sha1/sha1.c: revision 1.3
common/lib/libc/md/md4c.c: revision 1.3
sys/sys/rmd160.h: revision 1.2
sys/sys/sha2.h: revision 1.2
For all files under src/common, as well as related man pages under
src/lib/libc, or related headers under src/sys/sys: change u_intNN_t to
uintNN_t.
 1.3.70.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.3.70.1 21-Apr-2020  martin Sync with HEAD
 1.3 16-Feb-2008  apb branches: 1.3.70;
For all files under src/common, as well as related man pages under
src/lib/libc, or related headers under src/sys/sys: change u_intNN_t to
uintNN_t.
 1.2 27-Dec-2005  yamt branches: 1.2.6; 1.2.18;
make alpha kernel buildable again.
don't add __ prefix to bswap{16,32}
if defined(_KERNEL) || defined(_STANDALONE).
 1.1 20-Dec-2005  christos Merge libkern + libc common files. As requested by core.
 1.2.18.1 23-Mar-2008  matt sync with HEAD
 1.2.6.1 18-Nov-2008  snj Pull up following revision(s) (requested by tsutsui in ticket #1224):
common/lib/libc/gen/bswap64.c: revision 1.2
common/lib/libprop/prop_data.c: revision 1.10
common/lib/libc/inet/inet_addr.c: revision 1.2
common/lib/libc/hash/rmd160/rmd160.c: revision 1.4
common/lib/libc/gen/bswap32.c: revision 1.2
common/lib/libc/gen/bswap16.c: revision 1.2
common/lib/libc/arch/alpha/gen/byte_swap_4.S: revision 1.3
common/lib/libc/arch/sh3/string/memset.S: revision 1.2
common/lib/libc/arch/alpha/gen/byte_swap_2.S: revision 1.3
common/lib/libc/md/md5c.c: revision 1.3
common/lib/libc/hash/sha1/sha1.c: revision 1.3
common/lib/libc/md/md4c.c: revision 1.3
sys/sys/rmd160.h: revision 1.2
sys/sys/sha2.h: revision 1.2
For all files under src/common, as well as related man pages under
src/lib/libc, or related headers under src/sys/sys: change u_intNN_t to
uintNN_t.
 1.3.70.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.3.70.1 21-Apr-2020  martin Sync with HEAD
 1.2 21-Dec-2005  christos branches: 1.2.94;
Only build this if GPROF && !_STANDALONE
 1.1 20-Dec-2005  christos Merge libkern + libc common files. As requested by core.
 1.2.94.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.2.94.1 21-Apr-2020  martin Sync with HEAD
 1.1 20-Dec-2005  christos branches: 1.1.94;
Merge libkern + libc common files. As requested by core.
 1.1.94.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.1.94.1 21-Apr-2020  martin Sync with HEAD
 1.1 20-Dec-2005  christos branches: 1.1.94;
Merge libkern + libc common files. As requested by core.
 1.1.94.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.1.94.1 21-Apr-2020  martin Sync with HEAD
 1.1 20-Dec-2005  christos branches: 1.1.94;
Merge libkern + libc common files. As requested by core.
 1.1.94.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.1.94.1 21-Apr-2020  martin Sync with HEAD
 1.1 20-Dec-2005  christos branches: 1.1.94;
Merge libkern + libc common files. As requested by core.
 1.1.94.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.1.94.1 21-Apr-2020  martin Sync with HEAD
 1.1 20-Dec-2005  christos branches: 1.1.94;
Merge libkern + libc common files. As requested by core.
 1.1.94.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.1.94.1 21-Apr-2020  martin Sync with HEAD
 1.3 11-Oct-2020  skrll Define _ARM_ARCH_8 when __ARM_ARCH_8A (no trailing double underscore) as
it is defined by gcc.

__ARM_ARCH_8A__ (with trailing double underscore) seems to be a typo (or
maybe historical)
 1.2 02-Aug-2019  joerg ARMv6KZ has been misspelled by GCC since forever, but clang only
provides the correct name. Support both.
 1.1 27-Feb-2014  matt branches: 1.1.4; 1.1.8; 1.1.30;
Add a method to test the compiler for things like LDREX availability,
LDRD availability, THUMB2, EABI.
 1.1.30.3 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.1.30.2 21-Apr-2020  martin Sync with HEAD
 1.1.30.1 13-Apr-2020  martin Mostly merge changes from HEAD upto 20200411
 1.1.8.2 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.1.8.1 27-Feb-2014  tls file features.c was added on branch tls-maxphys on 2014-08-19 23:45:12 +0000
 1.1.4.2 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.1.4.1 27-Feb-2014  yamt file features.c was added on branch yamt-pagecache on 2014-05-22 11:26:28 +0000
 1.2 07-Jun-2015  matt branches: 1.2.16;
Use ${COPT} so it properly emits the right features doing MKCOMPAT
 1.1 27-Feb-2014  matt branches: 1.1.4; 1.1.6; 1.1.8;
Add a method to test the compiler for things like LDREX availability,
LDRD availability, THUMB2, EABI.
 1.1.8.2 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.1.8.1 27-Feb-2014  tls file features.mk was added on branch tls-maxphys on 2014-08-19 23:45:12 +0000
 1.1.6.1 10-Jun-2015  snj Pull up following revision(s) (requested by martin in ticket #837):
common/lib/libc/arch/arm/features.mk: revision 1.2
lib/libarch/Makefile: revision 1.29
lib/libarch/i386/Makefile.inc: revision 1.15
lib/libarch/x86_64/Makefile.inc: revision 1.5
share/mk/bsd.gcc.mk: revision 1.11
share/mk/bsd.prog.mk: revision 1.292
Make LIBCRT* depend on the right files if MLIBDIR from MKCOMPAT is defined.
Use ${COPT} so it properly emits the right features doing MKCOMPAT
Avoid defining MLIBDIR. Use ${MLIBDIR:Unone} instead.
 1.1.4.2 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.1.4.1 27-Feb-2014  yamt file features.mk was added on branch yamt-pagecache on 2014-05-22 11:26:28 +0000
 1.2.16.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.2.16.1 21-Apr-2020  martin Sync with HEAD
 1.29 07-Jun-2015  matt branches: 1.29.16;
Back out last change; fixed in the correct place.
 1.28 07-Jun-2015  joerg Require ARMv5TE to assemble.
 1.27 17-May-2015  justin Move arm sync_* changes to Makefile.inc
 1.26 14-Oct-2014  martin Provide C++ 2011 <atomic> support functions for hppa and arm.
 1.25 13-Oct-2014  martin Move the and_{16,8}_nv sources into the right (libc only) block.
 1.24 13-Oct-2014  martin Provide __sync_and_and_fetch_2 and __sync_and_and_fetch_1 for pre-ARMv6,
they are needed for the C++ 2011 <atomic> stuff.
 1.23 05-Jul-2014  joerg branches: 1.23.2;
Provide a basic implementation of __atomic_load_* and __atomic_store_*,
used by GCC and LLVM as backing for C11/C++11 atomics, if the hardware
is not known to have corresponding features. Include it on ARM as LLVM
and libc++ hit it when compiled for ARMv4.
 1.22 04-Mar-2014  matt branches: 1.22.2;
Don't export __sync* if _KERNEL || _STANDALONE are defined.
(except if _RUMPKERNEL is defined)
 1.21 04-Mar-2014  matt fix typo.
 1.20 04-Mar-2014  matt Add atomic_sub_64.S
 1.19 27-Feb-2014  matt Add atomic_cas_64 support for ARM EABI on V5TE and V5TEJ cpus.
(strd is atomic).
 1.18 22-Feb-2014  martin Missed one __sync_* op (or gcc4.8 does inline it, while 4.5 does not?)
 1.17 22-Feb-2014  martin Move the __sync_* ops added in the previous change to a libc-only section
 1.16 21-Feb-2014  martin Provide the missing __sync_* ops for earlier arm versions
 1.15 27-Jan-2014  matt Add _atomic_cas_16_up and _atomic_cas_8_up
 1.14 08-Nov-2013  matt Add support for the gcc __sync builtins.
Note that these need earmv6 or later to get the ldrex/strex instructions
 1.13 19-Aug-2013  matt Rework to allow thumb armv7 compilation.
Add atomic_simplelock.c for thumb
 1.12 06-Aug-2013  matt Only use MACHINE_ARCH if !_STANDALONE and CPPFLAGS/CFLAGS/CPUFLAGS don't
contain -mcpu or -march
 1.11 06-Aug-2013  matt Select ldrex/strex if ${MACHINE_ARCH:Mearmv[67]*}
 1.10 11-Sep-2012  matt branches: 1.10.2;
Add 64bit atomic ops for ARMv6+ (using ldrexd/strexd).
 1.9 16-Aug-2012  matt Actually use the assembly version of the atomic function if compiling
for armv6 or armv7 cpus. Use atomic_cas_ptr instead of _lock_cas so
we pick up the assembly version when it's used.
 1.8 04-Jan-2009  pooka branches: 1.8.8; 1.8.12;
allow inclusion of atomic ops in librump
 1.7 29-Sep-2008  ad branches: 1.7.8;
Allow atomic ops to be built as part of libpthread.
 1.6 29-Apr-2008  scw Implement _atomic_cas_up() in assembly code as the compiler cannot be
trusted to generate fully restartable code sequences.

Addresses lib/38482 for ARM and m68000.
 1.5 11-Feb-2008  ad branches: 1.5.4;
Only build atomic ops for libkern/libc.
 1.4 10-Feb-2008  ad Enable the atomic ops in userspace.
 1.3 29-Nov-2007  ad branches: 1.3.4;
Use the CAS-based inc/dec variants, since these CPUs don't have atomic
add in hardware (does arm?).
 1.2 29-Nov-2007  ad Atomic ops for arm.
 1.1 18-Apr-2007  thorpej branches: 1.1.2;
file Makefile.inc was initially added on branch thorpej-atomic.
 1.1.2.2 26-Aug-2007  matt Add armv6 variants of atomic ops which use ldrex/strex.
Add atomic_swap (all arm architectures) which uses swp and swpb.
Add membar_ops which uses armv6 data {sync, memory} barrier.
XXX let thorpej figure out the Makefile magic.
 1.1.2.1 18-Apr-2007  thorpej Atomic ops implementation for ARM. Everything is built on top of
atomic_cas_32(), which is itself a restartable atomic sequence.
 1.3.4.3 23-Mar-2008  matt sync with HEAD
 1.3.4.2 09-Jan-2008  matt sync with HEAD
 1.3.4.1 29-Nov-2007  matt file Makefile.inc was added on branch matt-armv6 on 2008-01-09 01:20:51 +0000
 1.5.4.1 18-May-2008  yamt sync with head.
 1.7.8.2 15-Feb-2014  matt Add 64-bit atomic ops if ARMv6+
 1.7.8.1 19-Dec-2013  matt Enable ldrex/strex based atomic ops on armv6/armv7
 1.8.12.1 27-Nov-2012  matt Pull atomic ops from HEAD.
 1.8.8.2 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.8.8.1 30-Oct-2012  yamt sync with head
 1.10.2.1 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.22.2.1 10-Aug-2014  tls Rebase.
 1.23.2.1 12-Nov-2014  snj Pull up following revision(s) (requested by martin in ticket #218):
common/lib/libc/arch/arm/atomic/Makefile.inc: revision 1.24-1.26
common/lib/libc/arch/hppa/atomic/Makefile.inc: revision 1.13
common/lib/libc/arch/mips/atomic/Makefile.inc: revision 1.13
common/lib/libc/arch/sh3/atomic/Makefile.inc: revision 1.7
common/lib/libc/arch/sparc/atomic/Makefile.inc: revision 1.18
common/lib/libc/arch/vax/atomic/Makefile.inc: revision 1.7
common/lib/libc/atomic/atomic_and_16_nv_cas.c: revision 1.2
common/lib/libc/atomic/atomic_and_8_nv_cas.c: revision 1.2
common/lib/libc/atomic/atomic_c11_compare_exchange_cas_16.c: revision 1.1-1.2
common/lib/libc/atomic/atomic_c11_compare_exchange_cas_32.c: revision 1.1-1.2
common/lib/libc/atomic/atomic_c11_compare_exchange_cas_8.c: revision 1.1-1.2
common/lib/libc/atomic/atomic_cas_by_cas32.c: revision 1.4
common/lib/libc/atomic/atomic_op_namespace.h: revision 1.7
Add __sync_val_compare_and_swap_{1,2} aliases for _atomic_cas_{8,16}
--
Provide __atomic_compare_exchange_N (as needed for the C11 2011 <atomic>
ops) via the corresponding CAS.
--
Hook __atomic_compare_exchange_N into vax libc.
--
Provide __sync_and_and_fetch_2 and __sync_and_and_fetch_1 for pre-ARMv6,
they are needed for the C++ 2011 <atomic> stuff.
--
Add C++ 2011 <atomic> support functions.
--
Move the and_{16,8}_nv sources into the right (libc only) block.
--
Provide <atomic> C++ 2011 support functions for mips and sh3.
--
Provide C++ 2011 <atomic> support functions for hppa and arm.
--
Provide prototypes to fix build with clang.
 1.29.16.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.29.16.1 21-Apr-2020  martin Sync with HEAD
 1.5 28-Jul-2021  skrll Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)

This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)

This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.4 10-Jul-2021  skrll s/ifdef _ARM_ARCH_6/if defined(_ARM_ARCH_6)/ for consistency. NFCI.
 1.3 23-Jun-2014  joerg branches: 1.3.4; 1.3.26; 1.3.28;
Add aliases for the builtins used to implement C11/C++11 atomics.
 1.2 04-Mar-2014  matt branches: 1.2.2; 1.2.4;
Don't export __sync* if _KERNEL || _STANDALONE are defined.
(except if _RUMPKERNEL is defined)
 1.1 08-Nov-2013  matt Add support for the gcc __sync builtins.
Note that these need earmv6 or later to get the ldrex/strex instructions
 1.2.4.2 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.2.4.1 04-Mar-2014  yamt file atomic_add_16.S was added on branch yamt-pagecache on 2014-05-22 11:26:28 +0000
 1.2.2.1 10-Aug-2014  tls Rebase.
 1.3.28.1 11-Aug-2021  martin Pull up following revision(s) (requested by skrll in ticket #1330):

common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.18
common/lib/libc/arch/arm/atomic/atomic_and_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.19
common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_inc_32.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_and_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_add_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_inc_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_cas_up.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_add_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_op_asm.h: revision 1.9
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.8
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_op_asm.h: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_and_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_cas_16.S: revision 1.3
common/lib/libc/arch/arm/atomic/sync_fetch_and_and_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_32.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_xor_32.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_cas_32.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_xor_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_add_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/sync_fetch_and_add_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_dec_64.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_add_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/sync_fetch_and_sub_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_dec_64.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_add_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_xor_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_add_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.14
common/lib/libc/arch/arm/atomic/atomic_or_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_nand_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_or_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_cas_8.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_inc_64.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_inc_64.S: revision 1.11
common/lib/libc/arch/arm/atomic/atomic_cas_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_dec_32.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_or_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_dec_32.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_xor_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_4.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/atomic_or_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_2.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_swap_64.S: revision 1.14
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_1.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_and_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_and_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_and_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.14

Give the thumb atomic ops a chance of working

s/ifdef _ARM_ARCH_6/if defined(_ARM_ARCH_6)/ for consistency. NFCI.

Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.

That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)
This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)
This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.3.26.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.3.26.1 21-Apr-2020  martin Sync with HEAD
 1.3.4.2 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.3.4.1 23-Jun-2014  tls file atomic_add_16.S was added on branch tls-maxphys on 2014-08-19 23:45:12 +0000
 1.10 28-Jul-2021  skrll Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)

This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)

This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.9 10-Jul-2021  skrll s/ifdef _ARM_ARCH_6/if defined(_ARM_ARCH_6)/ for consistency. NFCI.
 1.8 23-Jun-2014  joerg branches: 1.8.24; 1.8.26;
Add aliases for the builtins used to implement C11/C++11 atomics.
 1.7 22-Feb-2014  martin branches: 1.7.2;
Try to hide the C runtime implementation specific __sync_* ops from librump,
to avoid duplicates.
 1.6 08-Nov-2013  matt Add support for the gcc __sync builtins.
Note that these need earmv6 or later to get the ldrex/strex instructions
 1.5 11-Aug-2013  matt Use foo{s} since it reduces the ifdefs for thumb
 1.4 10-Aug-2013  matt Make these under Thumb2
 1.3 31-Aug-2012  matt branches: 1.3.2;
Add dmb/dsb instructions as required by the armv7 arch man.
 1.2 16-Aug-2008  matt branches: 1.2.2; 1.2.4; 1.2.24;
Add assembly versions of atomic ops with ldrex/strex
 1.1 26-Aug-2007  matt branches: 1.1.2;
file atomic_add_32.S was initially added on branch thorpej-atomic.
 1.1.2.2 27-Aug-2007  matt Don't use r4 (since it's caller saved). Use ip (it's available since we
are leaf functions).
 1.1.2.1 26-Aug-2007  matt Add armv6 variants of atomic ops which use ldrex/strex.
Add atomic_swap (all arm architectures) which uses swp and swpb.
Add membar_ops which uses armv6 data {sync, memory} barrier.
XXX let thorpej figure out the Makefile magic.
 1.2.24.1 27-Nov-2012  matt Pull atomic ops from HEAD.
 1.2.4.2 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.2.4.1 30-Oct-2012  yamt sync with head
 1.2.2.2 18-Sep-2008  wrstuden Sync with wrstuden-revivesa-base-2.
 1.2.2.1 16-Aug-2008  wrstuden file atomic_add_32.S was added on branch wrstuden-revivesa on 2008-09-18 04:54:18 +0000
 1.3.2.1 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.7.2.1 10-Aug-2014  tls Rebase.
 1.8.26.1 11-Aug-2021  martin Pull up following revision(s) (requested by skrll in ticket #1330):

common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.18
common/lib/libc/arch/arm/atomic/atomic_and_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.19
common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_inc_32.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_and_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_add_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_inc_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_cas_up.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_add_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_op_asm.h: revision 1.9
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.8
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_op_asm.h: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_and_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_cas_16.S: revision 1.3
common/lib/libc/arch/arm/atomic/sync_fetch_and_and_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_32.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_xor_32.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_cas_32.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_xor_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_add_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/sync_fetch_and_add_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_dec_64.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_add_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/sync_fetch_and_sub_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_dec_64.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_add_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_xor_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_add_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.14
common/lib/libc/arch/arm/atomic/atomic_or_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_nand_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_or_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_cas_8.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_inc_64.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_inc_64.S: revision 1.11
common/lib/libc/arch/arm/atomic/atomic_cas_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_dec_32.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_or_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_dec_32.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_xor_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_4.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/atomic_or_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_2.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_swap_64.S: revision 1.14
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_1.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_and_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_and_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_and_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.14

Give the thumb atomic ops a chance of working

s/ifdef _ARM_ARCH_6/if defined(_ARM_ARCH_6)/ for consistency. NFCI.

Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.

That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)
This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)
This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.8.24.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.8.24.1 21-Apr-2020  martin Sync with HEAD
 1.14 28-Jul-2021  skrll Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)

This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)

This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.13 10-Jul-2021  skrll s/ifdef _ARM_ARCH_6/if defined(_ARM_ARCH_6)/ for consistency. NFCI.
 1.12 28-Jun-2021  skrll Whitespace
 1.11 04-Mar-2014  matt branches: 1.11.26; 1.11.28;
Don't export __sync* if _KERNEL || _STANDALONE are defined.
(except if _RUMPKERNEL is defined)
 1.10 30-Nov-2013  joerg branches: 1.10.2;
Explicitly name the register pairs.
 1.9 08-Nov-2013  matt Add support for the gcc __sync builtins.
Note that these need earmv6 or later to get the ldrex/strex instructions
 1.8 20-Aug-2013  matt Push two registers to keep stack aligned.
 1.7 11-Aug-2013  matt Use foo{s} since it reduces the ifdefs for thumb
 1.6 10-Aug-2013  matt Make these under Thumb2
 1.5 10-Aug-2013  matt Never modify NLO/NHI (r2,r3) only LO/HI (r0,r1) so subsequent loops will work.
 1.4 10-Aug-2013  matt use push/pop
 1.3 13-Sep-2012  matt branches: 1.3.2; 1.3.4;
Correct copyright/fix comments.
 1.2 12-Sep-2012  matt Fix bas code, use ldr
 1.1 11-Sep-2012  matt branches: 1.1.2;
Add 64bit atomic ops for ARMv6+ (using ldrexd/strexd).
 1.1.2.2 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.1.2.1 20-Nov-2012  tls Resync to 2012-11-19 00:00:00 UTC
 1.3.4.2 27-Nov-2012  matt Pull atomic ops from HEAD.
 1.3.4.1 13-Sep-2012  matt file atomic_add_64.S was added on branch matt-nb6-plus on 2012-11-27 23:42:35 +0000
 1.3.2.3 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.3.2.2 30-Oct-2012  yamt sync with head
 1.3.2.1 13-Sep-2012  yamt file atomic_add_64.S was added on branch yamt-pagecache on 2012-10-30 18:46:10 +0000
 1.10.2.2 15-Feb-2014  matt Add 64-bit atomic ops if ARMv6+
 1.10.2.1 30-Nov-2013  matt file atomic_add_64.S was added on branch matt-nb5-mips64 on 2014-02-15 10:27:44 +0000
 1.11.28.2 11-Aug-2021  martin Pull up following revision(s) (requested by skrll in ticket #1330):

common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.18
common/lib/libc/arch/arm/atomic/atomic_and_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.19
common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_inc_32.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_and_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_add_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_inc_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_cas_up.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_add_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_op_asm.h: revision 1.9
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.8
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_op_asm.h: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_and_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_cas_16.S: revision 1.3
common/lib/libc/arch/arm/atomic/sync_fetch_and_and_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_32.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_xor_32.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_cas_32.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_xor_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_add_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/sync_fetch_and_add_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_dec_64.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_add_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/sync_fetch_and_sub_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_dec_64.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_add_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_xor_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_add_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.14
common/lib/libc/arch/arm/atomic/atomic_or_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_nand_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_or_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_cas_8.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_inc_64.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_inc_64.S: revision 1.11
common/lib/libc/arch/arm/atomic/atomic_cas_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_dec_32.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_or_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_dec_32.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_xor_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_4.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/atomic_or_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_2.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_swap_64.S: revision 1.14
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_1.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_and_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_and_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_and_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.14

Give the thumb atomic ops a chance of working

s/ifdef _ARM_ARCH_6/if defined(_ARM_ARCH_6)/ for consistency. NFCI.

Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.

That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)
This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)
This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.11.28.1 06-Jul-2021  martin Pull up following revision(s) (requested by skrll in ticket #1313):

common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/atomic_cas_8.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.3
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.11
common/lib/libc/arch/arm/atomic/atomic_swap_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.12

Whitespace
 1.11.26.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.11.26.1 21-Apr-2020  martin Sync with HEAD
 1.5 28-Jul-2021  skrll Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)

This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)

This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.4 10-Jul-2021  skrll s/ifdef _ARM_ARCH_6/if defined(_ARM_ARCH_6)/ for consistency. NFCI.
 1.3 23-Jun-2014  joerg branches: 1.3.4; 1.3.26; 1.3.28;
Add aliases for the builtins used to implement C11/C++11 atomics.
 1.2 04-Mar-2014  matt branches: 1.2.2; 1.2.4;
Don't export __sync* if _KERNEL || _STANDALONE are defined.
(except if _RUMPKERNEL is defined)
 1.1 08-Nov-2013  matt Add support for the gcc __sync builtins.
Note that these need earmv6 or later to get the ldrex/strex instructions
 1.2.4.2 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.2.4.1 04-Mar-2014  yamt file atomic_add_8.S was added on branch yamt-pagecache on 2014-05-22 11:26:28 +0000
 1.2.2.1 10-Aug-2014  tls Rebase.
 1.3.28.1 11-Aug-2021  martin Pull up following revision(s) (requested by skrll in ticket #1330):

common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.18
common/lib/libc/arch/arm/atomic/atomic_and_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.19
common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_inc_32.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_and_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_add_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_inc_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_cas_up.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_add_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_op_asm.h: revision 1.9
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.8
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_op_asm.h: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_and_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_cas_16.S: revision 1.3
common/lib/libc/arch/arm/atomic/sync_fetch_and_and_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_32.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_xor_32.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_cas_32.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_xor_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_add_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/sync_fetch_and_add_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_dec_64.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_add_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/sync_fetch_and_sub_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_dec_64.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_add_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_xor_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_add_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.14
common/lib/libc/arch/arm/atomic/atomic_or_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_nand_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_or_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_cas_8.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_inc_64.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_inc_64.S: revision 1.11
common/lib/libc/arch/arm/atomic/atomic_cas_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_dec_32.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_or_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_dec_32.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_xor_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_4.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/atomic_or_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_2.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_swap_64.S: revision 1.14
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_1.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_and_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_and_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_and_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.14

Give the thumb atomic ops a chance of working

s/ifdef _ARM_ARCH_6/if defined(_ARM_ARCH_6)/ for consistency. NFCI.

Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.

That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)
This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)
This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.3.26.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.3.26.1 21-Apr-2020  martin Sync with HEAD
 1.3.4.2 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.3.4.1 23-Jun-2014  tls file atomic_add_8.S was added on branch tls-maxphys on 2014-08-19 23:45:12 +0000
 1.5 28-Jul-2021  skrll Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)

This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)

This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.4 10-Jul-2021  skrll s/ifdef _ARM_ARCH_6/if defined(_ARM_ARCH_6)/ for consistency. NFCI.
 1.3 23-Jun-2014  joerg branches: 1.3.4; 1.3.26; 1.3.28;
Add aliases for the builtins used to implement C11/C++11 atomics.
 1.2 04-Mar-2014  matt branches: 1.2.2; 1.2.4;
Don't export __sync* if _KERNEL || _STANDALONE are defined.
(except if _RUMPKERNEL is defined)
 1.1 08-Nov-2013  matt Add support for the gcc __sync builtins.
Note that these need earmv6 or later to get the ldrex/strex instructions
 1.2.4.2 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.2.4.1 04-Mar-2014  yamt file atomic_and_16.S was added on branch yamt-pagecache on 2014-05-22 11:26:28 +0000
 1.2.2.1 10-Aug-2014  tls Rebase.
 1.3.28.1 11-Aug-2021  martin Pull up following revision(s) (requested by skrll in ticket #1330):

common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.18
common/lib/libc/arch/arm/atomic/atomic_and_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.19
common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_inc_32.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_and_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_add_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_inc_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_cas_up.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_add_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_op_asm.h: revision 1.9
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.8
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_op_asm.h: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_and_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_cas_16.S: revision 1.3
common/lib/libc/arch/arm/atomic/sync_fetch_and_and_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_32.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_xor_32.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_cas_32.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_xor_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_add_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/sync_fetch_and_add_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_dec_64.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_add_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/sync_fetch_and_sub_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_dec_64.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_add_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_xor_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_add_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.14
common/lib/libc/arch/arm/atomic/atomic_or_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_nand_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_or_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_cas_8.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_inc_64.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_inc_64.S: revision 1.11
common/lib/libc/arch/arm/atomic/atomic_cas_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_dec_32.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_or_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_dec_32.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_xor_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_4.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/atomic_or_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_2.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_swap_64.S: revision 1.14
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_1.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_and_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_and_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_and_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.14

Give the thumb atomic ops a chance of working

s/ifdef _ARM_ARCH_6/if defined(_ARM_ARCH_6)/ for consistency. NFCI.

Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.

That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)
This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)
This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.3.26.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.3.26.1 21-Apr-2020  martin Sync with HEAD
 1.3.4.2 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.3.4.1 23-Jun-2014  tls file atomic_and_16.S was added on branch tls-maxphys on 2014-08-19 23:45:12 +0000
 1.10 28-Jul-2021  skrll Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)

This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)

This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.9 10-Jul-2021  skrll s/ifdef _ARM_ARCH_6/if defined(_ARM_ARCH_6)/ for consistency. NFCI.
 1.8 23-Jun-2014  joerg branches: 1.8.24; 1.8.26;
Add aliases for the builtins used to implement C11/C++11 atomics.
 1.7 22-Feb-2014  martin branches: 1.7.2;
Try to hide the C runtime implementation specific __sync_* ops from librump,
to avoid duplicates.
 1.6 08-Nov-2013  matt Add support for the gcc __sync builtins.
Note that these need earmv6 or later to get the ldrex/strex instructions
 1.5 11-Aug-2013  matt Use foo{s} since it reduces the ifdefs for thumb
 1.4 10-Aug-2013  matt Make these under Thumb2
 1.3 31-Aug-2012  matt branches: 1.3.2;
Add dmb/dsb instructions as required by the armv7 arch man.
 1.2 16-Aug-2008  matt branches: 1.2.2; 1.2.4; 1.2.24;
Add assembly versions of atomic ops with ldrex/strex
 1.1 26-Aug-2007  matt branches: 1.1.2;
file atomic_and_32.S was initially added on branch thorpej-atomic.
 1.1.2.2 27-Aug-2007  matt Don't use r4 (since it's caller saved). Use ip (it's available since we
are leaf functions).
 1.1.2.1 26-Aug-2007  matt Add armv6 variants of atomic ops which use ldrex/strex.
Add atomic_swap (all arm architectures) which uses swp and swpb.
Add membar_ops which uses armv6 data {sync, memory} barrier.
XXX let thorpej figure out the Makefile magic.
 1.2.24.1 27-Nov-2012  matt Pull atomic ops from HEAD.
 1.2.4.2 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.2.4.1 30-Oct-2012  yamt sync with head
 1.2.2.2 18-Sep-2008  wrstuden Sync with wrstuden-revivesa-base-2.
 1.2.2.1 16-Aug-2008  wrstuden file atomic_and_32.S was added on branch wrstuden-revivesa on 2008-09-18 04:54:18 +0000
 1.3.2.1 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.7.2.1 10-Aug-2014  tls Rebase.
 1.8.26.1 11-Aug-2021  martin Pull up following revision(s) (requested by skrll in ticket #1330):

common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.18
common/lib/libc/arch/arm/atomic/atomic_and_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.19
common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_inc_32.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_and_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_add_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_inc_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_cas_up.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_add_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_op_asm.h: revision 1.9
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.8
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_op_asm.h: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_and_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_cas_16.S: revision 1.3
common/lib/libc/arch/arm/atomic/sync_fetch_and_and_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_32.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_xor_32.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_cas_32.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_xor_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_add_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/sync_fetch_and_add_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_dec_64.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_add_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/sync_fetch_and_sub_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_dec_64.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_add_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_xor_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_add_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.14
common/lib/libc/arch/arm/atomic/atomic_or_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_nand_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_or_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_cas_8.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_inc_64.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_inc_64.S: revision 1.11
common/lib/libc/arch/arm/atomic/atomic_cas_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_dec_32.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_or_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_dec_32.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_xor_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_4.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/atomic_or_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_2.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_swap_64.S: revision 1.14
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_1.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_and_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_and_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_and_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.14

Give the thumb atomic ops a chance of working

s/ifdef _ARM_ARCH_6/if defined(_ARM_ARCH_6)/ for consistency. NFCI.

Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.

That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)
This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)
This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.8.24.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.8.24.1 21-Apr-2020  martin Sync with HEAD
 1.13 28-Jul-2021  skrll Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)

This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)

This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.12 10-Jul-2021  skrll s/ifdef _ARM_ARCH_6/if defined(_ARM_ARCH_6)/ for consistency. NFCI.
 1.11 28-Jun-2021  skrll Whitespace
 1.10 04-Mar-2014  matt branches: 1.10.26; 1.10.28;
Don't export __sync* if _KERNEL || _STANDALONE are defined.
(except if _RUMPKERNEL is defined)
 1.9 30-Nov-2013  joerg branches: 1.9.2;
Use explicit form of register pair operations by specifying both.
 1.8 08-Nov-2013  matt Add support for the gcc __sync builtins.
Note that these need earmv6 or later to get the ldrex/strex instructions
 1.7 20-Aug-2013  matt Push two registers to keep stack aligned.
 1.6 11-Aug-2013  matt Use foo{s} since it reduces the ifdefs for thumb
 1.5 10-Aug-2013  matt Make these under Thumb2
 1.4 10-Aug-2013  matt Never modify NLO/NHI (r2,r3) only LO/HI (r0,r1) so subsequent loops will work.
 1.3 10-Aug-2013  matt Use push/pop
 1.2 13-Sep-2012  matt branches: 1.2.2; 1.2.4;
Correct copyright/fix comments.
 1.1 11-Sep-2012  matt branches: 1.1.2;
Add 64bit atomic ops for ARMv6+ (using ldrexd/strexd).
 1.1.2.2 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.1.2.1 20-Nov-2012  tls Resync to 2012-11-19 00:00:00 UTC
 1.2.4.2 27-Nov-2012  matt Pull atomic ops from HEAD.
 1.2.4.1 13-Sep-2012  matt file atomic_and_64.S was added on branch matt-nb6-plus on 2012-11-27 23:42:35 +0000
 1.2.2.3 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.2.2.2 30-Oct-2012  yamt sync with head
 1.2.2.1 13-Sep-2012  yamt file atomic_and_64.S was added on branch yamt-pagecache on 2012-10-30 18:46:10 +0000
 1.9.2.2 15-Feb-2014  matt Add 64-bit atomic ops if ARMv6+
 1.9.2.1 30-Nov-2013  matt file atomic_and_64.S was added on branch matt-nb5-mips64 on 2014-02-15 10:27:44 +0000
 1.10.28.2 11-Aug-2021  martin Pull up following revision(s) (requested by skrll in ticket #1330):

common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.18
common/lib/libc/arch/arm/atomic/atomic_and_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.19
common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_inc_32.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_and_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_add_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_inc_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_cas_up.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_add_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_op_asm.h: revision 1.9
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.8
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_op_asm.h: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_and_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_cas_16.S: revision 1.3
common/lib/libc/arch/arm/atomic/sync_fetch_and_and_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_32.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_xor_32.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_cas_32.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_xor_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_add_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/sync_fetch_and_add_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_dec_64.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_add_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/sync_fetch_and_sub_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_dec_64.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_add_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_xor_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_add_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.14
common/lib/libc/arch/arm/atomic/atomic_or_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_nand_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_or_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_cas_8.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_inc_64.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_inc_64.S: revision 1.11
common/lib/libc/arch/arm/atomic/atomic_cas_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_dec_32.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_or_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_dec_32.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_xor_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_4.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/atomic_or_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_2.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_swap_64.S: revision 1.14
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_1.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_and_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_and_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_and_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.14

Give the thumb atomic ops a chance of working

s/ifdef _ARM_ARCH_6/if defined(_ARM_ARCH_6)/ for consistency. NFCI.

Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.

That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)
This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)
This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.10.28.1 06-Jul-2021  martin Pull up following revision(s) (requested by skrll in ticket #1313):

common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/atomic_cas_8.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.3
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.11
common/lib/libc/arch/arm/atomic/atomic_swap_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.12

Whitespace
 1.10.26.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.10.26.1 21-Apr-2020  martin Sync with HEAD
 1.5 28-Jul-2021  skrll Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)

This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)

This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.4 10-Jul-2021  skrll s/ifdef _ARM_ARCH_6/if defined(_ARM_ARCH_6)/ for consistency. NFCI.
 1.3 23-Jun-2014  joerg branches: 1.3.4; 1.3.26; 1.3.28;
Add aliases for the builtins used to implement C11/C++11 atomics.
 1.2 04-Mar-2014  matt branches: 1.2.2; 1.2.4;
Don't export __sync* if _KERNEL || _STANDALONE are defined.
(except if _RUMPKERNEL is defined)
 1.1 08-Nov-2013  matt Add support for the gcc __sync builtins.
Note that these need earmv6 or later to get the ldrex/strex instructions
 1.2.4.2 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.2.4.1 04-Mar-2014  yamt file atomic_and_8.S was added on branch yamt-pagecache on 2014-05-22 11:26:28 +0000
 1.2.2.1 10-Aug-2014  tls Rebase.
 1.3.28.1 11-Aug-2021  martin Pull up following revision(s) (requested by skrll in ticket #1330):

common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.18
common/lib/libc/arch/arm/atomic/atomic_and_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.19
common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_inc_32.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_and_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_add_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_inc_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_cas_up.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_add_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_op_asm.h: revision 1.9
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.8
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_op_asm.h: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_and_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_cas_16.S: revision 1.3
common/lib/libc/arch/arm/atomic/sync_fetch_and_and_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_32.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_xor_32.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_cas_32.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_xor_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_add_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/sync_fetch_and_add_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_dec_64.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_add_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/sync_fetch_and_sub_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_dec_64.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_add_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_xor_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_add_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.14
common/lib/libc/arch/arm/atomic/atomic_or_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_nand_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_or_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_cas_8.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_inc_64.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_inc_64.S: revision 1.11
common/lib/libc/arch/arm/atomic/atomic_cas_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_dec_32.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_or_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_dec_32.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_xor_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_4.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/atomic_or_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_2.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_swap_64.S: revision 1.14
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_1.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_and_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_and_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_and_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.14

Give the thumb atomic ops a chance of working

s/ifdef _ARM_ARCH_6/if defined(_ARM_ARCH_6)/ for consistency. NFCI.

Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.

That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)
This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)
This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.3.26.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.3.26.1 21-Apr-2020  martin Sync with HEAD
 1.3.4.2 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.3.4.1 23-Jun-2014  tls file atomic_and_8.S was added on branch tls-maxphys on 2014-08-19 23:45:12 +0000
 1.3 28-Jul-2021  skrll Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)

This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)

This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.2 04-Mar-2014  matt branches: 1.2.4; 1.2.8; 1.2.30; 1.2.32;
Don't export __sync* if _KERNEL || _STANDALONE are defined.
(except if _RUMPKERNEL is defined)
 1.1 08-Nov-2013  matt Add support for the gcc __sync builtins.
Note that these need earmv6 or later to get the ldrex/strex instructions
 1.2.32.1 11-Aug-2021  martin Pull up following revision(s) (requested by skrll in ticket #1330):

common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.18
common/lib/libc/arch/arm/atomic/atomic_and_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.19
common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_inc_32.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_and_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_add_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_inc_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_cas_up.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_add_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_op_asm.h: revision 1.9
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.8
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_op_asm.h: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_and_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_cas_16.S: revision 1.3
common/lib/libc/arch/arm/atomic/sync_fetch_and_and_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_32.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_xor_32.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_cas_32.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_xor_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_add_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/sync_fetch_and_add_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_dec_64.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_add_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/sync_fetch_and_sub_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_dec_64.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_add_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_xor_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_add_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.14
common/lib/libc/arch/arm/atomic/atomic_or_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_nand_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_or_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_cas_8.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_inc_64.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_inc_64.S: revision 1.11
common/lib/libc/arch/arm/atomic/atomic_cas_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_dec_32.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_or_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_dec_32.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_xor_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_4.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/atomic_or_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_2.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_swap_64.S: revision 1.14
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_1.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_and_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_and_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_and_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.14

Give the thumb atomic ops a chance of working

s/ifdef _ARM_ARCH_6/if defined(_ARM_ARCH_6)/ for consistency. NFCI.

Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.

That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)
This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)
This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.2.30.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.2.30.1 21-Apr-2020  martin Sync with HEAD
 1.2.8.2 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.2.8.1 04-Mar-2014  tls file atomic_cas_16.S was added on branch tls-maxphys on 2014-08-19 23:45:12 +0000
 1.2.4.2 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.2.4.1 04-Mar-2014  yamt file atomic_cas_16.S was added on branch yamt-pagecache on 2014-05-22 11:26:28 +0000
 1.8 28-Jul-2021  skrll Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)

This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)

This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.7 04-Mar-2014  matt branches: 1.7.26; 1.7.28;
Don't export __sync* if _KERNEL || _STANDALONE are defined.
(except if _RUMPKERNEL is defined)
 1.6 08-Nov-2013  matt Add support for the gcc __sync builtins.
Note that these need earmv6 or later to get the ldrex/strex instructions
 1.5 10-Aug-2013  matt Make these under Thumb2
 1.4 29-Oct-2012  chs _ARCH_ARM_6 -> _ARM_ARCH_6
 1.3 31-Aug-2012  matt branches: 1.3.2;
Add dmb/dsb instructions as required by the armv7 arch man.
 1.2 16-Aug-2008  matt branches: 1.2.2; 1.2.4; 1.2.12; 1.2.24;
Add assembly versions of atomic ops with ldrex/strex
 1.1 18-Apr-2007  thorpej branches: 1.1.2;
file atomic_cas_32.S was initially added on branch thorpej-atomic.
 1.1.2.5 29-Aug-2007  matt Typo. cmpeq should just be cmp
 1.1.2.4 27-Aug-2007  matt Don't use r4 (since it's caller saved). Use ip (it's available since we
are leaf functions).
 1.1.2.3 26-Aug-2007  matt Add armv6 variants of atomic ops which use ldrex/strex.
Add atomic_swap (all arm architectures) which uses swp and swpb.
Add membar_ops which uses armv6 data {sync, memory} barrier.
XXX let thorpej figure out the Makefile magic.
 1.1.2.2 22-Apr-2007  thorpej Make sure namespace-cleansed aliases are avaialble for all atomic ops.
 1.1.2.1 18-Apr-2007  thorpej Atomic ops implementation for ARM. Everything is built on top of
atomic_cas_32(), which is itself a restartable atomic sequence.
 1.2.24.1 27-Nov-2012  matt Pull atomic ops from HEAD.
 1.2.12.1 19-Dec-2013  matt Enable ldrex/strex based atomic ops on armv6/armv7
 1.2.4.2 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.2.4.1 30-Oct-2012  yamt sync with head
 1.2.2.2 18-Sep-2008  wrstuden Sync with wrstuden-revivesa-base-2.
 1.2.2.1 16-Aug-2008  wrstuden file atomic_cas_32.S was added on branch wrstuden-revivesa on 2008-09-18 04:54:18 +0000
 1.3.2.2 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.3.2.1 20-Nov-2012  tls Resync to 2012-11-19 00:00:00 UTC
 1.7.28.1 11-Aug-2021  martin Pull up following revision(s) (requested by skrll in ticket #1330):

common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.18
common/lib/libc/arch/arm/atomic/atomic_and_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.19
common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_inc_32.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_and_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_add_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_inc_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_cas_up.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_add_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_op_asm.h: revision 1.9
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.8
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_op_asm.h: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_and_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_cas_16.S: revision 1.3
common/lib/libc/arch/arm/atomic/sync_fetch_and_and_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_32.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_xor_32.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_cas_32.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_xor_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_add_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/sync_fetch_and_add_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_dec_64.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_add_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/sync_fetch_and_sub_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_dec_64.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_add_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_xor_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_add_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.14
common/lib/libc/arch/arm/atomic/atomic_or_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_nand_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_or_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_cas_8.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_inc_64.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_inc_64.S: revision 1.11
common/lib/libc/arch/arm/atomic/atomic_cas_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_dec_32.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_or_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_dec_32.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_xor_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_4.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/atomic_or_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_2.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_swap_64.S: revision 1.14
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_1.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_and_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_and_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_and_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.14

Give the thumb atomic ops a chance of working

s/ifdef _ARM_ARCH_6/if defined(_ARM_ARCH_6)/ for consistency. NFCI.

Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.

That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)
This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)
This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.7.26.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.7.26.1 21-Apr-2020  martin Sync with HEAD
 1.12 28-Jul-2021  skrll Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)

This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)

This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.11 18-Feb-2019  martin branches: 1.11.2;
Add some atomic_cas_64_ni aliases
 1.10 05-Mar-2014  matt branches: 1.10.26;
apcs-gnu only passes one register on the stack.
ldrd always loads little endian (low address, low register).
 1.9 04-Mar-2014  matt Don't export __sync* if _KERNEL || _STANDALONE are defined.
(except if _RUMPKERNEL is defined)
 1.8 04-Mar-2014  matt Fix #if/#endif nesting
 1.7 04-Mar-2014  matt Fix non-EABI loading of argument. Deal with endian issues.
Fixes PR/48635
 1.6 04-Mar-2014  matt Fetch value from correct stack location. Push an even number of registers
so ldrd won't fail.
 1.5 30-Nov-2013  joerg branches: 1.5.2;
Use explicit form of register pair operations by specifying both.
 1.4 08-Nov-2013  matt Add support for the gcc __sync builtins.
Note that these need earmv6 or later to get the ldrex/strex instructions
 1.3 10-Aug-2013  matt Make these under Thumb2
 1.2 29-Oct-2012  chs branches: 1.2.2; 1.2.4;
_ARCH_ARM_6 -> _ARM_ARCH_6
 1.1 11-Sep-2012  matt branches: 1.1.2;
Add 64bit atomic ops for ARMv6+ (using ldrexd/strexd).
 1.1.2.2 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.1.2.1 20-Nov-2012  tls Resync to 2012-11-19 00:00:00 UTC
 1.2.4.2 27-Nov-2012  matt Pull atomic ops from HEAD.
 1.2.4.1 29-Oct-2012  matt file atomic_cas_64.S was added on branch matt-nb6-plus on 2012-11-27 23:42:35 +0000
 1.2.2.3 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.2.2.2 30-Oct-2012  yamt sync with head
 1.2.2.1 29-Oct-2012  yamt file atomic_cas_64.S was added on branch yamt-pagecache on 2012-10-30 18:46:10 +0000
 1.5.2.2 15-Feb-2014  matt Add 64-bit atomic ops if ARMv6+
 1.5.2.1 30-Nov-2013  matt file atomic_cas_64.S was added on branch matt-nb5-mips64 on 2014-02-15 10:27:44 +0000
 1.10.26.2 21-Apr-2020  martin Sync with HEAD
 1.10.26.1 10-Jun-2019  christos Sync with HEAD
 1.11.2.1 11-Aug-2021  martin Pull up following revision(s) (requested by skrll in ticket #1330):

common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.18
common/lib/libc/arch/arm/atomic/atomic_and_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.19
common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_inc_32.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_and_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_add_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_inc_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_cas_up.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_add_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_op_asm.h: revision 1.9
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.8
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_op_asm.h: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_and_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_cas_16.S: revision 1.3
common/lib/libc/arch/arm/atomic/sync_fetch_and_and_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_32.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_xor_32.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_cas_32.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_xor_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_add_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/sync_fetch_and_add_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_dec_64.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_add_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/sync_fetch_and_sub_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_dec_64.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_add_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_xor_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_add_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.14
common/lib/libc/arch/arm/atomic/atomic_or_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_nand_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_or_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_cas_8.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_inc_64.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_inc_64.S: revision 1.11
common/lib/libc/arch/arm/atomic/atomic_cas_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_dec_32.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_or_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_dec_32.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_xor_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_4.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/atomic_or_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_2.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_swap_64.S: revision 1.14
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_1.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_and_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_and_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_and_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.14

Give the thumb atomic ops a chance of working

s/ifdef _ARM_ARCH_6/if defined(_ARM_ARCH_6)/ for consistency. NFCI.

Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.

That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)
This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)
This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.9 28-Jul-2021  skrll Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)

This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)

This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.8 29-Jun-2021  skrll Whitespace
 1.7 04-Mar-2014  matt branches: 1.7.26; 1.7.28;
Don't export __sync* if _KERNEL || _STANDALONE are defined.
(except if _RUMPKERNEL is defined)
 1.6 08-Nov-2013  matt Add support for the gcc __sync builtins.
Note that these need earmv6 or later to get the ldrex/strex instructions
 1.5 10-Aug-2013  matt Make these under Thumb2
 1.4 29-Oct-2012  chs _ARCH_ARM_6 -> _ARM_ARCH_6
 1.3 31-Aug-2012  matt branches: 1.3.2;
Add dmb/dsb instructions as required by the armv7 arch man.
 1.2 16-Aug-2012  matt Actually use the assembly version of the atomic function if compiling
for armv6 or armv7 cpus. Use atomic_cas_ptr instead of _lock_cas so
we pick up the assembly version when it's used.
 1.1 18-Nov-2008  matt branches: 1.1.8; 1.1.12;
Add an atomic_cas_8 which uses ldrex/strex.
 1.1.12.1 27-Nov-2012  matt Pull atomic ops from HEAD.
 1.1.8.2 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.1.8.1 30-Oct-2012  yamt sync with head
 1.3.2.2 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.3.2.1 20-Nov-2012  tls Resync to 2012-11-19 00:00:00 UTC
 1.7.28.2 11-Aug-2021  martin Pull up following revision(s) (requested by skrll in ticket #1330):

common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.18
common/lib/libc/arch/arm/atomic/atomic_and_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.19
common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_inc_32.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_and_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_add_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_inc_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_cas_up.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_add_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_op_asm.h: revision 1.9
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.8
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_op_asm.h: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_and_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_cas_16.S: revision 1.3
common/lib/libc/arch/arm/atomic/sync_fetch_and_and_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_32.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_xor_32.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_cas_32.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_xor_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_add_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/sync_fetch_and_add_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_dec_64.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_add_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/sync_fetch_and_sub_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_dec_64.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_add_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_xor_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_add_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.14
common/lib/libc/arch/arm/atomic/atomic_or_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_nand_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_or_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_cas_8.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_inc_64.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_inc_64.S: revision 1.11
common/lib/libc/arch/arm/atomic/atomic_cas_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_dec_32.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_or_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_dec_32.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_xor_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_4.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/atomic_or_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_2.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_swap_64.S: revision 1.14
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_1.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_and_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_and_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_and_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.14

Give the thumb atomic ops a chance of working

s/ifdef _ARM_ARCH_6/if defined(_ARM_ARCH_6)/ for consistency. NFCI.

Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.

That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)
This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)
This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.7.28.1 06-Jul-2021  martin Pull up following revision(s) (requested by skrll in ticket #1313):

common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/atomic_cas_8.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.3
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.11
common/lib/libc/arch/arm/atomic/atomic_swap_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.12

Whitespace
 1.7.26.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.7.26.1 21-Apr-2020  martin Sync with HEAD
 1.8 09-Mar-2020  skrll Give the thumb atomic ops a chance of working
 1.7 04-Mar-2014  matt branches: 1.7.26; 1.7.28;
Load new value from correct stack location in _atomic_cas_64_up
 1.6 27-Feb-2014  matt Add atomic_cas_64 support for ARM EABI on V5TE and V5TEJ cpus.
(strd is atomic).
 1.5 27-Jan-2014  matt Add _atomic_cas_16_up and _atomic_cas_8_up
 1.4 19-Aug-2013  matt Thumbify
 1.3 10-Aug-2013  matt Make these under Thumb2
 1.2 25-May-2008  chs branches: 1.2.2; 1.2.24;
enable profiling of assembly functions.
 1.1 29-Apr-2008  scw branches: 1.1.2; 1.1.4;
Implement _atomic_cas_up() in assembly code as the compiler cannot be
trusted to generate fully restartable code sequences.

Addresses lib/38482 for ARM and m68000.
 1.1.4.3 04-Jun-2008  yamt sync with head
 1.1.4.2 18-May-2008  yamt sync with head.
 1.1.4.1 29-Apr-2008  yamt file atomic_cas_up.S was added on branch yamt-pf42 on 2008-05-18 12:28:44 +0000
 1.1.2.1 23-Jun-2008  wrstuden Sync w/ -current. 34 merge conflicts to follow.
 1.2.24.1 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.2.2.1 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.7.28.1 11-Aug-2021  martin Pull up following revision(s) (requested by skrll in ticket #1330):

common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.18
common/lib/libc/arch/arm/atomic/atomic_and_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.19
common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_inc_32.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_and_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_add_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_inc_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_cas_up.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_add_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_op_asm.h: revision 1.9
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.8
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_op_asm.h: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_and_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_cas_16.S: revision 1.3
common/lib/libc/arch/arm/atomic/sync_fetch_and_and_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_32.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_xor_32.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_cas_32.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_xor_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_add_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/sync_fetch_and_add_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_dec_64.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_add_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/sync_fetch_and_sub_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_dec_64.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_add_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_xor_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_add_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.14
common/lib/libc/arch/arm/atomic/atomic_or_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_nand_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_or_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_cas_8.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_inc_64.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_inc_64.S: revision 1.11
common/lib/libc/arch/arm/atomic/atomic_cas_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_dec_32.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_or_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_dec_32.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_xor_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_4.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/atomic_or_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_2.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_swap_64.S: revision 1.14
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_1.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_and_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_and_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_and_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.14

Give the thumb atomic ops a chance of working

s/ifdef _ARM_ARCH_6/if defined(_ARM_ARCH_6)/ for consistency. NFCI.

Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.

That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)
This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)
This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.7.26.2 21-Apr-2020  martin Sync with HEAD
 1.7.26.1 08-Apr-2020  martin Merge changes from current as of 20200406
 1.7 28-Jul-2021  skrll Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)

This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)

This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.6 10-Jul-2021  skrll s/ifdef _ARM_ARCH_6/if defined(_ARM_ARCH_6)/ for consistency. NFCI.
 1.5 11-Aug-2013  matt branches: 1.5.26; 1.5.28;
Use foo{s} since it reduces the ifdefs for thumb
 1.4 10-Aug-2013  matt Make these under Thumb2
 1.3 31-Aug-2012  matt branches: 1.3.2;
Add dmb/dsb instructions as required by the armv7 arch man.
 1.2 16-Aug-2008  matt branches: 1.2.2; 1.2.4; 1.2.24;
Add assembly versions of atomic ops with ldrex/strex
 1.1 26-Aug-2007  matt branches: 1.1.2;
file atomic_dec_32.S was initially added on branch thorpej-atomic.
 1.1.2.1 26-Aug-2007  matt Add armv6 variants of atomic ops which use ldrex/strex.
Add atomic_swap (all arm architectures) which uses swp and swpb.
Add membar_ops which uses armv6 data {sync, memory} barrier.
XXX let thorpej figure out the Makefile magic.
 1.2.24.1 27-Nov-2012  matt Pull atomic ops from HEAD.
 1.2.4.2 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.2.4.1 30-Oct-2012  yamt sync with head
 1.2.2.2 18-Sep-2008  wrstuden Sync with wrstuden-revivesa-base-2.
 1.2.2.1 16-Aug-2008  wrstuden file atomic_dec_32.S was added on branch wrstuden-revivesa on 2008-09-18 04:54:18 +0000
 1.3.2.1 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.5.28.1 11-Aug-2021  martin Pull up following revision(s) (requested by skrll in ticket #1330):

common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.18
common/lib/libc/arch/arm/atomic/atomic_and_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.19
common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_inc_32.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_and_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_add_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_inc_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_cas_up.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_add_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_op_asm.h: revision 1.9
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.8
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_op_asm.h: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_and_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_cas_16.S: revision 1.3
common/lib/libc/arch/arm/atomic/sync_fetch_and_and_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_32.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_xor_32.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_cas_32.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_xor_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_add_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/sync_fetch_and_add_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_dec_64.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_add_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/sync_fetch_and_sub_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_dec_64.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_add_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_xor_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_add_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.14
common/lib/libc/arch/arm/atomic/atomic_or_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_nand_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_or_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_cas_8.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_inc_64.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_inc_64.S: revision 1.11
common/lib/libc/arch/arm/atomic/atomic_cas_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_dec_32.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_or_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_dec_32.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_xor_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_4.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/atomic_or_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_2.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_swap_64.S: revision 1.14
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_1.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_and_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_and_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_and_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.14

Give the thumb atomic ops a chance of working

s/ifdef _ARM_ARCH_6/if defined(_ARM_ARCH_6)/ for consistency. NFCI.

Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.

That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)
This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)
This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.5.26.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.5.26.1 21-Apr-2020  martin Sync with HEAD
 1.9 28-Jul-2021  skrll Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)

This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)

This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.8 10-Jul-2021  skrll s/ifdef _ARM_ARCH_6/if defined(_ARM_ARCH_6)/ for consistency. NFCI.
 1.7 30-Nov-2013  joerg branches: 1.7.2; 1.7.28; 1.7.30;
Use explicit form of register pair operations by specifying both.
 1.6 11-Aug-2013  matt Use foo{s} since it reduces the ifdefs for thumb
 1.5 10-Aug-2013  matt Make these under Thumb2
 1.4 10-Aug-2013  matt Use r2 instead of ip
 1.3 10-Aug-2013  matt Use subs/sbc
Use push/pop
Don't adjust return for atomic_dec_64 since it's void and returns nothing
 1.2 13-Sep-2012  matt branches: 1.2.2; 1.2.4;
Correct copyright/fix comments.
 1.1 11-Sep-2012  matt branches: 1.1.2;
Add 64bit atomic ops for ARMv6+ (using ldrexd/strexd).
 1.1.2.2 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.1.2.1 20-Nov-2012  tls Resync to 2012-11-19 00:00:00 UTC
 1.2.4.2 27-Nov-2012  matt Pull atomic ops from HEAD.
 1.2.4.1 13-Sep-2012  matt file atomic_dec_64.S was added on branch matt-nb6-plus on 2012-11-27 23:42:35 +0000
 1.2.2.3 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.2.2.2 30-Oct-2012  yamt sync with head
 1.2.2.1 13-Sep-2012  yamt file atomic_dec_64.S was added on branch yamt-pagecache on 2012-10-30 18:46:11 +0000
 1.7.30.1 11-Aug-2021  martin Pull up following revision(s) (requested by skrll in ticket #1330):

common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.18
common/lib/libc/arch/arm/atomic/atomic_and_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.19
common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_inc_32.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_and_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_add_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_inc_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_cas_up.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_add_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_op_asm.h: revision 1.9
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.8
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_op_asm.h: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_and_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_cas_16.S: revision 1.3
common/lib/libc/arch/arm/atomic/sync_fetch_and_and_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_32.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_xor_32.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_cas_32.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_xor_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_add_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/sync_fetch_and_add_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_dec_64.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_add_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/sync_fetch_and_sub_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_dec_64.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_add_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_xor_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_add_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.14
common/lib/libc/arch/arm/atomic/atomic_or_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_nand_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_or_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_cas_8.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_inc_64.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_inc_64.S: revision 1.11
common/lib/libc/arch/arm/atomic/atomic_cas_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_dec_32.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_or_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_dec_32.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_xor_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_4.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/atomic_or_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_2.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_swap_64.S: revision 1.14
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_1.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_and_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_and_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_and_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.14

Give the thumb atomic ops a chance of working

s/ifdef _ARM_ARCH_6/if defined(_ARM_ARCH_6)/ for consistency. NFCI.

Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.

That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)
This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)
This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.7.28.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.7.28.1 21-Apr-2020  martin Sync with HEAD
 1.7.2.2 15-Feb-2014  matt Add 64-bit atomic ops if ARMv6+
 1.7.2.1 30-Nov-2013  matt file atomic_dec_64.S was added on branch matt-nb5-mips64 on 2014-02-15 10:27:44 +0000
 1.9 28-Jul-2021  skrll Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)

This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)

This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.8 10-Jul-2021  skrll s/ifdef _ARM_ARCH_6/if defined(_ARM_ARCH_6)/ for consistency. NFCI.
 1.7 08-Nov-2013  matt branches: 1.7.26; 1.7.28;
Add support for the gcc __sync builtins.
Note that these need earmv6 or later to get the ldrex/strex instructions
 1.6 11-Aug-2013  matt Use foo{s} since it reduces the ifdefs for thumb
 1.5 10-Aug-2013  matt Make these under Thumb2
 1.4 29-Oct-2012  chs _ARCH_ARM_6 -> _ARM_ARCH_6
 1.3 31-Aug-2012  matt branches: 1.3.2;
Add dmb/dsb instructions as required by the armv7 arch man.
 1.2 16-Aug-2008  matt branches: 1.2.2; 1.2.4; 1.2.24;
Add assembly versions of atomic ops with ldrex/strex
 1.1 26-Aug-2007  matt branches: 1.1.2;
file atomic_inc_32.S was initially added on branch thorpej-atomic.
 1.1.2.1 26-Aug-2007  matt Add armv6 variants of atomic ops which use ldrex/strex.
Add atomic_swap (all arm architectures) which uses swp and swpb.
Add membar_ops which uses armv6 data {sync, memory} barrier.
XXX let thorpej figure out the Makefile magic.
 1.2.24.1 27-Nov-2012  matt Pull atomic ops from HEAD.
 1.2.4.2 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.2.4.1 30-Oct-2012  yamt sync with head
 1.2.2.2 18-Sep-2008  wrstuden Sync with wrstuden-revivesa-base-2.
 1.2.2.1 16-Aug-2008  wrstuden file atomic_inc_32.S was added on branch wrstuden-revivesa on 2008-09-18 04:54:18 +0000
 1.3.2.2 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.3.2.1 20-Nov-2012  tls Resync to 2012-11-19 00:00:00 UTC
 1.7.28.1 11-Aug-2021  martin Pull up following revision(s) (requested by skrll in ticket #1330):

common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.18
common/lib/libc/arch/arm/atomic/atomic_and_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.19
common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_inc_32.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_and_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_add_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_inc_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_cas_up.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_add_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_op_asm.h: revision 1.9
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.8
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_op_asm.h: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_and_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_cas_16.S: revision 1.3
common/lib/libc/arch/arm/atomic/sync_fetch_and_and_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_32.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_xor_32.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_cas_32.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_xor_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_add_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/sync_fetch_and_add_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_dec_64.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_add_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/sync_fetch_and_sub_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_dec_64.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_add_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_xor_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_add_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.14
common/lib/libc/arch/arm/atomic/atomic_or_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_nand_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_or_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_cas_8.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_inc_64.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_inc_64.S: revision 1.11
common/lib/libc/arch/arm/atomic/atomic_cas_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_dec_32.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_or_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_dec_32.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_xor_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_4.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/atomic_or_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_2.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_swap_64.S: revision 1.14
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_1.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_and_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_and_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_and_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.14

Give the thumb atomic ops a chance of working

s/ifdef _ARM_ARCH_6/if defined(_ARM_ARCH_6)/ for consistency. NFCI.

Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.

That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)
This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)
This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.7.26.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.7.26.1 21-Apr-2020  martin Sync with HEAD
 1.11 28-Jul-2021  skrll Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)

This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)

This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.10 10-Jul-2021  skrll s/ifdef _ARM_ARCH_6/if defined(_ARM_ARCH_6)/ for consistency. NFCI.
 1.9 30-Nov-2013  joerg branches: 1.9.2; 1.9.28; 1.9.30;
Use explicit form of register pair operations by specifying both.
 1.8 11-Aug-2013  matt Use foo{s} since it reduces the ifdefs for thumb
 1.7 10-Aug-2013  matt Make these under Thumb2
 1.6 10-Aug-2013  matt Use r2 instead of ip
 1.5 10-Aug-2013  matt Don't adjust return of atomic_inc_64 since it's void and doesn't return
anything.
 1.4 10-Aug-2013  matt Fix add -> adds
 1.3 29-Oct-2012  chs branches: 1.3.2; 1.3.4;
_ARCH_ARM_6 -> _ARM_ARCH_6
 1.2 13-Sep-2012  matt Correct copyright/fix comments.
 1.1 11-Sep-2012  matt branches: 1.1.2;
Add 64bit atomic ops for ARMv6+ (using ldrexd/strexd).
 1.1.2.2 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.1.2.1 20-Nov-2012  tls Resync to 2012-11-19 00:00:00 UTC
 1.3.4.2 27-Nov-2012  matt Pull atomic ops from HEAD.
 1.3.4.1 29-Oct-2012  matt file atomic_inc_64.S was added on branch matt-nb6-plus on 2012-11-27 23:42:35 +0000
 1.3.2.3 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.3.2.2 30-Oct-2012  yamt sync with head
 1.3.2.1 29-Oct-2012  yamt file atomic_inc_64.S was added on branch yamt-pagecache on 2012-10-30 18:46:11 +0000
 1.9.30.1 11-Aug-2021  martin Pull up following revision(s) (requested by skrll in ticket #1330):

common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.18
common/lib/libc/arch/arm/atomic/atomic_and_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.19
common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_inc_32.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_and_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_add_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_inc_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_cas_up.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_add_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_op_asm.h: revision 1.9
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.8
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_op_asm.h: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_and_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_cas_16.S: revision 1.3
common/lib/libc/arch/arm/atomic/sync_fetch_and_and_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_32.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_xor_32.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_cas_32.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_xor_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_add_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/sync_fetch_and_add_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_dec_64.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_add_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/sync_fetch_and_sub_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_dec_64.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_add_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_xor_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_add_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.14
common/lib/libc/arch/arm/atomic/atomic_or_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_nand_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_or_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_cas_8.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_inc_64.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_inc_64.S: revision 1.11
common/lib/libc/arch/arm/atomic/atomic_cas_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_dec_32.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_or_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_dec_32.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_xor_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_4.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/atomic_or_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_2.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_swap_64.S: revision 1.14
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_1.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_and_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_and_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_and_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.14

Give the thumb atomic ops a chance of working

s/ifdef _ARM_ARCH_6/if defined(_ARM_ARCH_6)/ for consistency. NFCI.

Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.

That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)
This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)
This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.9.28.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.9.28.1 21-Apr-2020  martin Sync with HEAD
 1.9.2.2 15-Feb-2014  matt Add 64-bit atomic ops if ARMv6+
 1.9.2.1 30-Nov-2013  matt file atomic_inc_64.S was added on branch matt-nb5-mips64 on 2014-02-15 10:27:44 +0000
 1.5 28-Jul-2021  skrll Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)

This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)

This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.4 10-Jul-2021  skrll s/ifdef _ARM_ARCH_6/if defined(_ARM_ARCH_6)/ for consistency. NFCI.
 1.3 11-Dec-2015  skrll branches: 1.3.16; 1.3.18;
Use gcc 4.4 and later operation for nand, i.e.
*ptr = ~(tmp & value) instead of *ptr = ~tmp & value

There was also another bug in sync_fetch_and_nand_8 which I've also fixed.

PR port-arm32/50513: Incorrect logic for atomic_nand_xx.S
 1.2 04-Mar-2014  matt branches: 1.2.4; 1.2.6; 1.2.8;
Don't export __sync* if _KERNEL || _STANDALONE are defined.
(except if _RUMPKERNEL is defined)
 1.1 08-Nov-2013  matt Add support for the gcc __sync builtins.
Note that these need earmv6 or later to get the ldrex/strex instructions
 1.2.8.2 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.2.8.1 04-Mar-2014  tls file atomic_nand_16.S was added on branch tls-maxphys on 2014-08-19 23:45:12 +0000
 1.2.6.1 26-Feb-2016  snj Pull up following revision(s) (requested by skrll in ticket #1105):
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.3
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.3
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.3
common/lib/libc/arch/arm/atomic/sync_fetch_and_nand_8.S: revision 1.5
Use gcc 4.4 and later operation for nand, i.e.
*ptr = ~(tmp & value) instead of *ptr = ~tmp & value
There was also another bug in sync_fetch_and_nand_8 which I've also fixed.
PR port-arm32/50513: Incorrect logic for atomic_nand_xx.S
 1.2.4.2 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.2.4.1 04-Mar-2014  yamt file atomic_nand_16.S was added on branch yamt-pagecache on 2014-05-22 11:26:28 +0000
 1.3.18.1 11-Aug-2021  martin Pull up following revision(s) (requested by skrll in ticket #1330):

common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.18
common/lib/libc/arch/arm/atomic/atomic_and_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.19
common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_inc_32.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_and_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_add_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_inc_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_cas_up.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_add_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_op_asm.h: revision 1.9
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.8
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_op_asm.h: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_and_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_cas_16.S: revision 1.3
common/lib/libc/arch/arm/atomic/sync_fetch_and_and_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_32.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_xor_32.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_cas_32.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_xor_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_add_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/sync_fetch_and_add_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_dec_64.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_add_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/sync_fetch_and_sub_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_dec_64.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_add_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_xor_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_add_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.14
common/lib/libc/arch/arm/atomic/atomic_or_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_nand_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_or_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_cas_8.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_inc_64.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_inc_64.S: revision 1.11
common/lib/libc/arch/arm/atomic/atomic_cas_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_dec_32.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_or_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_dec_32.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_xor_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_4.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/atomic_or_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_2.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_swap_64.S: revision 1.14
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_1.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_and_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_and_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_and_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.14

Give the thumb atomic ops a chance of working

s/ifdef _ARM_ARCH_6/if defined(_ARM_ARCH_6)/ for consistency. NFCI.

Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.

That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)
This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)
This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.3.16.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.3.16.1 21-Apr-2020  martin Sync with HEAD
 1.5 28-Jul-2021  skrll Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)

This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)

This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.4 10-Jul-2021  skrll s/ifdef _ARM_ARCH_6/if defined(_ARM_ARCH_6)/ for consistency. NFCI.
 1.3 11-Dec-2015  skrll branches: 1.3.16; 1.3.18;
Use gcc 4.4 and later operation for nand, i.e.
*ptr = ~(tmp & value) instead of *ptr = ~tmp & value

There was also another bug in sync_fetch_and_nand_8 which I've also fixed.

PR port-arm32/50513: Incorrect logic for atomic_nand_xx.S
 1.2 04-Mar-2014  matt branches: 1.2.4; 1.2.6; 1.2.8;
Don't export __sync* if _KERNEL || _STANDALONE are defined.
(except if _RUMPKERNEL is defined)
 1.1 08-Nov-2013  matt Add support for the gcc __sync builtins.
Note that these need earmv6 or later to get the ldrex/strex instructions
 1.2.8.2 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.2.8.1 04-Mar-2014  tls file atomic_nand_32.S was added on branch tls-maxphys on 2014-08-19 23:45:12 +0000
 1.2.6.1 26-Feb-2016  snj Pull up following revision(s) (requested by skrll in ticket #1105):
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.3
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.3
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.3
common/lib/libc/arch/arm/atomic/sync_fetch_and_nand_8.S: revision 1.5
Use gcc 4.4 and later operation for nand, i.e.
*ptr = ~(tmp & value) instead of *ptr = ~tmp & value
There was also another bug in sync_fetch_and_nand_8 which I've also fixed.
PR port-arm32/50513: Incorrect logic for atomic_nand_xx.S
 1.2.4.2 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.2.4.1 04-Mar-2014  yamt file atomic_nand_32.S was added on branch yamt-pagecache on 2014-05-22 11:26:28 +0000
 1.3.18.1 11-Aug-2021  martin Pull up following revision(s) (requested by skrll in ticket #1330):

common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.18
common/lib/libc/arch/arm/atomic/atomic_and_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.19
common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_inc_32.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_and_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_add_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_inc_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_cas_up.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_add_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_op_asm.h: revision 1.9
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.8
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_op_asm.h: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_and_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_cas_16.S: revision 1.3
common/lib/libc/arch/arm/atomic/sync_fetch_and_and_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_32.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_xor_32.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_cas_32.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_xor_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_add_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/sync_fetch_and_add_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_dec_64.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_add_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/sync_fetch_and_sub_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_dec_64.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_add_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_xor_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_add_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.14
common/lib/libc/arch/arm/atomic/atomic_or_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_nand_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_or_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_cas_8.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_inc_64.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_inc_64.S: revision 1.11
common/lib/libc/arch/arm/atomic/atomic_cas_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_dec_32.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_or_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_dec_32.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_xor_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_4.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/atomic_or_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_2.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_swap_64.S: revision 1.14
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_1.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_and_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_and_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_and_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.14

Give the thumb atomic ops a chance of working

s/ifdef _ARM_ARCH_6/if defined(_ARM_ARCH_6)/ for consistency. NFCI.

Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.

That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)
This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)
This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.3.16.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.3.16.1 21-Apr-2020  martin Sync with HEAD
 1.7 28-Jul-2021  skrll Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)

This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)

This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.6 10-Jul-2021  skrll s/ifdef _ARM_ARCH_6/if defined(_ARM_ARCH_6)/ for consistency. NFCI.
 1.5 28-Jun-2021  skrll Whitespace
 1.4 11-Dec-2015  skrll branches: 1.4.16; 1.4.18;
Use gcc 4.4 and later operation for nand, i.e.
*ptr = ~(tmp & value) instead of *ptr = ~tmp & value

There was also another bug in sync_fetch_and_nand_8 which I've also fixed.

PR port-arm32/50513: Incorrect logic for atomic_nand_xx.S
 1.3 04-Mar-2014  matt branches: 1.3.4; 1.3.6; 1.3.8;
Don't export __sync* if _KERNEL || _STANDALONE are defined.
(except if _RUMPKERNEL is defined)
 1.2 30-Nov-2013  joerg Use explicit form of register pair operations by specifying both.
 1.1 08-Nov-2013  matt Add support for the gcc __sync builtins.
Note that these need earmv6 or later to get the ldrex/strex instructions
 1.3.8.2 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.3.8.1 04-Mar-2014  tls file atomic_nand_64.S was added on branch tls-maxphys on 2014-08-19 23:45:12 +0000
 1.3.6.1 26-Feb-2016  snj Pull up following revision(s) (requested by skrll in ticket #1105):
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.3
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.3
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.3
common/lib/libc/arch/arm/atomic/sync_fetch_and_nand_8.S: revision 1.5
Use gcc 4.4 and later operation for nand, i.e.
*ptr = ~(tmp & value) instead of *ptr = ~tmp & value
There was also another bug in sync_fetch_and_nand_8 which I've also fixed.
PR port-arm32/50513: Incorrect logic for atomic_nand_xx.S
 1.3.4.2 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.3.4.1 04-Mar-2014  yamt file atomic_nand_64.S was added on branch yamt-pagecache on 2014-05-22 11:26:28 +0000
 1.4.18.2 11-Aug-2021  martin Pull up following revision(s) (requested by skrll in ticket #1330):

common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.18
common/lib/libc/arch/arm/atomic/atomic_and_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.19
common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_inc_32.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_and_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_add_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_inc_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_cas_up.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_add_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_op_asm.h: revision 1.9
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.8
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_op_asm.h: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_and_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_cas_16.S: revision 1.3
common/lib/libc/arch/arm/atomic/sync_fetch_and_and_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_32.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_xor_32.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_cas_32.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_xor_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_add_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/sync_fetch_and_add_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_dec_64.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_add_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/sync_fetch_and_sub_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_dec_64.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_add_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_xor_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_add_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.14
common/lib/libc/arch/arm/atomic/atomic_or_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_nand_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_or_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_cas_8.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_inc_64.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_inc_64.S: revision 1.11
common/lib/libc/arch/arm/atomic/atomic_cas_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_dec_32.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_or_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_dec_32.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_xor_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_4.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/atomic_or_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_2.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_swap_64.S: revision 1.14
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_1.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_and_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_and_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_and_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.14

Give the thumb atomic ops a chance of working

s/ifdef _ARM_ARCH_6/if defined(_ARM_ARCH_6)/ for consistency. NFCI.

Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.

That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)
This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)
This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.4.18.1 06-Jul-2021  martin Pull up following revision(s) (requested by skrll in ticket #1313):

common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/atomic_cas_8.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.3
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.11
common/lib/libc/arch/arm/atomic/atomic_swap_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.12

Whitespace
 1.4.16.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.4.16.1 21-Apr-2020  martin Sync with HEAD
 1.5 28-Jul-2021  skrll Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)

This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)

This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.4 10-Jul-2021  skrll s/ifdef _ARM_ARCH_6/if defined(_ARM_ARCH_6)/ for consistency. NFCI.
 1.3 11-Dec-2015  skrll branches: 1.3.16; 1.3.18;
Use gcc 4.4 and later operation for nand, i.e.
*ptr = ~(tmp & value) instead of *ptr = ~tmp & value

There was also another bug in sync_fetch_and_nand_8 which I've also fixed.

PR port-arm32/50513: Incorrect logic for atomic_nand_xx.S
 1.2 04-Mar-2014  matt branches: 1.2.4; 1.2.6; 1.2.8;
Don't export __sync* if _KERNEL || _STANDALONE are defined.
(except if _RUMPKERNEL is defined)
 1.1 08-Nov-2013  matt Add support for the gcc __sync builtins.
Note that these need earmv6 or later to get the ldrex/strex instructions
 1.2.8.2 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.2.8.1 04-Mar-2014  tls file atomic_nand_8.S was added on branch tls-maxphys on 2014-08-19 23:45:12 +0000
 1.2.6.1 26-Feb-2016  snj Pull up following revision(s) (requested by skrll in ticket #1105):
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.3
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.3
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.3
common/lib/libc/arch/arm/atomic/sync_fetch_and_nand_8.S: revision 1.5
Use gcc 4.4 and later operation for nand, i.e.
*ptr = ~(tmp & value) instead of *ptr = ~tmp & value
There was also another bug in sync_fetch_and_nand_8 which I've also fixed.
PR port-arm32/50513: Incorrect logic for atomic_nand_xx.S
 1.2.4.2 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.2.4.1 04-Mar-2014  yamt file atomic_nand_8.S was added on branch yamt-pagecache on 2014-05-22 11:26:28 +0000
 1.3.18.1 11-Aug-2021  martin Pull up following revision(s) (requested by skrll in ticket #1330):

common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.18
common/lib/libc/arch/arm/atomic/atomic_and_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.19
common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_inc_32.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_and_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_add_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_inc_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_cas_up.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_add_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_op_asm.h: revision 1.9
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.8
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_op_asm.h: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_and_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_cas_16.S: revision 1.3
common/lib/libc/arch/arm/atomic/sync_fetch_and_and_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_32.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_xor_32.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_cas_32.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_xor_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_add_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/sync_fetch_and_add_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_dec_64.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_add_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/sync_fetch_and_sub_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_dec_64.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_add_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_xor_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_add_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.14
common/lib/libc/arch/arm/atomic/atomic_or_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_nand_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_or_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_cas_8.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_inc_64.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_inc_64.S: revision 1.11
common/lib/libc/arch/arm/atomic/atomic_cas_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_dec_32.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_or_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_dec_32.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_xor_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_4.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/atomic_or_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_2.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_swap_64.S: revision 1.14
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_1.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_and_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_and_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_and_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.14

Give the thumb atomic ops a chance of working

s/ifdef _ARM_ARCH_6/if defined(_ARM_ARCH_6)/ for consistency. NFCI.

Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.

That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)
This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)
This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.3.16.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.3.16.1 21-Apr-2020  martin Sync with HEAD
 1.10 28-Jul-2021  simonb #define<tab> consistency.
 1.9 28-Jul-2021  skrll Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)

This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)

This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.8 16-Sep-2019  skrll Traiing whitespace.
 1.7 17-May-2015  justin branches: 1.7.16; 1.7.18;
Do not build arm toolchain symbols in the rump kernel

These symbols will be provided at link time and will be
duplicate symbols in rump kernel and libc if defined.

Many have been fixed previously, but these were missed
as did not have a test.
 1.6 04-Mar-2014  matt Don't export __sync* if _KERNEL || _STANDALONE are defined.
(except if _RUMPKERNEL is defined)
 1.5 22-Feb-2014  martin Try to hide the C runtime implementation specific __sync_* ops from librump,
to avoid duplicates.
 1.4 08-Nov-2013  matt Add support for the gcc __sync builtins.
Note that these need earmv6 or later to get the ldrex/strex instructions
 1.3 11-Sep-2012  matt branches: 1.3.2;
Add 64bit atomic ops for ARMv6+ (using ldrexd/strexd).
 1.2 16-Aug-2008  matt branches: 1.2.2; 1.2.4; 1.2.12; 1.2.24;
Add assembly versions of atomic ops with ldrex/strex
 1.1 18-Apr-2007  thorpej branches: 1.1.2;
file atomic_op_asm.h was initially added on branch thorpej-atomic.
 1.1.2.1 18-Apr-2007  thorpej Atomic ops implementation for ARM. Everything is built on top of
atomic_cas_32(), which is itself a restartable atomic sequence.
 1.2.24.1 27-Nov-2012  matt Pull atomic ops from HEAD.
 1.2.12.1 15-Feb-2014  matt Add 64-bit atomic ops if ARMv6+
 1.2.4.2 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.2.4.1 30-Oct-2012  yamt sync with head
 1.2.2.2 18-Sep-2008  wrstuden Sync with wrstuden-revivesa-base-2.
 1.2.2.1 16-Aug-2008  wrstuden file atomic_op_asm.h was added on branch wrstuden-revivesa on 2008-09-18 04:54:18 +0000
 1.3.2.1 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.7.18.1 11-Aug-2021  martin Pull up following revision(s) (requested by skrll in ticket #1330):

common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.18
common/lib/libc/arch/arm/atomic/atomic_and_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.19
common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_inc_32.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_and_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_add_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_inc_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_cas_up.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_add_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_op_asm.h: revision 1.9
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.8
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_op_asm.h: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_and_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_cas_16.S: revision 1.3
common/lib/libc/arch/arm/atomic/sync_fetch_and_and_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_32.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_xor_32.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_cas_32.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_xor_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_add_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/sync_fetch_and_add_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_dec_64.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_add_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/sync_fetch_and_sub_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_dec_64.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_add_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_xor_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_add_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.14
common/lib/libc/arch/arm/atomic/atomic_or_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_nand_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_or_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_cas_8.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_inc_64.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_inc_64.S: revision 1.11
common/lib/libc/arch/arm/atomic/atomic_cas_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_dec_32.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_or_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_dec_32.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_xor_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_4.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/atomic_or_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_2.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_swap_64.S: revision 1.14
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_1.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_and_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_and_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_and_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.14

Give the thumb atomic ops a chance of working

s/ifdef _ARM_ARCH_6/if defined(_ARM_ARCH_6)/ for consistency. NFCI.

Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.

That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)
This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)
This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.7.16.2 21-Apr-2020  martin Sync with HEAD
 1.7.16.1 13-Apr-2020  martin Mostly merge changes from HEAD upto 20200411
 1.5 28-Jul-2021  skrll Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)

This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)

This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.4 10-Jul-2021  skrll s/ifdef _ARM_ARCH_6/if defined(_ARM_ARCH_6)/ for consistency. NFCI.
 1.3 23-Jun-2014  joerg branches: 1.3.4; 1.3.26; 1.3.28;
Add aliases for the builtins used to implement C11/C++11 atomics.
 1.2 04-Mar-2014  matt branches: 1.2.2; 1.2.4;
Don't export __sync* if _KERNEL || _STANDALONE are defined.
(except if _RUMPKERNEL is defined)
 1.1 08-Nov-2013  matt Add support for the gcc __sync builtins.
Note that these need earmv6 or later to get the ldrex/strex instructions
 1.2.4.2 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.2.4.1 04-Mar-2014  yamt file atomic_or_16.S was added on branch yamt-pagecache on 2014-05-22 11:26:28 +0000
 1.2.2.1 10-Aug-2014  tls Rebase.
 1.3.28.1 11-Aug-2021  martin Pull up following revision(s) (requested by skrll in ticket #1330):

common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.18
common/lib/libc/arch/arm/atomic/atomic_and_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.19
common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_inc_32.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_and_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_add_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_inc_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_cas_up.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_add_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_op_asm.h: revision 1.9
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.8
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_op_asm.h: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_and_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_cas_16.S: revision 1.3
common/lib/libc/arch/arm/atomic/sync_fetch_and_and_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_32.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_xor_32.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_cas_32.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_xor_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_add_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/sync_fetch_and_add_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_dec_64.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_add_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/sync_fetch_and_sub_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_dec_64.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_add_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_xor_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_add_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.14
common/lib/libc/arch/arm/atomic/atomic_or_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_nand_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_or_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_cas_8.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_inc_64.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_inc_64.S: revision 1.11
common/lib/libc/arch/arm/atomic/atomic_cas_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_dec_32.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_or_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_dec_32.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_xor_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_4.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/atomic_or_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_2.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_swap_64.S: revision 1.14
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_1.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_and_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_and_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_and_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.14

Give the thumb atomic ops a chance of working

s/ifdef _ARM_ARCH_6/if defined(_ARM_ARCH_6)/ for consistency. NFCI.

Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.

That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)
This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)
This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.3.26.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.3.26.1 21-Apr-2020  martin Sync with HEAD
 1.3.4.2 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.3.4.1 23-Jun-2014  tls file atomic_or_16.S was added on branch tls-maxphys on 2014-08-19 23:45:12 +0000
 1.10 28-Jul-2021  skrll Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)

This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)

This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.9 10-Jul-2021  skrll s/ifdef _ARM_ARCH_6/if defined(_ARM_ARCH_6)/ for consistency. NFCI.
 1.8 23-Jun-2014  joerg branches: 1.8.24; 1.8.26;
Add aliases for the builtins used to implement C11/C++11 atomics.
 1.7 22-Feb-2014  martin branches: 1.7.2;
Try to hide the C runtime implementation specific __sync_* ops from librump,
to avoid duplicates.
 1.6 08-Nov-2013  matt Add support for the gcc __sync builtins.
Note that these need earmv6 or later to get the ldrex/strex instructions
 1.5 11-Aug-2013  matt Use foo{s} since it reduces the ifdefs for thumb
 1.4 10-Aug-2013  matt Make these under Thumb2
 1.3 31-Aug-2012  matt branches: 1.3.2;
Add dmb/dsb instructions as required by the armv7 arch man.
 1.2 16-Aug-2008  matt branches: 1.2.2; 1.2.4; 1.2.24;
Add assembly versions of atomic ops with ldrex/strex
 1.1 26-Aug-2007  matt branches: 1.1.2;
file atomic_or_32.S was initially added on branch thorpej-atomic.
 1.1.2.2 27-Aug-2007  matt Don't use r4 (since it's caller saved). Use ip (it's available since we
are leaf functions).
 1.1.2.1 26-Aug-2007  matt Add armv6 variants of atomic ops which use ldrex/strex.
Add atomic_swap (all arm architectures) which uses swp and swpb.
Add membar_ops which uses armv6 data {sync, memory} barrier.
XXX let thorpej figure out the Makefile magic.
 1.2.24.1 27-Nov-2012  matt Pull atomic ops from HEAD.
 1.2.4.2 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.2.4.1 30-Oct-2012  yamt sync with head
 1.2.2.2 18-Sep-2008  wrstuden Sync with wrstuden-revivesa-base-2.
 1.2.2.1 16-Aug-2008  wrstuden file atomic_or_32.S was added on branch wrstuden-revivesa on 2008-09-18 04:54:18 +0000
 1.3.2.1 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.7.2.1 10-Aug-2014  tls Rebase.
 1.8.26.1 11-Aug-2021  martin Pull up following revision(s) (requested by skrll in ticket #1330):

common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.18
common/lib/libc/arch/arm/atomic/atomic_and_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.19
common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_inc_32.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_and_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_add_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_inc_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_cas_up.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_add_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_op_asm.h: revision 1.9
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.8
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_op_asm.h: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_and_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_cas_16.S: revision 1.3
common/lib/libc/arch/arm/atomic/sync_fetch_and_and_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_32.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_xor_32.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_cas_32.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_xor_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_add_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/sync_fetch_and_add_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_dec_64.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_add_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/sync_fetch_and_sub_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_dec_64.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_add_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_xor_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_add_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.14
common/lib/libc/arch/arm/atomic/atomic_or_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_nand_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_or_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_cas_8.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_inc_64.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_inc_64.S: revision 1.11
common/lib/libc/arch/arm/atomic/atomic_cas_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_dec_32.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_or_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_dec_32.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_xor_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_4.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/atomic_or_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_2.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_swap_64.S: revision 1.14
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_1.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_and_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_and_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_and_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.14

Give the thumb atomic ops a chance of working

s/ifdef _ARM_ARCH_6/if defined(_ARM_ARCH_6)/ for consistency. NFCI.

Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.

That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)
This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)
This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.8.24.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.8.24.1 21-Apr-2020  martin Sync with HEAD
 1.14 28-Jul-2021  skrll Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)

This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)

This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.13 10-Jul-2021  skrll s/ifdef _ARM_ARCH_6/if defined(_ARM_ARCH_6)/ for consistency. NFCI.
 1.12 28-Jun-2021  skrll Whitespace
 1.11 15-Sep-2019  skrll __sync_{,x}or_and_fetch_8 should return new value... make it so.
 1.10 04-Mar-2014  matt branches: 1.10.18; 1.10.26; 1.10.28;
Don't export __sync* if _KERNEL || _STANDALONE are defined.
(except if _RUMPKERNEL is defined)
 1.9 30-Nov-2013  joerg branches: 1.9.2;
Use explicit form of register pair operations by specifying both.
 1.8 08-Nov-2013  matt Add support for the gcc __sync builtins.
Note that these need earmv6 or later to get the ldrex/strex instructions
 1.7 20-Aug-2013  matt Push two registers to keep stack aligned.
 1.6 11-Aug-2013  matt Use foo{s} since it reduces the ifdefs for thumb
 1.5 10-Aug-2013  matt Make these under Thumb2
 1.4 10-Aug-2013  matt Never modify NLO/NHI (r2,r3) only LO/HI (r0,r1) so subsequent loops will work.
 1.3 10-Aug-2013  matt use push/pop
 1.2 13-Sep-2012  matt branches: 1.2.2; 1.2.4;
Correct copyright/fix comments.
 1.1 11-Sep-2012  matt branches: 1.1.2;
Add 64bit atomic ops for ARMv6+ (using ldrexd/strexd).
 1.1.2.2 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.1.2.1 20-Nov-2012  tls Resync to 2012-11-19 00:00:00 UTC
 1.2.4.2 27-Nov-2012  matt Pull atomic ops from HEAD.
 1.2.4.1 13-Sep-2012  matt file atomic_or_64.S was added on branch matt-nb6-plus on 2012-11-27 23:42:35 +0000
 1.2.2.3 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.2.2.2 30-Oct-2012  yamt sync with head
 1.2.2.1 13-Sep-2012  yamt file atomic_or_64.S was added on branch yamt-pagecache on 2012-10-30 18:46:11 +0000
 1.9.2.2 15-Feb-2014  matt Add 64-bit atomic ops if ARMv6+
 1.9.2.1 30-Nov-2013  matt file atomic_or_64.S was added on branch matt-nb5-mips64 on 2014-02-15 10:27:44 +0000
 1.10.28.3 11-Aug-2021  martin Pull up following revision(s) (requested by skrll in ticket #1330):

common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.18
common/lib/libc/arch/arm/atomic/atomic_and_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.19
common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_inc_32.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_and_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_add_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_inc_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_cas_up.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_add_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_op_asm.h: revision 1.9
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.8
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_op_asm.h: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_and_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_cas_16.S: revision 1.3
common/lib/libc/arch/arm/atomic/sync_fetch_and_and_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_32.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_xor_32.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_cas_32.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_xor_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_add_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/sync_fetch_and_add_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_dec_64.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_add_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/sync_fetch_and_sub_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_dec_64.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_add_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_xor_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_add_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.14
common/lib/libc/arch/arm/atomic/atomic_or_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_nand_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_or_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_cas_8.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_inc_64.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_inc_64.S: revision 1.11
common/lib/libc/arch/arm/atomic/atomic_cas_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_dec_32.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_or_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_dec_32.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_xor_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_4.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/atomic_or_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_2.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_swap_64.S: revision 1.14
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_1.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_and_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_and_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_and_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.14

Give the thumb atomic ops a chance of working

s/ifdef _ARM_ARCH_6/if defined(_ARM_ARCH_6)/ for consistency. NFCI.

Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.

That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)
This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)
This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.10.28.2 06-Jul-2021  martin Pull up following revision(s) (requested by skrll in ticket #1313):

common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/atomic_cas_8.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.3
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.11
common/lib/libc/arch/arm/atomic/atomic_swap_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.12

Whitespace
 1.10.28.1 17-Sep-2019  martin Pull up following revision(s) (requested by skrll in ticket #202):

common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.11

__sync_{,x}or_and_fetch_8 should return new value... make it so.
 1.10.26.2 21-Apr-2020  martin Sync with HEAD
 1.10.26.1 13-Apr-2020  martin Mostly merge changes from HEAD upto 20200411
 1.10.18.1 17-Sep-2019  martin Pull up following revision(s) (requested by skrll in ticket #1375):

common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.11

__sync_{,x}or_and_fetch_8 should return new value... make it so.
 1.5 28-Jul-2021  skrll Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)

This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)

This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.4 10-Jul-2021  skrll s/ifdef _ARM_ARCH_6/if defined(_ARM_ARCH_6)/ for consistency. NFCI.
 1.3 23-Jun-2014  joerg branches: 1.3.4; 1.3.26; 1.3.28;
Add aliases for the builtins used to implement C11/C++11 atomics.
 1.2 04-Mar-2014  matt branches: 1.2.2; 1.2.4;
Don't export __sync* if _KERNEL || _STANDALONE are defined.
(except if _RUMPKERNEL is defined)
 1.1 08-Nov-2013  matt Add support for the gcc __sync builtins.
Note that these need earmv6 or later to get the ldrex/strex instructions
 1.2.4.2 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.2.4.1 04-Mar-2014  yamt file atomic_or_8.S was added on branch yamt-pagecache on 2014-05-22 11:26:28 +0000
 1.2.2.1 10-Aug-2014  tls Rebase.
 1.3.28.1 11-Aug-2021  martin Pull up following revision(s) (requested by skrll in ticket #1330):

common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.18
common/lib/libc/arch/arm/atomic/atomic_and_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.19
common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_inc_32.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_and_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_add_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_inc_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_cas_up.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_add_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_op_asm.h: revision 1.9
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.8
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_op_asm.h: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_and_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_cas_16.S: revision 1.3
common/lib/libc/arch/arm/atomic/sync_fetch_and_and_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_32.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_xor_32.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_cas_32.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_xor_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_add_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/sync_fetch_and_add_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_dec_64.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_add_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/sync_fetch_and_sub_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_dec_64.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_add_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_xor_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_add_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.14
common/lib/libc/arch/arm/atomic/atomic_or_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_nand_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_or_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_cas_8.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_inc_64.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_inc_64.S: revision 1.11
common/lib/libc/arch/arm/atomic/atomic_cas_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_dec_32.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_or_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_dec_32.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_xor_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_4.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/atomic_or_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_2.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_swap_64.S: revision 1.14
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_1.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_and_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_and_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_and_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.14

Give the thumb atomic ops a chance of working

s/ifdef _ARM_ARCH_6/if defined(_ARM_ARCH_6)/ for consistency. NFCI.

Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.

That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)
This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)
This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.3.26.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.3.26.1 21-Apr-2020  martin Sync with HEAD
 1.3.4.2 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.3.4.1 23-Jun-2014  tls file atomic_or_8.S was added on branch tls-maxphys on 2014-08-19 23:45:12 +0000
 1.2 16-Aug-2013  matt branches: 1.2.4; 1.2.8; 1.2.30;
Add a hidden version for libpthread.
 1.1 15-Aug-2013  matt When compiling for Thumb1, the swp instruction is not availble. So this
stub is added to provide __cpu_simple_lock and __cpu_simple_lock_try via
thumb interwork support. It is compiled with -mno-thumb.
 1.2.30.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.2.30.1 21-Apr-2020  martin Sync with HEAD
 1.2.8.2 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.2.8.1 16-Aug-2013  tls file atomic_simplelock.c was added on branch tls-maxphys on 2014-08-19 23:45:12 +0000
 1.2.4.2 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.2.4.1 16-Aug-2013  yamt file atomic_simplelock.c was added on branch yamt-pagecache on 2014-05-22 11:26:28 +0000
 1.5 28-Jul-2021  skrll Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)

This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)

This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.4 10-Jul-2021  skrll s/ifdef _ARM_ARCH_6/if defined(_ARM_ARCH_6)/ for consistency. NFCI.
 1.3 28-Jun-2021  skrll Whitespace
 1.2 04-Mar-2014  matt branches: 1.2.4; 1.2.8; 1.2.30; 1.2.32;
Don't export __sync* if _KERNEL || _STANDALONE are defined.
(except if _RUMPKERNEL is defined)
 1.1 08-Nov-2013  matt Add support for the gcc __sync builtins.
Note that these need earmv6 or later to get the ldrex/strex instructions
 1.2.32.2 11-Aug-2021  martin Pull up following revision(s) (requested by skrll in ticket #1330):

common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.18
common/lib/libc/arch/arm/atomic/atomic_and_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.19
common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_inc_32.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_and_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_add_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_inc_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_cas_up.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_add_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_op_asm.h: revision 1.9
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.8
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_op_asm.h: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_and_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_cas_16.S: revision 1.3
common/lib/libc/arch/arm/atomic/sync_fetch_and_and_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_32.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_xor_32.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_cas_32.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_xor_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_add_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/sync_fetch_and_add_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_dec_64.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_add_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/sync_fetch_and_sub_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_dec_64.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_add_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_xor_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_add_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.14
common/lib/libc/arch/arm/atomic/atomic_or_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_nand_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_or_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_cas_8.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_inc_64.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_inc_64.S: revision 1.11
common/lib/libc/arch/arm/atomic/atomic_cas_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_dec_32.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_or_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_dec_32.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_xor_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_4.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/atomic_or_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_2.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_swap_64.S: revision 1.14
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_1.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_and_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_and_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_and_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.14

Give the thumb atomic ops a chance of working

s/ifdef _ARM_ARCH_6/if defined(_ARM_ARCH_6)/ for consistency. NFCI.

Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.

That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)
This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)
This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.2.32.1 06-Jul-2021  martin Pull up following revision(s) (requested by skrll in ticket #1313):

common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/atomic_cas_8.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.3
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.11
common/lib/libc/arch/arm/atomic/atomic_swap_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.12

Whitespace
 1.2.30.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.2.30.1 21-Apr-2020  martin Sync with HEAD
 1.2.8.2 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.2.8.1 04-Mar-2014  tls file atomic_sub_64.S was added on branch tls-maxphys on 2014-08-19 23:45:12 +0000
 1.2.4.2 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.2.4.1 04-Mar-2014  yamt file atomic_sub_64.S was added on branch yamt-pagecache on 2014-05-22 11:26:28 +0000
 1.19 28-Jul-2021  skrll Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)

This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)

This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.18 10-Jul-2021  skrll s/ifdef _ARM_ARCH_6/if defined(_ARM_ARCH_6)/ for consistency. NFCI.
 1.17 26-Apr-2021  skrll Add the appropriate memory barrier before the lock is cleared in
__sync_lock_release_{1,2,4,8}. That is, all reads and write for in inner
shareability domain before the lock clear store.
 1.16 24-Apr-2021  skrll Trailing whitespace
 1.15 24-Apr-2021  skrll Fix __sync_lock_release_4 to actually zeroise the whole 4bytes/32bits.
 1.14 17-May-2015  justin branches: 1.14.8; 1.14.16; 1.14.18;
Do not build arm toolchain symbols in the rump kernel

These symbols will be provided at link time and will be
duplicate symbols in rump kernel and libc if defined.

Many have been fixed previously, but these were missed
as did not have a test.
 1.13 17-Apr-2015  skrll Use the right register in previous. Spotted by matt@
 1.12 17-Apr-2015  skrll ARM ARM D7.3.2 - ensure all previous accesses are observed before
the lock is cleared
 1.11 28-Jun-2014  joerg Add aliases for the C11/C++11 spelling of the CAS primitives.
 1.10 04-Mar-2014  matt branches: 1.10.2;
Don't export __sync* if _KERNEL || _STANDALONE are defined.
(except if _RUMPKERNEL is defined)
 1.9 30-Jan-2014  matt switch to unified syntax
 1.8 08-Nov-2013  matt Add support for the gcc __sync builtins.
Note that these need earmv6 or later to get the ldrex/strex instructions
 1.7 11-Aug-2013  matt Use foo{s} since it reduces the ifdefs for thumb
 1.6 10-Aug-2013  matt Make these under Thumb2
 1.5 28-Jan-2013  matt Change movsne to movnes for clang.
 1.4 31-Aug-2012  matt branches: 1.4.2;
Add dmb/dsb instructions as required by the armv7 arch man.
 1.3 16-Aug-2012  matt Actually use the assembly version of the atomic function if compiling
for armv6 or armv7 cpus. Use atomic_cas_ptr instead of _lock_cas so
we pick up the assembly version when it's used.
 1.2 16-Aug-2008  matt branches: 1.2.2; 1.2.4; 1.2.12; 1.2.24;
Add assembly versions of atomic ops with ldrex/strex
 1.1 26-Aug-2007  matt branches: 1.1.2;
file atomic_swap.S was initially added on branch thorpej-atomic.
 1.1.2.1 26-Aug-2007  matt Add armv6 variants of atomic ops which use ldrex/strex.
Add atomic_swap (all arm architectures) which uses swp and swpb.
Add membar_ops which uses armv6 data {sync, memory} barrier.
XXX let thorpej figure out the Makefile magic.
 1.2.24.1 27-Nov-2012  matt Pull atomic ops from HEAD.
 1.2.12.1 24-Mar-2014  matt Use ldrex/strex instead of swp when possible.
 1.2.4.2 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.2.4.1 30-Oct-2012  yamt sync with head
 1.2.2.2 18-Sep-2008  wrstuden Sync with wrstuden-revivesa-base-2.
 1.2.2.1 16-Aug-2008  wrstuden file atomic_swap.S was added on branch wrstuden-revivesa on 2008-09-18 04:54:18 +0000
 1.4.2.2 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.4.2.1 25-Feb-2013  tls resync with head
 1.10.2.1 10-Aug-2014  tls Rebase.
 1.14.18.3 11-Aug-2021  martin Pull up following revision(s) (requested by skrll in ticket #1330):

common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.18
common/lib/libc/arch/arm/atomic/atomic_and_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.19
common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_inc_32.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_and_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_add_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_inc_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_cas_up.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_add_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_op_asm.h: revision 1.9
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.8
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_op_asm.h: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_and_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_cas_16.S: revision 1.3
common/lib/libc/arch/arm/atomic/sync_fetch_and_and_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_32.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_xor_32.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_cas_32.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_xor_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_add_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/sync_fetch_and_add_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_dec_64.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_add_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/sync_fetch_and_sub_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_dec_64.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_add_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_xor_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_add_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.14
common/lib/libc/arch/arm/atomic/atomic_or_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_nand_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_or_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_cas_8.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_inc_64.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_inc_64.S: revision 1.11
common/lib/libc/arch/arm/atomic/atomic_cas_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_dec_32.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_or_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_dec_32.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_xor_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_4.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/atomic_or_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_2.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_swap_64.S: revision 1.14
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_1.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_and_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_and_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_and_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.14

Give the thumb atomic ops a chance of working

s/ifdef _ARM_ARCH_6/if defined(_ARM_ARCH_6)/ for consistency. NFCI.

Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.

That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)
This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)
This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.14.18.2 30-Apr-2021  martin Pull up following revision(s) (requested by skrll in ticket #1261):

sys/arch/arm/include/lock.h: revision 1.38
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.17

Add the appropriate memory barrier before the lock is cleared in
__sync_lock_release_{1,2,4,8}. That is, all reads and write for the
inner shareability domain before the lock clear store.

Improve the membar_ops barriers - no need to use dsb and wait for
completion. Also, we only to act on the inner shareability domain.

Fix the barrier confusion. From Riastradh - thanks!.
 1.14.18.1 26-Apr-2021  martin Pull up following revision(s) (requested by skrll in ticket #1254):

sys/arch/arm/include/lock.h: revision 1.37
common/lib/libc/arch/arm/atomic/atomic_swap_64.S: revision 1.11
common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.15
common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.16
sys/arch/arm/include/lock.h: revision 1.36

Fix ARMv8 instructions

Fix __sync_lock_release_4 to actually zeroise the whole 4bytes/32bits.

Trailing whitespace

Change #ifdef FOO to #if defined(FOO). NFCI.
 1.14.16.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.14.16.1 21-Apr-2020  martin Sync with HEAD
 1.14.8.1 26-Apr-2021  martin Pull up following revision(s) (requested by skrll in ticket #1672):

sys/arch/arm/include/lock.h: revision 1.37
common/lib/libc/arch/arm/atomic/atomic_swap_64.S: revision 1.11
common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.15
common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.16
sys/arch/arm/include/lock.h: revision 1.36

Fix ARMv8 instructions

Fix __sync_lock_release_4 to actually zeroise the whole 4bytes/32bits.

Trailing whitespace

Change #ifdef FOO to #if defined(FOO). NFCI.
 1.7 28-Jul-2021  skrll Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)

This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)

This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.6 10-Jul-2021  skrll s/ifdef _ARM_ARCH_6/if defined(_ARM_ARCH_6)/ for consistency. NFCI.
 1.5 26-Apr-2021  skrll Add the appropriate memory barrier before the lock is cleared in
__sync_lock_release_{1,2,4,8}. That is, all reads and write for in inner
shareability domain before the lock clear store.
 1.4 17-May-2015  justin branches: 1.4.16; 1.4.18;
Do not build arm toolchain symbols in the rump kernel

These symbols will be provided at link time and will be
duplicate symbols in rump kernel and libc if defined.

Many have been fixed previously, but these were missed
as did not have a test.
 1.3 28-Jun-2014  joerg branches: 1.3.4;
Add aliases for the C11/C++11 spelling of the CAS primitives.
 1.2 04-Mar-2014  matt branches: 1.2.2; 1.2.4;
Don't export __sync* if _KERNEL || _STANDALONE are defined.
(except if _RUMPKERNEL is defined)
 1.1 08-Nov-2013  matt Add support for the gcc __sync builtins.
Note that these need earmv6 or later to get the ldrex/strex instructions
 1.2.4.2 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.2.4.1 04-Mar-2014  yamt file atomic_swap_16.S was added on branch yamt-pagecache on 2014-05-22 11:26:28 +0000
 1.2.2.1 10-Aug-2014  tls Rebase.
 1.3.4.2 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.3.4.1 28-Jun-2014  tls file atomic_swap_16.S was added on branch tls-maxphys on 2014-08-19 23:45:12 +0000
 1.4.18.2 11-Aug-2021  martin Pull up following revision(s) (requested by skrll in ticket #1330):

common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.18
common/lib/libc/arch/arm/atomic/atomic_and_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.19
common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_inc_32.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_and_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_add_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_inc_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_cas_up.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_add_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_op_asm.h: revision 1.9
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.8
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_op_asm.h: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_and_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_cas_16.S: revision 1.3
common/lib/libc/arch/arm/atomic/sync_fetch_and_and_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_32.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_xor_32.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_cas_32.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_xor_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_add_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/sync_fetch_and_add_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_dec_64.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_add_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/sync_fetch_and_sub_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_dec_64.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_add_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_xor_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_add_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.14
common/lib/libc/arch/arm/atomic/atomic_or_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_nand_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_or_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_cas_8.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_inc_64.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_inc_64.S: revision 1.11
common/lib/libc/arch/arm/atomic/atomic_cas_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_dec_32.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_or_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_dec_32.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_xor_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_4.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/atomic_or_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_2.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_swap_64.S: revision 1.14
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_1.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_and_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_and_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_and_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.14

Give the thumb atomic ops a chance of working

s/ifdef _ARM_ARCH_6/if defined(_ARM_ARCH_6)/ for consistency. NFCI.

Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.

That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)
This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)
This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.4.18.1 30-Apr-2021  martin Pull up following revision(s) (requested by skrll in ticket #1261):

sys/arch/arm/include/lock.h: revision 1.38
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.17

Add the appropriate memory barrier before the lock is cleared in
__sync_lock_release_{1,2,4,8}. That is, all reads and write for the
inner shareability domain before the lock clear store.

Improve the membar_ops barriers - no need to use dsb and wait for
completion. Also, we only to act on the inner shareability domain.

Fix the barrier confusion. From Riastradh - thanks!.
 1.4.16.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.4.16.1 21-Apr-2020  martin Sync with HEAD
 1.15 01-Aug-2021  andvar s/overwriten/overwritten/ in comments.
 1.14 28-Jul-2021  skrll Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)

This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)

This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.13 28-Jun-2021  skrll Whitespace
 1.12 26-Apr-2021  skrll Add the appropriate memory barrier before the lock is cleared in
__sync_lock_release_{1,2,4,8}. That is, all reads and write for in inner
shareability domain before the lock clear store.
 1.11 24-Apr-2021  skrll Trailing whitespace
 1.10 17-May-2015  justin branches: 1.10.8; 1.10.16; 1.10.18;
Do not build arm toolchain symbols in the rump kernel

These symbols will be provided at link time and will be
duplicate symbols in rump kernel and libc if defined.

Many have been fixed previously, but these were missed
as did not have a test.
 1.9 28-Jun-2014  joerg Add aliases for the C11/C++11 spelling of the CAS primitives.
 1.8 04-Mar-2014  matt branches: 1.8.2;
Don't export __sync* if _KERNEL || _STANDALONE are defined.
(except if _RUMPKERNEL is defined)
 1.7 30-Nov-2013  joerg Use explicit form of register pair operations by specifying both.
 1.6 08-Nov-2013  matt Add support for the gcc __sync builtins.
Note that these need earmv6 or later to get the ldrex/strex instructions
 1.5 20-Aug-2013  matt Push two registers to keep stack aligned.
 1.4 10-Aug-2013  matt Make these under Thumb2
 1.3 10-Aug-2013  matt cmpne -> cmp
Use push/pop
 1.2 13-Sep-2012  matt branches: 1.2.2; 1.2.4;
Correct copyright/fix comments.
 1.1 11-Sep-2012  matt branches: 1.1.2;
Add 64bit atomic ops for ARMv6+ (using ldrexd/strexd).
 1.1.2.2 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.1.2.1 20-Nov-2012  tls Resync to 2012-11-19 00:00:00 UTC
 1.2.4.2 27-Nov-2012  matt Pull atomic ops from HEAD.
 1.2.4.1 13-Sep-2012  matt file atomic_swap_64.S was added on branch matt-nb6-plus on 2012-11-27 23:42:35 +0000
 1.2.2.3 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.2.2.2 30-Oct-2012  yamt sync with head
 1.2.2.1 13-Sep-2012  yamt file atomic_swap_64.S was added on branch yamt-pagecache on 2012-10-30 18:46:12 +0000
 1.8.2.1 10-Aug-2014  tls Rebase.
 1.10.18.4 11-Aug-2021  martin Pull up following revision(s) (requested by skrll in ticket #1330):

common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.18
common/lib/libc/arch/arm/atomic/atomic_and_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.19
common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_inc_32.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_and_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_add_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_inc_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_cas_up.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_add_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_op_asm.h: revision 1.9
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.8
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_op_asm.h: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_and_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_cas_16.S: revision 1.3
common/lib/libc/arch/arm/atomic/sync_fetch_and_and_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_32.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_xor_32.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_cas_32.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_xor_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_add_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/sync_fetch_and_add_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_dec_64.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_add_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/sync_fetch_and_sub_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_dec_64.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_add_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_xor_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_add_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.14
common/lib/libc/arch/arm/atomic/atomic_or_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_nand_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_or_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_cas_8.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_inc_64.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_inc_64.S: revision 1.11
common/lib/libc/arch/arm/atomic/atomic_cas_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_dec_32.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_or_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_dec_32.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_xor_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_4.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/atomic_or_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_2.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_swap_64.S: revision 1.14
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_1.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_and_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_and_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_and_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.14

Give the thumb atomic ops a chance of working

s/ifdef _ARM_ARCH_6/if defined(_ARM_ARCH_6)/ for consistency. NFCI.

Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.

That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)
This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)
This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.10.18.3 06-Jul-2021  martin Pull up following revision(s) (requested by skrll in ticket #1313):

common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/atomic_cas_8.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.3
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.11
common/lib/libc/arch/arm/atomic/atomic_swap_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.12

Whitespace
 1.10.18.2 30-Apr-2021  martin Pull up following revision(s) (requested by skrll in ticket #1261):

sys/arch/arm/include/lock.h: revision 1.38
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.17

Add the appropriate memory barrier before the lock is cleared in
__sync_lock_release_{1,2,4,8}. That is, all reads and write for the
inner shareability domain before the lock clear store.

Improve the membar_ops barriers - no need to use dsb and wait for
completion. Also, we only to act on the inner shareability domain.

Fix the barrier confusion. From Riastradh - thanks!.
 1.10.18.1 26-Apr-2021  martin Pull up following revision(s) (requested by skrll in ticket #1254):

sys/arch/arm/include/lock.h: revision 1.37
common/lib/libc/arch/arm/atomic/atomic_swap_64.S: revision 1.11
common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.15
common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.16
sys/arch/arm/include/lock.h: revision 1.36

Fix ARMv8 instructions

Fix __sync_lock_release_4 to actually zeroise the whole 4bytes/32bits.

Trailing whitespace

Change #ifdef FOO to #if defined(FOO). NFCI.
 1.10.16.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.10.16.1 21-Apr-2020  martin Sync with HEAD
 1.10.8.1 26-Apr-2021  martin Pull up following revision(s) (requested by skrll in ticket #1672):

sys/arch/arm/include/lock.h: revision 1.37
common/lib/libc/arch/arm/atomic/atomic_swap_64.S: revision 1.11
common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.15
common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.16
sys/arch/arm/include/lock.h: revision 1.36

Fix ARMv8 instructions

Fix __sync_lock_release_4 to actually zeroise the whole 4bytes/32bits.

Trailing whitespace

Change #ifdef FOO to #if defined(FOO). NFCI.
 1.5 28-Jul-2021  skrll Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)

This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)

This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.4 10-Jul-2021  skrll s/ifdef _ARM_ARCH_6/if defined(_ARM_ARCH_6)/ for consistency. NFCI.
 1.3 23-Jun-2014  joerg branches: 1.3.4; 1.3.26; 1.3.28;
Add aliases for the builtins used to implement C11/C++11 atomics.
 1.2 04-Mar-2014  matt branches: 1.2.2; 1.2.4;
Don't export __sync* if _KERNEL || _STANDALONE are defined.
(except if _RUMPKERNEL is defined)
 1.1 08-Nov-2013  matt Add support for the gcc __sync builtins.
Note that these need earmv6 or later to get the ldrex/strex instructions
 1.2.4.2 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.2.4.1 04-Mar-2014  yamt file atomic_xor_16.S was added on branch yamt-pagecache on 2014-05-22 11:26:28 +0000
 1.2.2.1 10-Aug-2014  tls Rebase.
 1.3.28.1 11-Aug-2021  martin Pull up following revision(s) (requested by skrll in ticket #1330):

common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.18
common/lib/libc/arch/arm/atomic/atomic_and_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.19
common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_inc_32.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_and_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_add_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_inc_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_cas_up.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_add_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_op_asm.h: revision 1.9
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.8
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_op_asm.h: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_and_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_cas_16.S: revision 1.3
common/lib/libc/arch/arm/atomic/sync_fetch_and_and_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_32.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_xor_32.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_cas_32.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_xor_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_add_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/sync_fetch_and_add_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_dec_64.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_add_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/sync_fetch_and_sub_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_dec_64.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_add_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_xor_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_add_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.14
common/lib/libc/arch/arm/atomic/atomic_or_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_nand_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_or_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_cas_8.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_inc_64.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_inc_64.S: revision 1.11
common/lib/libc/arch/arm/atomic/atomic_cas_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_dec_32.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_or_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_dec_32.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_xor_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_4.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/atomic_or_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_2.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_swap_64.S: revision 1.14
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_1.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_and_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_and_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_and_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.14

Give the thumb atomic ops a chance of working

s/ifdef _ARM_ARCH_6/if defined(_ARM_ARCH_6)/ for consistency. NFCI.

Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.

That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)
This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)
This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.3.26.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.3.26.1 21-Apr-2020  martin Sync with HEAD
 1.3.4.2 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.3.4.1 23-Jun-2014  tls file atomic_xor_16.S was added on branch tls-maxphys on 2014-08-19 23:45:12 +0000
 1.5 28-Jul-2021  skrll Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)

This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)

This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.4 10-Jul-2021  skrll s/ifdef _ARM_ARCH_6/if defined(_ARM_ARCH_6)/ for consistency. NFCI.
 1.3 23-Jun-2014  joerg branches: 1.3.4; 1.3.26; 1.3.28;
Add aliases for the builtins used to implement C11/C++11 atomics.
 1.2 04-Mar-2014  matt branches: 1.2.2; 1.2.4;
Don't export __sync* if _KERNEL || _STANDALONE are defined.
(except if _RUMPKERNEL is defined)
 1.1 08-Nov-2013  matt Add support for the gcc __sync builtins.
Note that these need earmv6 or later to get the ldrex/strex instructions
 1.2.4.2 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.2.4.1 04-Mar-2014  yamt file atomic_xor_32.S was added on branch yamt-pagecache on 2014-05-22 11:26:28 +0000
 1.2.2.1 10-Aug-2014  tls Rebase.
 1.3.28.1 11-Aug-2021  martin Pull up following revision(s) (requested by skrll in ticket #1330):

common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.18
common/lib/libc/arch/arm/atomic/atomic_and_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.19
common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_inc_32.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_and_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_add_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_inc_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_cas_up.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_add_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_op_asm.h: revision 1.9
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.8
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_op_asm.h: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_and_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_cas_16.S: revision 1.3
common/lib/libc/arch/arm/atomic/sync_fetch_and_and_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_32.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_xor_32.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_cas_32.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_xor_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_add_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/sync_fetch_and_add_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_dec_64.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_add_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/sync_fetch_and_sub_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_dec_64.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_add_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_xor_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_add_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.14
common/lib/libc/arch/arm/atomic/atomic_or_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_nand_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_or_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_cas_8.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_inc_64.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_inc_64.S: revision 1.11
common/lib/libc/arch/arm/atomic/atomic_cas_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_dec_32.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_or_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_dec_32.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_xor_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_4.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/atomic_or_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_2.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_swap_64.S: revision 1.14
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_1.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_and_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_and_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_and_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.14

Give the thumb atomic ops a chance of working

s/ifdef _ARM_ARCH_6/if defined(_ARM_ARCH_6)/ for consistency. NFCI.

Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.

That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)
This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)
This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.3.26.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.3.26.1 21-Apr-2020  martin Sync with HEAD
 1.3.4.2 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.3.4.1 23-Jun-2014  tls file atomic_xor_32.S was added on branch tls-maxphys on 2014-08-19 23:45:12 +0000
 1.7 28-Jul-2021  skrll Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)

This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)

This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.6 10-Jul-2021  skrll s/ifdef _ARM_ARCH_6/if defined(_ARM_ARCH_6)/ for consistency. NFCI.
 1.5 28-Jun-2021  skrll Whitespace
 1.4 15-Sep-2019  skrll __sync_{,x}or_and_fetch_8 should return new value... make it so.
 1.3 04-Mar-2014  matt branches: 1.3.4; 1.3.8; 1.3.22; 1.3.30; 1.3.32;
Don't export __sync* if _KERNEL || _STANDALONE are defined.
(except if _RUMPKERNEL is defined)
 1.2 30-Nov-2013  joerg Use explicit form of register pair operations by specifying both.
 1.1 08-Nov-2013  matt Add support for the gcc __sync builtins.
Note that these need earmv6 or later to get the ldrex/strex instructions
 1.3.32.3 11-Aug-2021  martin Pull up following revision(s) (requested by skrll in ticket #1330):

common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.18
common/lib/libc/arch/arm/atomic/atomic_and_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.19
common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_inc_32.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_and_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_add_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_inc_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_cas_up.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_add_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_op_asm.h: revision 1.9
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.8
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_op_asm.h: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_and_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_cas_16.S: revision 1.3
common/lib/libc/arch/arm/atomic/sync_fetch_and_and_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_32.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_xor_32.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_cas_32.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_xor_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_add_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/sync_fetch_and_add_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_dec_64.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_add_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/sync_fetch_and_sub_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_dec_64.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_add_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_xor_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_add_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.14
common/lib/libc/arch/arm/atomic/atomic_or_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_nand_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_or_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_cas_8.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_inc_64.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_inc_64.S: revision 1.11
common/lib/libc/arch/arm/atomic/atomic_cas_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_dec_32.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_or_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_dec_32.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_xor_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_4.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/atomic_or_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_2.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_swap_64.S: revision 1.14
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_1.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_and_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_and_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_and_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.14

Give the thumb atomic ops a chance of working

s/ifdef _ARM_ARCH_6/if defined(_ARM_ARCH_6)/ for consistency. NFCI.

Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.

That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)
This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)
This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.3.32.2 06-Jul-2021  martin Pull up following revision(s) (requested by skrll in ticket #1313):

common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/atomic_cas_8.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.3
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.11
common/lib/libc/arch/arm/atomic/atomic_swap_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.12

Whitespace
 1.3.32.1 17-Sep-2019  martin Pull up following revision(s) (requested by skrll in ticket #202):

common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.11

__sync_{,x}or_and_fetch_8 should return new value... make it so.
 1.3.30.2 21-Apr-2020  martin Sync with HEAD
 1.3.30.1 13-Apr-2020  martin Mostly merge changes from HEAD upto 20200411
 1.3.22.1 17-Sep-2019  martin Pull up following revision(s) (requested by skrll in ticket #1375):

common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.11

__sync_{,x}or_and_fetch_8 should return new value... make it so.
 1.3.8.2 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.3.8.1 04-Mar-2014  tls file atomic_xor_64.S was added on branch tls-maxphys on 2014-08-19 23:45:12 +0000
 1.3.4.2 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.3.4.1 04-Mar-2014  yamt file atomic_xor_64.S was added on branch yamt-pagecache on 2014-05-22 11:26:28 +0000
 1.5 28-Jul-2021  skrll Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)

This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)

This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.4 10-Jul-2021  skrll s/ifdef _ARM_ARCH_6/if defined(_ARM_ARCH_6)/ for consistency. NFCI.
 1.3 23-Jun-2014  joerg branches: 1.3.4; 1.3.26; 1.3.28;
Add aliases for the builtins used to implement C11/C++11 atomics.
 1.2 04-Mar-2014  matt branches: 1.2.2; 1.2.4;
Don't export __sync* if _KERNEL || _STANDALONE are defined.
(except if _RUMPKERNEL is defined)
 1.1 08-Nov-2013  matt Add support for the gcc __sync builtins.
Note that these need earmv6 or later to get the ldrex/strex instructions
 1.2.4.2 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.2.4.1 04-Mar-2014  yamt file atomic_xor_8.S was added on branch yamt-pagecache on 2014-05-22 11:26:28 +0000
 1.2.2.1 10-Aug-2014  tls Rebase.
 1.3.28.1 11-Aug-2021  martin Pull up following revision(s) (requested by skrll in ticket #1330):

common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.18
common/lib/libc/arch/arm/atomic/atomic_and_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.19
common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_inc_32.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_and_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_add_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_inc_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_cas_up.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_add_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_op_asm.h: revision 1.9
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.8
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_op_asm.h: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_and_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_cas_16.S: revision 1.3
common/lib/libc/arch/arm/atomic/sync_fetch_and_and_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_32.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_xor_32.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_cas_32.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_xor_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_add_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/sync_fetch_and_add_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_dec_64.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_add_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/sync_fetch_and_sub_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_dec_64.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_add_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_xor_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_add_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.14
common/lib/libc/arch/arm/atomic/atomic_or_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_nand_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_or_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_cas_8.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_inc_64.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_inc_64.S: revision 1.11
common/lib/libc/arch/arm/atomic/atomic_cas_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_dec_32.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_or_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_dec_32.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_xor_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_4.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/atomic_or_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_2.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_swap_64.S: revision 1.14
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_1.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_and_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_and_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_and_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.14

Give the thumb atomic ops a chance of working

s/ifdef _ARM_ARCH_6/if defined(_ARM_ARCH_6)/ for consistency. NFCI.

Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.

That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)
This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)
This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.3.26.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.3.26.1 21-Apr-2020  martin Sync with HEAD
 1.3.4.2 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.3.4.1 23-Jun-2014  tls file atomic_xor_8.S was added on branch tls-maxphys on 2014-08-19 23:45:12 +0000
 1.10 09-Apr-2022  riastradh Introduce membar_acquire/release. Deprecate membar_enter/exit.

The names membar_enter/exit were unclear, and the documentation of
membar_enter has disagreed with the implementations on sparc,
powerpc, and even x86(!) for the entire time it has been in NetBSD.

The terms `acquire' and `release' are ubiquitous in the literature
today, and have been adopted in the C and C++ standards to mean
load-before-load/store and load/store-before-store, respectively,
which are exactly the orderings required by acquiring and releasing a
mutex, as well as other useful applications like decrementing a
reference count and then freeing the underlying object if it went to
zero.

Originally I proposed changing one word in the documentation for
membar_enter to make it load-before-load/store instead of
store-before-load/store, i.e., to make it an acquire barrier. I
proposed this on the grounds that

(a) all implementations guarantee load-before-load/store,
(b) some implementations fail to guarantee store-before-load/store,
and
(c) all uses in-tree assume load-before-load/store.

I verified parts (a) and (b) (except, for (a), powerpc didn't even
guarantee load-before-load/store -- isync isn't necessarily enough;
need lwsync in general -- but it _almost_ did, and it certainly didn't
guarantee store-before-load/store).

Part (c) might not be correct, however: under the mistaken assumption
that atomic-r/m/w then membar-w/rw is equivalent to atomic-r/m/w then
membar-r/rw, I only audited the cases of membar_enter that _aren't_
immediately after an atomic-r/m/w. All of those cases assume
load-before-load/store. But my assumption was wrong -- there are
cases of atomic-r/m/w then membar-w/rw that would be broken by
changing to atomic-r/m/w then membar-r/rw:

https://mail-index.netbsd.org/tech-kern/2022/03/29/msg028044.html

Furthermore, the name membar_enter has been adopted in other places
like OpenBSD where it actually does follow the documentation and
guarantee store-before-load/store, even if that order is not useful.
So the name membar_enter currently lives in a bad place where it
means either of two things -- r/rw or w/rw.

With this change, we deprecate membar_enter/exit, introduce
membar_acquire/release as better names for the useful pair (r/rw and
rw/w), and make sure the implementation of membar_enter guarantees
both what was documented _and_ what was implemented, making it an
alias for membar_sync.

While here, rework all of the membar_* definitions and aliases. The
new logic follows a rule to make it easier to audit:

membar_X is defined as an alias for membar_Y iff membar_X is
guaranteed by membar_Y.

The `no stronger than' relation is (the transitive closure of):

- membar_consumer (r/r) is guaranteed by membar_acquire (r/rw)
- membar_producer (w/w) is guaranteed by membar_release (rw/w)
- membar_acquire (r/rw) is guaranteed by membar_sync (rw/rw)
- membar_release (rw/w) is guaranteed by membar_sync (rw/rw)

And, for the deprecated membars:

- membar_enter (whether r/rw, w/rw, or rw/rw) is guaranteed by
membar_sync (rw/rw)
- membar_exit (rw/w) is guaranteed by membar_release (rw/w)

(membar_exit is identical to membar_release, but the name is
deprecated.)

Finally, while here, annotate some of the instructions with their
semantics. For powerpc, leave an essay with citations on the
unfortunate but -- as far as I can tell -- necessary decision to use
lwsync, not isync, for membar_acquire and membar_consumer.

Also add membar(3) and atomic(3) man page links.
 1.9 28-Jul-2021  skrll Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)

This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)

This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.8 10-Jul-2021  skrll s/ifdef _ARM_ARCH_6/if defined(_ARM_ARCH_6)/ for consistency. NFCI.
 1.7 27-Apr-2021  skrll Improve the membar_ops barriers - no need to use dsb and wait for
completion. Also, we only to act on the inner shareability domain.
 1.6 28-Mar-2014  skrll branches: 1.6.26; 1.6.28;
Ensure SBZ register is zero
 1.5 04-Mar-2014  matt Don't export __sync* if _KERNEL || _STANDALONE are defined.
(except if _RUMPKERNEL is defined)
 1.4 08-Nov-2013  matt Add support for the gcc __sync builtins.
Note that these need earmv6 or later to get the ldrex/strex instructions
 1.3 16-Aug-2012  matt branches: 1.3.2;
Actually use the assembly version of the atomic function if compiling
for armv6 or armv7 cpus. Use atomic_cas_ptr instead of _lock_cas so
we pick up the assembly version when it's used.
 1.2 16-Aug-2008  matt branches: 1.2.2; 1.2.4; 1.2.24;
Add assembly versions of atomic ops with ldrex/strex
 1.1 26-Aug-2007  matt branches: 1.1.2;
file membar_ops.S was initially added on branch thorpej-atomic.
 1.1.2.1 26-Aug-2007  matt Add armv6 variants of atomic ops which use ldrex/strex.
Add atomic_swap (all arm architectures) which uses swp and swpb.
Add membar_ops which uses armv6 data {sync, memory} barrier.
XXX let thorpej figure out the Makefile magic.
 1.2.24.1 27-Nov-2012  matt Pull atomic ops from HEAD.
 1.2.4.2 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.2.4.1 30-Oct-2012  yamt sync with head
 1.2.2.2 18-Sep-2008  wrstuden Sync with wrstuden-revivesa-base-2.
 1.2.2.1 16-Aug-2008  wrstuden file membar_ops.S was added on branch wrstuden-revivesa on 2008-09-18 04:54:18 +0000
 1.3.2.1 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.6.28.2 11-Aug-2021  martin Pull up following revision(s) (requested by skrll in ticket #1330):

common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.18
common/lib/libc/arch/arm/atomic/atomic_and_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.19
common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_inc_32.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_and_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_add_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_inc_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_cas_up.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_add_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_op_asm.h: revision 1.9
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.8
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_op_asm.h: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_and_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_cas_16.S: revision 1.3
common/lib/libc/arch/arm/atomic/sync_fetch_and_and_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_32.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_xor_32.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_cas_32.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_xor_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_add_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/sync_fetch_and_add_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_dec_64.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_add_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/sync_fetch_and_sub_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_dec_64.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_add_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_xor_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_add_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.14
common/lib/libc/arch/arm/atomic/atomic_or_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_nand_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_or_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_cas_8.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_inc_64.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_inc_64.S: revision 1.11
common/lib/libc/arch/arm/atomic/atomic_cas_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_dec_32.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_or_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_dec_32.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_xor_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_4.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/atomic_or_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_2.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_swap_64.S: revision 1.14
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_1.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_and_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_and_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_and_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.14

Give the thumb atomic ops a chance of working

s/ifdef _ARM_ARCH_6/if defined(_ARM_ARCH_6)/ for consistency. NFCI.

Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.

That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)
This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)
This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.6.28.1 30-Apr-2021  martin Pull up following revision(s) (requested by skrll in ticket #1261):

sys/arch/arm/include/lock.h: revision 1.38
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.17

Add the appropriate memory barrier before the lock is cleared in
__sync_lock_release_{1,2,4,8}. That is, all reads and write for the
inner shareability domain before the lock clear store.

Improve the membar_ops barriers - no need to use dsb and wait for
completion. Also, we only to act on the inner shareability domain.

Fix the barrier confusion. From Riastradh - thanks!.
 1.6.26.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.6.26.1 21-Apr-2020  martin Sync with HEAD
 1.4 28-Jul-2021  skrll Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)

This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)

This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.3 17-May-2015  justin branches: 1.3.16; 1.3.18;
Move arm sync_* changes to Makefile.inc
 1.2 17-May-2015  justin Do not build arm toolchain symbols in the rump kernel

These symbols will be provided at link time and will be
duplicate symbols in rump kernel and libc if defined.

Many have been fixed previously, but these were missed
as did not have a test.
 1.1 08-Nov-2013  matt branches: 1.1.4; 1.1.8;
Add support for the gcc __sync builtins.
Note that these need earmv6 or later to get the ldrex/strex instructions
 1.1.8.2 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.1.8.1 08-Nov-2013  tls file sync_bool_compare_and_swap_1.S was added on branch tls-maxphys on 2014-08-19 23:45:12 +0000
 1.1.4.2 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.1.4.1 08-Nov-2013  yamt file sync_bool_compare_and_swap_1.S was added on branch yamt-pagecache on 2014-05-22 11:26:28 +0000
 1.3.18.1 11-Aug-2021  martin Pull up following revision(s) (requested by skrll in ticket #1330):

common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.18
common/lib/libc/arch/arm/atomic/atomic_and_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.19
common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_inc_32.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_and_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_add_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_inc_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_cas_up.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_add_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_op_asm.h: revision 1.9
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.8
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_op_asm.h: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_and_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_cas_16.S: revision 1.3
common/lib/libc/arch/arm/atomic/sync_fetch_and_and_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_32.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_xor_32.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_cas_32.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_xor_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_add_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/sync_fetch_and_add_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_dec_64.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_add_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/sync_fetch_and_sub_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_dec_64.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_add_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_xor_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_add_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.14
common/lib/libc/arch/arm/atomic/atomic_or_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_nand_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_or_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_cas_8.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_inc_64.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_inc_64.S: revision 1.11
common/lib/libc/arch/arm/atomic/atomic_cas_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_dec_32.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_or_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_dec_32.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_xor_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_4.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/atomic_or_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_2.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_swap_64.S: revision 1.14
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_1.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_and_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_and_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_and_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.14

Give the thumb atomic ops a chance of working

s/ifdef _ARM_ARCH_6/if defined(_ARM_ARCH_6)/ for consistency. NFCI.

Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.

That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)
This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)
This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.3.16.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.3.16.1 21-Apr-2020  martin Sync with HEAD
 1.4 28-Jul-2021  skrll Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)

This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)

This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.3 17-May-2015  justin branches: 1.3.16; 1.3.18;
Move arm sync_* changes to Makefile.inc
 1.2 17-May-2015  justin Do not build arm toolchain symbols in the rump kernel

These symbols will be provided at link time and will be
duplicate symbols in rump kernel and libc if defined.

Many have been fixed previously, but these were missed
as did not have a test.
 1.1 08-Nov-2013  matt branches: 1.1.4; 1.1.8;
Add support for the gcc __sync builtins.
Note that these need earmv6 or later to get the ldrex/strex instructions
 1.1.8.2 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.1.8.1 08-Nov-2013  tls file sync_bool_compare_and_swap_2.S was added on branch tls-maxphys on 2014-08-19 23:45:12 +0000
 1.1.4.2 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.1.4.1 08-Nov-2013  yamt file sync_bool_compare_and_swap_2.S was added on branch yamt-pagecache on 2014-05-22 11:26:28 +0000
 1.3.18.1 11-Aug-2021  martin Pull up following revision(s) (requested by skrll in ticket #1330):

common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.18
common/lib/libc/arch/arm/atomic/atomic_and_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.19
common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_inc_32.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_and_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_add_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_inc_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_cas_up.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_add_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_op_asm.h: revision 1.9
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.8
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_op_asm.h: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_and_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_cas_16.S: revision 1.3
common/lib/libc/arch/arm/atomic/sync_fetch_and_and_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_32.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_xor_32.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_cas_32.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_xor_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_add_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/sync_fetch_and_add_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_dec_64.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_add_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/sync_fetch_and_sub_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_dec_64.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_add_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_xor_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_add_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.14
common/lib/libc/arch/arm/atomic/atomic_or_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_nand_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_or_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_cas_8.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_inc_64.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_inc_64.S: revision 1.11
common/lib/libc/arch/arm/atomic/atomic_cas_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_dec_32.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_or_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_dec_32.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_xor_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_4.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/atomic_or_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_2.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_swap_64.S: revision 1.14
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_1.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_and_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_and_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_and_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.14

Give the thumb atomic ops a chance of working

s/ifdef _ARM_ARCH_6/if defined(_ARM_ARCH_6)/ for consistency. NFCI.

Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.

That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)
This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)
This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.3.16.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.3.16.1 21-Apr-2020  martin Sync with HEAD
 1.4 28-Jul-2021  skrll Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)

This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)

This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.3 17-May-2015  justin branches: 1.3.16; 1.3.18;
Move arm sync_* changes to Makefile.inc
 1.2 17-May-2015  justin Do not build arm toolchain symbols in the rump kernel

These symbols will be provided at link time and will be
duplicate symbols in rump kernel and libc if defined.

Many have been fixed previously, but these were missed
as did not have a test.
 1.1 08-Nov-2013  matt branches: 1.1.4; 1.1.8;
Add support for the gcc __sync builtins.
Note that these need earmv6 or later to get the ldrex/strex instructions
 1.1.8.2 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.1.8.1 08-Nov-2013  tls file sync_bool_compare_and_swap_4.S was added on branch tls-maxphys on 2014-08-19 23:45:12 +0000
 1.1.4.2 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.1.4.1 08-Nov-2013  yamt file sync_bool_compare_and_swap_4.S was added on branch yamt-pagecache on 2014-05-22 11:26:28 +0000
 1.3.18.1 11-Aug-2021  martin Pull up following revision(s) (requested by skrll in ticket #1330):

common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.18
common/lib/libc/arch/arm/atomic/atomic_and_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.19
common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_inc_32.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_and_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_add_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_inc_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_cas_up.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_add_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_op_asm.h: revision 1.9
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.8
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_op_asm.h: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_and_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_cas_16.S: revision 1.3
common/lib/libc/arch/arm/atomic/sync_fetch_and_and_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_32.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_xor_32.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_cas_32.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_xor_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_add_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/sync_fetch_and_add_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_dec_64.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_add_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/sync_fetch_and_sub_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_dec_64.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_add_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_xor_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_add_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.14
common/lib/libc/arch/arm/atomic/atomic_or_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_nand_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_or_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_cas_8.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_inc_64.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_inc_64.S: revision 1.11
common/lib/libc/arch/arm/atomic/atomic_cas_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_dec_32.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_or_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_dec_32.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_xor_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_4.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/atomic_or_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_2.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_swap_64.S: revision 1.14
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_1.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_and_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_and_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_and_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.14

Give the thumb atomic ops a chance of working

s/ifdef _ARM_ARCH_6/if defined(_ARM_ARCH_6)/ for consistency. NFCI.

Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.

That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)
This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)
This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.3.16.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.3.16.1 21-Apr-2020  martin Sync with HEAD
 1.5 28-Jul-2021  skrll Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)

This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)

This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.4 17-May-2015  justin branches: 1.4.16; 1.4.18;
Move arm sync_* changes to Makefile.inc
 1.3 17-May-2015  justin Do not build arm toolchain symbols in the rump kernel

These symbols will be provided at link time and will be
duplicate symbols in rump kernel and libc if defined.

Many have been fixed previously, but these were missed
as did not have a test.
 1.2 30-Nov-2013  joerg branches: 1.2.4; 1.2.8;
Use explicit form of register pair operations by specifying both.
 1.1 08-Nov-2013  matt Add support for the gcc __sync builtins.
Note that these need earmv6 or later to get the ldrex/strex instructions
 1.2.8.2 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.2.8.1 30-Nov-2013  tls file sync_bool_compare_and_swap_8.S was added on branch tls-maxphys on 2014-08-19 23:45:12 +0000
 1.2.4.2 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.2.4.1 30-Nov-2013  yamt file sync_bool_compare_and_swap_8.S was added on branch yamt-pagecache on 2014-05-22 11:26:28 +0000
 1.4.18.1 11-Aug-2021  martin Pull up following revision(s) (requested by skrll in ticket #1330):

common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.18
common/lib/libc/arch/arm/atomic/atomic_and_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.19
common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_inc_32.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_and_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_add_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_inc_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_cas_up.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_add_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_op_asm.h: revision 1.9
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.8
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_op_asm.h: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_and_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_cas_16.S: revision 1.3
common/lib/libc/arch/arm/atomic/sync_fetch_and_and_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_32.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_xor_32.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_cas_32.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_xor_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_add_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/sync_fetch_and_add_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_dec_64.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_add_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/sync_fetch_and_sub_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_dec_64.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_add_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_xor_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_add_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.14
common/lib/libc/arch/arm/atomic/atomic_or_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_nand_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_or_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_cas_8.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_inc_64.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_inc_64.S: revision 1.11
common/lib/libc/arch/arm/atomic/atomic_cas_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_dec_32.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_or_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_dec_32.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_xor_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_4.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/atomic_or_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_2.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_swap_64.S: revision 1.14
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_1.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_and_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_and_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_and_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.14

Give the thumb atomic ops a chance of working

s/ifdef _ARM_ARCH_6/if defined(_ARM_ARCH_6)/ for consistency. NFCI.

Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.

That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)
This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)
This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.4.16.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.4.16.1 21-Apr-2020  martin Sync with HEAD
 1.6 28-Jul-2021  skrll Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)

This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)

This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.5 17-May-2015  justin branches: 1.5.16; 1.5.18;
Move arm sync_* changes to Makefile.inc
 1.4 17-May-2015  justin Do not build arm toolchain symbols in the rump kernel

These symbols will be provided at link time and will be
duplicate symbols in rump kernel and libc if defined.

Many have been fixed previously, but these were missed
as did not have a test.
 1.3 23-Jun-2014  joerg branches: 1.3.4;
Add aliases for the builtins used to implement C11/C++11 atomics.
 1.2 30-Nov-2013  joerg branches: 1.2.2; 1.2.4;
Use explicit form of register pair operations by specifying both.
 1.1 08-Nov-2013  matt Add support for the gcc __sync builtins.
Note that these need earmv6 or later to get the ldrex/strex instructions
 1.2.4.2 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.2.4.1 30-Nov-2013  yamt file sync_fetch_and_add_8.S was added on branch yamt-pagecache on 2014-05-22 11:26:28 +0000
 1.2.2.1 10-Aug-2014  tls Rebase.
 1.3.4.2 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.3.4.1 23-Jun-2014  tls file sync_fetch_and_add_8.S was added on branch tls-maxphys on 2014-08-19 23:45:12 +0000
 1.5.18.1 11-Aug-2021  martin Pull up following revision(s) (requested by skrll in ticket #1330):

common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.18
common/lib/libc/arch/arm/atomic/atomic_and_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.19
common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_inc_32.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_and_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_add_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_inc_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_cas_up.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_add_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_op_asm.h: revision 1.9
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.8
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_op_asm.h: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_and_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_cas_16.S: revision 1.3
common/lib/libc/arch/arm/atomic/sync_fetch_and_and_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_32.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_xor_32.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_cas_32.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_xor_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_add_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/sync_fetch_and_add_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_dec_64.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_add_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/sync_fetch_and_sub_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_dec_64.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_add_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_xor_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_add_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.14
common/lib/libc/arch/arm/atomic/atomic_or_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_nand_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_or_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_cas_8.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_inc_64.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_inc_64.S: revision 1.11
common/lib/libc/arch/arm/atomic/atomic_cas_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_dec_32.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_or_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_dec_32.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_xor_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_4.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/atomic_or_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_2.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_swap_64.S: revision 1.14
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_1.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_and_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_and_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_and_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.14

Give the thumb atomic ops a chance of working

s/ifdef _ARM_ARCH_6/if defined(_ARM_ARCH_6)/ for consistency. NFCI.

Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.

That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)
This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)
This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.5.16.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.5.16.1 21-Apr-2020  martin Sync with HEAD
 1.6 28-Jul-2021  skrll Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)

This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)

This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.5 17-May-2015  justin branches: 1.5.16; 1.5.18;
Move arm sync_* changes to Makefile.inc
 1.4 17-May-2015  justin Do not build arm toolchain symbols in the rump kernel

These symbols will be provided at link time and will be
duplicate symbols in rump kernel and libc if defined.

Many have been fixed previously, but these were missed
as did not have a test.
 1.3 23-Jun-2014  joerg branches: 1.3.4;
Add aliases for the builtins used to implement C11/C++11 atomics.
 1.2 30-Nov-2013  joerg branches: 1.2.2; 1.2.4;
Use explicit form of register pair operations by specifying both.
 1.1 08-Nov-2013  matt Add support for the gcc __sync builtins.
Note that these need earmv6 or later to get the ldrex/strex instructions
 1.2.4.2 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.2.4.1 30-Nov-2013  yamt file sync_fetch_and_and_8.S was added on branch yamt-pagecache on 2014-05-22 11:26:28 +0000
 1.2.2.1 10-Aug-2014  tls Rebase.
 1.3.4.2 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.3.4.1 23-Jun-2014  tls file sync_fetch_and_and_8.S was added on branch tls-maxphys on 2014-08-19 23:45:12 +0000
 1.5.18.1 11-Aug-2021  martin Pull up following revision(s) (requested by skrll in ticket #1330):

common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.18
common/lib/libc/arch/arm/atomic/atomic_and_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.19
common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_inc_32.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_and_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_add_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_inc_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_cas_up.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_add_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_op_asm.h: revision 1.9
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.8
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_op_asm.h: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_and_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_cas_16.S: revision 1.3
common/lib/libc/arch/arm/atomic/sync_fetch_and_and_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_32.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_xor_32.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_cas_32.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_xor_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_add_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/sync_fetch_and_add_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_dec_64.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_add_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/sync_fetch_and_sub_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_dec_64.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_add_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_xor_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_add_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.14
common/lib/libc/arch/arm/atomic/atomic_or_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_nand_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_or_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_cas_8.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_inc_64.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_inc_64.S: revision 1.11
common/lib/libc/arch/arm/atomic/atomic_cas_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_dec_32.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_or_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_dec_32.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_xor_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_4.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/atomic_or_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_2.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_swap_64.S: revision 1.14
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_1.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_and_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_and_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_and_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.14

Give the thumb atomic ops a chance of working

s/ifdef _ARM_ARCH_6/if defined(_ARM_ARCH_6)/ for consistency. NFCI.

Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.

That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)
This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)
This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.5.16.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.5.16.1 21-Apr-2020  martin Sync with HEAD
 1.6 28-Jul-2021  skrll Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)

This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)

This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.5 11-Dec-2015  skrll branches: 1.5.16; 1.5.18;
Use gcc 4.4 and later operation for nand, i.e.
*ptr = ~(tmp & value) instead of *ptr = ~tmp & value

There was also another bug in sync_fetch_and_nand_8 which I've also fixed.

PR port-arm32/50513: Incorrect logic for atomic_nand_xx.S
 1.4 17-May-2015  justin Move arm sync_* changes to Makefile.inc
 1.3 17-May-2015  justin Do not build arm toolchain symbols in the rump kernel

These symbols will be provided at link time and will be
duplicate symbols in rump kernel and libc if defined.

Many have been fixed previously, but these were missed
as did not have a test.
 1.2 30-Nov-2013  joerg branches: 1.2.4; 1.2.6; 1.2.8;
Use explicit form of register pair operations by specifying both.
 1.1 08-Nov-2013  matt Add support for the gcc __sync builtins.
Note that these need earmv6 or later to get the ldrex/strex instructions
 1.2.8.2 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.2.8.1 30-Nov-2013  tls file sync_fetch_and_nand_8.S was added on branch tls-maxphys on 2014-08-19 23:45:12 +0000
 1.2.6.1 26-Feb-2016  snj Pull up following revision(s) (requested by skrll in ticket #1105):
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.3
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.3
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.3
common/lib/libc/arch/arm/atomic/sync_fetch_and_nand_8.S: revision 1.5
Use gcc 4.4 and later operation for nand, i.e.
*ptr = ~(tmp & value) instead of *ptr = ~tmp & value
There was also another bug in sync_fetch_and_nand_8 which I've also fixed.
PR port-arm32/50513: Incorrect logic for atomic_nand_xx.S
 1.2.4.2 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.2.4.1 30-Nov-2013  yamt file sync_fetch_and_nand_8.S was added on branch yamt-pagecache on 2014-05-22 11:26:28 +0000
 1.5.18.1 11-Aug-2021  martin Pull up following revision(s) (requested by skrll in ticket #1330):

common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.18
common/lib/libc/arch/arm/atomic/atomic_and_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.19
common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_inc_32.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_and_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_add_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_inc_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_cas_up.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_add_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_op_asm.h: revision 1.9
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.8
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_op_asm.h: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_and_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_cas_16.S: revision 1.3
common/lib/libc/arch/arm/atomic/sync_fetch_and_and_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_32.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_xor_32.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_cas_32.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_xor_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_add_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/sync_fetch_and_add_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_dec_64.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_add_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/sync_fetch_and_sub_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_dec_64.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_add_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_xor_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_add_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.14
common/lib/libc/arch/arm/atomic/atomic_or_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_nand_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_or_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_cas_8.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_inc_64.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_inc_64.S: revision 1.11
common/lib/libc/arch/arm/atomic/atomic_cas_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_dec_32.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_or_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_dec_32.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_xor_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_4.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/atomic_or_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_2.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_swap_64.S: revision 1.14
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_1.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_and_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_and_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_and_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.14

Give the thumb atomic ops a chance of working

s/ifdef _ARM_ARCH_6/if defined(_ARM_ARCH_6)/ for consistency. NFCI.

Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.

That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)
This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)
This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.5.16.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.5.16.1 21-Apr-2020  martin Sync with HEAD
 1.6 28-Jul-2021  skrll Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)

This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)

This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.5 17-May-2015  justin branches: 1.5.16; 1.5.18;
Move arm sync_* changes to Makefile.inc
 1.4 17-May-2015  justin Do not build arm toolchain symbols in the rump kernel

These symbols will be provided at link time and will be
duplicate symbols in rump kernel and libc if defined.

Many have been fixed previously, but these were missed
as did not have a test.
 1.3 23-Jun-2014  joerg branches: 1.3.4;
Add aliases for the builtins used to implement C11/C++11 atomics.
 1.2 30-Nov-2013  joerg branches: 1.2.2; 1.2.4;
Use explicit form of register pair operations by specifying both.
 1.1 08-Nov-2013  matt Add support for the gcc __sync builtins.
Note that these need earmv6 or later to get the ldrex/strex instructions
 1.2.4.2 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.2.4.1 30-Nov-2013  yamt file sync_fetch_and_or_8.S was added on branch yamt-pagecache on 2014-05-22 11:26:28 +0000
 1.2.2.1 10-Aug-2014  tls Rebase.
 1.3.4.2 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.3.4.1 23-Jun-2014  tls file sync_fetch_and_or_8.S was added on branch tls-maxphys on 2014-08-19 23:45:12 +0000
 1.5.18.1 11-Aug-2021  martin Pull up following revision(s) (requested by skrll in ticket #1330):

common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.18
common/lib/libc/arch/arm/atomic/atomic_and_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.19
common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_inc_32.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_and_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_add_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_inc_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_cas_up.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_add_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_op_asm.h: revision 1.9
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.8
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_op_asm.h: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_and_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_cas_16.S: revision 1.3
common/lib/libc/arch/arm/atomic/sync_fetch_and_and_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_32.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_xor_32.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_cas_32.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_xor_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_add_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/sync_fetch_and_add_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_dec_64.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_add_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/sync_fetch_and_sub_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_dec_64.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_add_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_xor_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_add_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.14
common/lib/libc/arch/arm/atomic/atomic_or_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_nand_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_or_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_cas_8.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_inc_64.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_inc_64.S: revision 1.11
common/lib/libc/arch/arm/atomic/atomic_cas_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_dec_32.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_or_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_dec_32.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_xor_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_4.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/atomic_or_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_2.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_swap_64.S: revision 1.14
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_1.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_and_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_and_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_and_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.14

Give the thumb atomic ops a chance of working

s/ifdef _ARM_ARCH_6/if defined(_ARM_ARCH_6)/ for consistency. NFCI.

Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.

That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)
This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)
This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.5.16.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.5.16.1 21-Apr-2020  martin Sync with HEAD
 1.6 28-Jul-2021  skrll Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)

This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)

This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.5 17-May-2015  justin branches: 1.5.16; 1.5.18;
Move arm sync_* changes to Makefile.inc
 1.4 17-May-2015  justin Do not build arm toolchain symbols in the rump kernel

These symbols will be provided at link time and will be
duplicate symbols in rump kernel and libc if defined.

Many have been fixed previously, but these were missed
as did not have a test.
 1.3 23-Jun-2014  joerg branches: 1.3.4;
Add aliases for the builtins used to implement C11/C++11 atomics.
 1.2 30-Nov-2013  joerg branches: 1.2.2; 1.2.4;
Use explicit form of register pair operations by specifying both.
 1.1 08-Nov-2013  matt Add support for the gcc __sync builtins.
Note that these need earmv6 or later to get the ldrex/strex instructions
 1.2.4.2 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.2.4.1 30-Nov-2013  yamt file sync_fetch_and_sub_8.S was added on branch yamt-pagecache on 2014-05-22 11:26:28 +0000
 1.2.2.1 10-Aug-2014  tls Rebase.
 1.3.4.2 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.3.4.1 23-Jun-2014  tls file sync_fetch_and_sub_8.S was added on branch tls-maxphys on 2014-08-19 23:45:12 +0000
 1.5.18.1 11-Aug-2021  martin Pull up following revision(s) (requested by skrll in ticket #1330):

common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.18
common/lib/libc/arch/arm/atomic/atomic_and_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.19
common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_inc_32.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_and_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_add_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_inc_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_cas_up.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_add_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_op_asm.h: revision 1.9
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.8
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_op_asm.h: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_and_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_cas_16.S: revision 1.3
common/lib/libc/arch/arm/atomic/sync_fetch_and_and_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_32.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_xor_32.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_cas_32.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_xor_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_add_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/sync_fetch_and_add_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_dec_64.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_add_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/sync_fetch_and_sub_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_dec_64.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_add_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_xor_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_add_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.14
common/lib/libc/arch/arm/atomic/atomic_or_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_nand_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_or_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_cas_8.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_inc_64.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_inc_64.S: revision 1.11
common/lib/libc/arch/arm/atomic/atomic_cas_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_dec_32.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_or_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_dec_32.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_xor_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_4.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/atomic_or_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_2.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_swap_64.S: revision 1.14
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_1.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_and_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_and_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_and_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.14

Give the thumb atomic ops a chance of working

s/ifdef _ARM_ARCH_6/if defined(_ARM_ARCH_6)/ for consistency. NFCI.

Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.

That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)
This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)
This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.5.16.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.5.16.1 21-Apr-2020  martin Sync with HEAD
 1.6 28-Jul-2021  skrll Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.
That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)

This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)

This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.5 17-May-2015  justin branches: 1.5.16; 1.5.18;
Move arm sync_* changes to Makefile.inc
 1.4 17-May-2015  justin Do not build arm toolchain symbols in the rump kernel

These symbols will be provided at link time and will be
duplicate symbols in rump kernel and libc if defined.

Many have been fixed previously, but these were missed
as did not have a test.
 1.3 23-Jun-2014  joerg branches: 1.3.4;
Add aliases for the builtins used to implement C11/C++11 atomics.
 1.2 30-Nov-2013  joerg branches: 1.2.2; 1.2.4;
Use explicit form of register pair operations by specifying both.
 1.1 08-Nov-2013  matt Add support for the gcc __sync builtins.
Note that these need earmv6 or later to get the ldrex/strex instructions
 1.2.4.2 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.2.4.1 30-Nov-2013  yamt file sync_fetch_and_xor_8.S was added on branch yamt-pagecache on 2014-05-22 11:26:28 +0000
 1.2.2.1 10-Aug-2014  tls Rebase.
 1.3.4.2 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.3.4.1 23-Jun-2014  tls file sync_fetch_and_xor_8.S was added on branch tls-maxphys on 2014-08-19 23:45:12 +0000
 1.5.18.1 11-Aug-2021  martin Pull up following revision(s) (requested by skrll in ticket #1330):

common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.18
common/lib/libc/arch/arm/atomic/atomic_and_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap.S: revision 1.19
common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_inc_32.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_and_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_add_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_xor_64.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_inc_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_cas_up.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_add_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_op_asm.h: revision 1.9
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.8
common/lib/libc/arch/arm/atomic/membar_ops.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_op_asm.h: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_and_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_cas_16.S: revision 1.3
common/lib/libc/arch/arm/atomic/sync_fetch_and_and_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_32.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_xor_32.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_64.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_cas_32.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_xor_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_swap_16.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_add_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/sync_fetch_and_add_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_dec_64.S: revision 1.8
common/lib/libc/arch/arm/atomic/atomic_add_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/sync_fetch_and_sub_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_xor_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_dec_64.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_add_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_xor_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_add_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_or_64.S: revision 1.14
common/lib/libc/arch/arm/atomic/atomic_or_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_nand_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_16.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_or_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_or_32.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_nand_32.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_cas_8.S: revision 1.9
common/lib/libc/arch/arm/atomic/atomic_inc_64.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_inc_64.S: revision 1.11
common/lib/libc/arch/arm/atomic/atomic_cas_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_or_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_dec_32.S: revision 1.6
common/lib/libc/arch/arm/atomic/atomic_or_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_dec_32.S: revision 1.7
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_sub_64.S: revision 1.5
common/lib/libc/arch/arm/atomic/sync_fetch_and_xor_8.S: revision 1.6
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_4.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.12
common/lib/libc/arch/arm/atomic/atomic_or_32.S: revision 1.10
common/lib/libc/arch/arm/atomic/atomic_and_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_2.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_swap_64.S: revision 1.14
common/lib/libc/arch/arm/atomic/sync_bool_compare_and_swap_1.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_and_8.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.13
common/lib/libc/arch/arm/atomic/atomic_nand_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_and_8.S: revision 1.5
common/lib/libc/arch/arm/atomic/atomic_and_16.S: revision 1.4
common/lib/libc/arch/arm/atomic/atomic_add_64.S: revision 1.14

Give the thumb atomic ops a chance of working

s/ifdef _ARM_ARCH_6/if defined(_ARM_ARCH_6)/ for consistency. NFCI.

Remove memory barriers from the atomic_ops(3) atomic operations. They're
not needed for correctness.

Add the correct memory barriers to the gcc legacy __sync built-in
functions for atomic memory access. From the gcc documentation:

In most cases, these built-in functions are considered a full barrier.

That is, no memory operand is moved across the operation, either forward
or backward. Further, instructions are issued as necessary to prevent the
processor from speculating loads across the operation and from queuing
stores after the operation.

type __sync_lock_test_and_set (type *ptr, type value, ...)
This built-in function is not a full barrier, but rather an acquire
barrier. This means that references after the operation cannot move to
(or be speculated to) before the operation, but previous memory stores
may not be globally visible yet, and previous memory loads may not yet
be satisfied.

void __sync_lock_release (type *ptr, ...)
This built-in function is not a full barrier, but rather a release
barrier. This means that all previous memory stores are globally
visible, and all previous memory loads have been satisfied, but
following memory reads are not prevented from being speculated to
before the barrier.
 1.5.16.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.5.16.1 21-Apr-2020  martin Sync with HEAD
 1.3 29-Jan-2014  joerg branches: 1.3.4; 1.3.28;
Kernel and standalone code uses panic() for broken code.
 1.2 08-May-2013  matt When using EABI, call __aeabi_{i,l}div0 when diving by 0.
 1.1 23-Jan-2013  matt branches: 1.1.4;
Add EABI support routines to raise SIGFPE on integer divide by 0.
 1.1.4.4 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.1.4.3 23-Jun-2013  tls resync from head
 1.1.4.2 25-Feb-2013  tls resync with head
 1.1.4.1 23-Jan-2013  tls file __aeabi_idiv0.c was added on branch tls-maxphys on 2013-02-25 00:23:55 +0000
 1.3.28.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.3.28.1 21-Apr-2020  martin Sync with HEAD
 1.3.4.2 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.3.4.1 29-Jan-2014  yamt file __aeabi_idiv0.c was added on branch yamt-pagecache on 2014-05-22 11:26:28 +0000
 1.3 29-Jan-2014  joerg branches: 1.3.4; 1.3.28;
Kernel and standalone code uses panic() for broken code.
 1.2 08-May-2013  matt When using EABI, call __aeabi_{i,l}div0 when diving by 0.
 1.1 23-Jan-2013  matt branches: 1.1.4;
Add EABI support routines to raise SIGFPE on integer divide by 0.
 1.1.4.4 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.1.4.3 23-Jun-2013  tls resync from head
 1.1.4.2 25-Feb-2013  tls resync with head
 1.1.4.1 23-Jan-2013  tls file __aeabi_ldiv0.c was added on branch tls-maxphys on 2013-02-25 00:23:55 +0000
 1.3.28.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.3.28.1 21-Apr-2020  martin Sync with HEAD
 1.3.4.2 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.3.4.1 29-Jan-2014  yamt file __aeabi_ldiv0.c was added on branch yamt-pagecache on 2014-05-22 11:26:28 +0000
 1.6 19-Aug-2013  matt branches: 1.6.26;
Use STRONG_ALIAS
Add thumb variation
 1.5 27-Nov-2012  matt Use the armv6 rev/rev16 if armv6 or later
 1.4 28-Apr-2008  martin branches: 1.4.4; 1.4.24; 1.4.26;
Remove clause 3 and 4 from TNF licenses
 1.3 04-Feb-2006  uwe branches: 1.3.18; 1.3.26;
Define htonl &co as alternative names to bswap only if little endian.
I'm not sure if idempotent versions need to be defined for big endian,
as the man page says in that case they are defined idempotent macros.

PR port-arm/32734
 1.2 04-Feb-2006  uwe libc wants __bswapNN, kernel wants bswapNN. That was not accounted
for during the merge of kernel and libc versions. Fix to match
e.g. i386 code.
 1.1 20-Dec-2005  christos Merge libkern + libc common files. As requested by core.
 1.3.26.1 18-May-2008  yamt sync with head.
 1.3.18.1 28-Aug-2007  matt Add thumb versions (or force to arm32 mode when in thumb mode).
 1.4.26.2 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.4.26.1 25-Feb-2013  tls resync with head
 1.4.24.1 28-Nov-2012  matt Use rev/rev16 on armv6 and later.
 1.4.4.2 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.4.4.1 16-Jan-2013  yamt sync with (a bit old) head
 1.6.26.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.6.26.1 21-Apr-2020  martin Sync with HEAD
 1.9 11-Dec-2020  dholland arm bswap32: fix fatal typo in thumb code (PR 55854)
 1.8 09-Dec-2020  dholland arm bswap32: Improve the comments showing the byte flow.

It's confusing to use 1-4 for bytes 1-4 and then 0 for literal zero,
so use a-d for bytes 1-4.
 1.7 19-Aug-2013  matt branches: 1.7.26;
Add thumb version
Use STRONG_ALIAS
 1.6 16-Jan-2013  matt Add some comments to illustrate what is actually happening.
 1.5 27-Nov-2012  matt Use the armv6 rev/rev16 if armv6 or later
 1.4 28-Apr-2008  martin branches: 1.4.4; 1.4.24; 1.4.26;
Remove clause 3 and 4 from TNF licenses
 1.3 04-Feb-2006  uwe branches: 1.3.18; 1.3.26;
Define htonl &co as alternative names to bswap only if little endian.
I'm not sure if idempotent versions need to be defined for big endian,
as the man page says in that case they are defined idempotent macros.

PR port-arm/32734
 1.2 04-Feb-2006  uwe libc wants __bswapNN, kernel wants bswapNN. That was not accounted
for during the merge of kernel and libc versions. Fix to match
e.g. i386 code.
 1.1 20-Dec-2005  christos Merge libkern + libc common files. As requested by core.
 1.3.26.1 18-May-2008  yamt sync with head.
 1.3.18.1 28-Aug-2007  matt Add thumb versions (or force to arm32 mode when in thumb mode).
 1.4.26.2 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.4.26.1 25-Feb-2013  tls resync with head
 1.4.24.1 28-Nov-2012  matt Use rev/rev16 on armv6 and later.
 1.4.4.3 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.4.4.2 23-Jan-2013  yamt sync with head
 1.4.4.1 16-Jan-2013  yamt sync with (a bit old) head
 1.7.26.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.7.26.1 21-Apr-2020  martin Sync with HEAD
 1.5 29-Jan-2014  joerg Include compiler-rt in libc, libm and libkern.
 1.4 19-Aug-2013  matt This is ARM only
 1.3 20-Jun-2013  matt Add support for __ARM_ARCH_EXT_IDIV__ which is set for those cores
with hardware divide instructions. Note that gcc 4.5.x doesn't support
this so this is just latent. gcc 4.7.x does.
 1.2 08-May-2013  matt When using EABI, call __aeabi_{i,l}div0 when diving by 0.
 1.1 30-Oct-2012  christos branches: 1.1.2; 1.1.4; 1.1.6; 1.1.8;
split udivsi3 and divsi3 to fix static linking. This could be done better.
 1.1.8.3 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.1.8.2 16-Jan-2013  yamt sync with (a bit old) head
 1.1.8.1 30-Oct-2012  yamt file divide.S was added on branch yamt-pagecache on 2013-01-16 05:25:53 +0000
 1.1.6.2 28-Nov-2012  matt Merge from HEAD.
split udivsi3 and divsi3 to fix static linking.
 1.1.6.1 30-Oct-2012  matt file divide.S was added on branch matt-nb6-plus on 2012-11-28 01:53:42 +0000
 1.1.4.4 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.1.4.3 23-Jun-2013  tls resync from head
 1.1.4.2 20-Nov-2012  tls Resync to 2012-11-19 00:00:00 UTC
 1.1.4.1 30-Oct-2012  tls file divide.S was added on branch tls-maxphys on 2012-11-20 02:57:29 +0000
 1.1.2.2 19-Nov-2012  riz Pull up following revision(s) (requested by christos in ticket #654):
common/lib/libc/arch/arm/gen/modsi3.S: revision 1.1
common/lib/libc/arch/arm/gen/umodsi3.S: revision 1.1
sys/lib/libkern/arch/arm/Makefile.inc: revision 1.11
sys/lib/libkern/arch/arm/Makefile.inc: revision 1.12
common/lib/libc/arch/arm/gen/divide.S: revision 1.1
lib/libc/arch/arm/gen/Makefile.inc: revision 1.20
common/lib/libc/arch/arm/gen/divsi3.S: revision 1.3
common/lib/libc/arch/arm/gen/udivsi3.S: revision 1.1
common/lib/libc/arch/arm/gen/divsi3.S: revision 1.4
Split out modsi3 and umodsi3 from the divsi3 file. This is so that
we don't get re-defined symbols in libc from libgcc in static linking.
Example: cc -pthread -static main-calls-pthread-create.c
add split files.
add new divsi3 related files.
add new files to fix static linking
split udivsi3 and divsi3 to fix static linking. This could be done better.
Fixes PR#47139
 1.1.2.1 30-Oct-2012  riz file divide.S was added on branch netbsd-6 on 2012-11-19 18:38:02 +0000
 1.14 29-Jan-2014  joerg Include compiler-rt in libc, libm and libkern.
 1.13 12-Sep-2013  joerg Pass PICFLAGS down to cc-as-as and use __PIC__ to decide if it is small
vs big PIC mode. Retire -DPIC and -DBIGPIC.
 1.12 09-Sep-2013  matt Remove movw/movt due to linker problems.
Check for 0 divisor and __aeabi_idiv0 if needed (EABI && _LIBC only).
 1.11 09-Sep-2013  matt s/__libc/_libc/
 1.10 08-Sep-2013  matt Support using hwdiv instructions if those are available.
But only for EABI.
 1.9 05-Sep-2013  matt Use __ARM_EABI__ and new __UNWIND_TABLES__ to decide when to use .cfi ops
 1.8 22-Aug-2013  matt Don't include .cfi info if _KERNEL || _STANDALONE
 1.7 19-Aug-2013  matt Thumbify (and use .cfi ops)
 1.6 20-Jun-2013  matt Add support for __ARM_ARCH_EXT_IDIV__ which is set for those cores
with hardware divide instructions. Note that gcc 4.5.x doesn't support
this so this is just latent. gcc 4.7.x does.
 1.5 08-May-2013  matt When using EABI, call __aeabi_{i,l}div0 when diving by 0.
 1.4 30-Oct-2012  christos split udivsi3 and divsi3 to fix static linking. This could be done better.
 1.3 10-Oct-2012  christos Split out modsi3 and umodsi3 from the divsi3 file. This is so that
we don't get re-defined symbols in libc from libgcc in static linking.
Example: cc -pthread -static main-calls-pthread-create.c
 1.2 05-Aug-2012  matt branches: 1.2.2;
For __udivsi3 and __divsi3, add their EABI aliases as alternate entry
points.
 1.1 20-Dec-2005  christos branches: 1.1.18; 1.1.50; 1.1.52; 1.1.54;
Merge libkern + libc common files. As requested by core.
 1.1.54.1 28-Nov-2012  matt Merge from HEAD.
split udivsi3 and divsi3 to fix static linking.
 1.1.52.2 20-Nov-2012  msaitoh The last commit message said rev. 1.4 was included, but it wasn't in divsi3.
Apply rev. 1.4.
 1.1.52.1 19-Nov-2012  riz Pull up following revision(s) (requested by christos in ticket #654):
common/lib/libc/arch/arm/gen/modsi3.S: revision 1.1
common/lib/libc/arch/arm/gen/umodsi3.S: revision 1.1
sys/lib/libkern/arch/arm/Makefile.inc: revision 1.11
sys/lib/libkern/arch/arm/Makefile.inc: revision 1.12
common/lib/libc/arch/arm/gen/divide.S: revision 1.1
lib/libc/arch/arm/gen/Makefile.inc: revision 1.20
common/lib/libc/arch/arm/gen/divsi3.S: revision 1.3
common/lib/libc/arch/arm/gen/udivsi3.S: revision 1.1
common/lib/libc/arch/arm/gen/divsi3.S: revision 1.4
Split out modsi3 and umodsi3 from the divsi3 file. This is so that
we don't get re-defined symbols in libc from libgcc in static linking.
Example: cc -pthread -static main-calls-pthread-create.c
add split files.
add new divsi3 related files.
add new files to fix static linking
split udivsi3 and divsi3 to fix static linking. This could be done better.
Fixes PR#47139
 1.1.50.3 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.1.50.2 16-Jan-2013  yamt sync with (a bit old) head
 1.1.50.1 30-Oct-2012  yamt sync with head
 1.1.18.1 28-Aug-2007  matt Add thumb versions (or force to arm32 mode when in thumb mode).
 1.2.2.3 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.2.2.2 23-Jun-2013  tls resync from head
 1.2.2.1 20-Nov-2012  tls Resync to 2012-11-19 00:00:00 UTC
 1.5 29-Jan-2014  joerg Include compiler-rt in libc, libm and libkern.
 1.4 15-Aug-2013  matt Only assemble if !__ARM_EABI__
 1.3 20-Jun-2013  matt Add support for __ARM_ARCH_EXT_IDIV__ which is set for those cores
with hardware divide instructions. Note that gcc 4.5.x doesn't support
this so this is just latent. gcc 4.7.x does.
 1.2 28-Nov-2012  matt branches: 1.2.2;
Optimize.
 1.1 10-Oct-2012  christos branches: 1.1.2; 1.1.4; 1.1.6;
Split out modsi3 and umodsi3 from the divsi3 file. This is so that
we don't get re-defined symbols in libc from libgcc in static linking.
Example: cc -pthread -static main-calls-pthread-create.c
 1.1.6.5 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.1.6.4 23-Jun-2013  tls resync from head
 1.1.6.3 25-Feb-2013  tls resync with head
 1.1.6.2 20-Nov-2012  tls Resync to 2012-11-19 00:00:00 UTC
 1.1.6.1 10-Oct-2012  tls file modsi3.S was added on branch tls-maxphys on 2012-11-20 02:57:29 +0000
 1.1.4.2 19-Nov-2012  riz Pull up following revision(s) (requested by christos in ticket #654):
common/lib/libc/arch/arm/gen/modsi3.S: revision 1.1
common/lib/libc/arch/arm/gen/umodsi3.S: revision 1.1
sys/lib/libkern/arch/arm/Makefile.inc: revision 1.11
sys/lib/libkern/arch/arm/Makefile.inc: revision 1.12
common/lib/libc/arch/arm/gen/divide.S: revision 1.1
lib/libc/arch/arm/gen/Makefile.inc: revision 1.20
common/lib/libc/arch/arm/gen/divsi3.S: revision 1.3
common/lib/libc/arch/arm/gen/udivsi3.S: revision 1.1
common/lib/libc/arch/arm/gen/divsi3.S: revision 1.4
Split out modsi3 and umodsi3 from the divsi3 file. This is so that
we don't get re-defined symbols in libc from libgcc in static linking.
Example: cc -pthread -static main-calls-pthread-create.c
add split files.
add new divsi3 related files.
add new files to fix static linking
split udivsi3 and divsi3 to fix static linking. This could be done better.
Fixes PR#47139
 1.1.4.1 10-Oct-2012  riz file modsi3.S was added on branch netbsd-6 on 2012-11-19 18:38:01 +0000
 1.1.2.4 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.1.2.3 16-Jan-2013  yamt sync with (a bit old) head
 1.1.2.2 30-Oct-2012  yamt sync with head
 1.1.2.1 10-Oct-2012  yamt file modsi3.S was added on branch yamt-pagecache on 2012-10-30 18:46:12 +0000
 1.2.2.2 28-Nov-2012  matt Merge from HEAD.
split udivsi3 and divsi3 to fix static linking.
 1.2.2.1 28-Nov-2012  matt file modsi3.S was added on branch matt-nb6-plus on 2012-11-28 01:53:42 +0000
 1.2 18-Dec-2012  matt branches: 1.2.2; 1.2.6; 1.2.36;
Don't need to include assym.h
Add a missing comma.
 1.1 17-Dec-2012  matt Add a routine to create an up to an 128 bitmask returned
in VFP/NEON q0 starting at the rightmost bit (bit 0).
 1.2.36.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.2.36.1 21-Apr-2020  martin Sync with HEAD
 1.2.6.2 25-Feb-2013  tls resync with head
 1.2.6.1 18-Dec-2012  tls file neon_mask.S was added on branch tls-maxphys on 2013-02-25 00:23:55 +0000
 1.2.2.2 23-Jan-2013  yamt sync with head
 1.2.2.1 18-Dec-2012  yamt file neon_mask.S was added on branch yamt-pagecache on 2013-01-23 00:04:06 +0000
 1.10 29-Jan-2014  joerg Include compiler-rt in libc, libm and libkern.
 1.9 12-Sep-2013  joerg Pass PICFLAGS down to cc-as-as and use __PIC__ to decide if it is small
vs big PIC mode. Retire -DPIC and -DBIGPIC.
 1.8 09-Sep-2013  matt Remove movw/movt due to linker problems.
Check for 0 divisor and __aeabi_idiv0 if needed (EABI && _LIBC only).
 1.7 09-Sep-2013  matt s/__libc/_libc/
 1.6 08-Sep-2013  matt Support using hwdiv instructions if those are available.
But only for EABI.
 1.5 05-Sep-2013  matt Use __ARM_EABI__ and new __UNWIND_TABLES__ to decide when to use .cfi ops
 1.4 22-Aug-2013  matt Don't include .cfi info if _KERNEL || _STANDALONE
 1.3 19-Aug-2013  matt thumbify
add .cfi ops (for thumb)
 1.2 20-Jun-2013  matt Add support for __ARM_ARCH_EXT_IDIV__ which is set for those cores
with hardware divide instructions. Note that gcc 4.5.x doesn't support
this so this is just latent. gcc 4.7.x does.
 1.1 30-Oct-2012  christos branches: 1.1.2; 1.1.4; 1.1.6; 1.1.8;
split udivsi3 and divsi3 to fix static linking. This could be done better.
 1.1.8.3 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.1.8.2 16-Jan-2013  yamt sync with (a bit old) head
 1.1.8.1 30-Oct-2012  yamt file udivsi3.S was added on branch yamt-pagecache on 2013-01-16 05:25:53 +0000
 1.1.6.2 28-Nov-2012  matt Merge from HEAD.
split udivsi3 and divsi3 to fix static linking.
 1.1.6.1 30-Oct-2012  matt file udivsi3.S was added on branch matt-nb6-plus on 2012-11-28 01:53:42 +0000
 1.1.4.4 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.1.4.3 23-Jun-2013  tls resync from head
 1.1.4.2 20-Nov-2012  tls Resync to 2012-11-19 00:00:00 UTC
 1.1.4.1 30-Oct-2012  tls file udivsi3.S was added on branch tls-maxphys on 2012-11-20 02:57:29 +0000
 1.1.2.2 19-Nov-2012  riz Pull up following revision(s) (requested by christos in ticket #654):
common/lib/libc/arch/arm/gen/modsi3.S: revision 1.1
common/lib/libc/arch/arm/gen/umodsi3.S: revision 1.1
sys/lib/libkern/arch/arm/Makefile.inc: revision 1.11
sys/lib/libkern/arch/arm/Makefile.inc: revision 1.12
common/lib/libc/arch/arm/gen/divide.S: revision 1.1
lib/libc/arch/arm/gen/Makefile.inc: revision 1.20
common/lib/libc/arch/arm/gen/divsi3.S: revision 1.3
common/lib/libc/arch/arm/gen/udivsi3.S: revision 1.1
common/lib/libc/arch/arm/gen/divsi3.S: revision 1.4
Split out modsi3 and umodsi3 from the divsi3 file. This is so that
we don't get re-defined symbols in libc from libgcc in static linking.
Example: cc -pthread -static main-calls-pthread-create.c
add split files.
add new divsi3 related files.
add new files to fix static linking
split udivsi3 and divsi3 to fix static linking. This could be done better.
Fixes PR#47139
 1.1.2.1 30-Oct-2012  riz file udivsi3.S was added on branch netbsd-6 on 2012-11-19 18:38:02 +0000
 1.5 29-Jan-2014  joerg Include compiler-rt in libc, libm and libkern.
 1.4 15-Aug-2013  matt Only assemble if !__ARM_EABI__
 1.3 20-Jun-2013  matt Add support for __ARM_ARCH_EXT_IDIV__ which is set for those cores
with hardware divide instructions. Note that gcc 4.5.x doesn't support
this so this is just latent. gcc 4.7.x does.
 1.2 28-Nov-2012  matt branches: 1.2.2;
Optimize.
 1.1 10-Oct-2012  christos branches: 1.1.2; 1.1.4; 1.1.6;
Split out modsi3 and umodsi3 from the divsi3 file. This is so that
we don't get re-defined symbols in libc from libgcc in static linking.
Example: cc -pthread -static main-calls-pthread-create.c
 1.1.6.5 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.1.6.4 23-Jun-2013  tls resync from head
 1.1.6.3 25-Feb-2013  tls resync with head
 1.1.6.2 20-Nov-2012  tls Resync to 2012-11-19 00:00:00 UTC
 1.1.6.1 10-Oct-2012  tls file umodsi3.S was added on branch tls-maxphys on 2012-11-20 02:57:29 +0000
 1.1.4.2 19-Nov-2012  riz Pull up following revision(s) (requested by christos in ticket #654):
common/lib/libc/arch/arm/gen/modsi3.S: revision 1.1
common/lib/libc/arch/arm/gen/umodsi3.S: revision 1.1
sys/lib/libkern/arch/arm/Makefile.inc: revision 1.11
sys/lib/libkern/arch/arm/Makefile.inc: revision 1.12
common/lib/libc/arch/arm/gen/divide.S: revision 1.1
lib/libc/arch/arm/gen/Makefile.inc: revision 1.20
common/lib/libc/arch/arm/gen/divsi3.S: revision 1.3
common/lib/libc/arch/arm/gen/udivsi3.S: revision 1.1
common/lib/libc/arch/arm/gen/divsi3.S: revision 1.4
Split out modsi3 and umodsi3 from the divsi3 file. This is so that
we don't get re-defined symbols in libc from libgcc in static linking.
Example: cc -pthread -static main-calls-pthread-create.c
add split files.
add new divsi3 related files.
add new files to fix static linking
split udivsi3 and divsi3 to fix static linking. This could be done better.
Fixes PR#47139
 1.1.4.1 10-Oct-2012  riz file umodsi3.S was added on branch netbsd-6 on 2012-11-19 18:38:02 +0000
 1.1.2.4 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.1.2.3 16-Jan-2013  yamt sync with (a bit old) head
 1.1.2.2 30-Oct-2012  yamt sync with head
 1.1.2.1 10-Oct-2012  yamt file umodsi3.S was added on branch yamt-pagecache on 2012-10-30 18:46:12 +0000
 1.2.2.2 28-Nov-2012  matt Merge from HEAD.
split udivsi3 and divsi3 to fix static linking.
 1.2.2.1 28-Nov-2012  matt file umodsi3.S was added on branch matt-nb6-plus on 2012-11-28 01:53:42 +0000
 1.2 06-Aug-2012  matt branches: 1.2.4; 1.2.36;
Change include to <arm/aeabi.h>
 1.1 06-Aug-2012  matt Add long long / unsigned long long comparision routines. These differ
from cmpdi2 in they return strcmp like values (-1, 0, 1).
 1.2.36.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.2.36.1 21-Apr-2020  martin Sync with HEAD
 1.2.4.2 30-Oct-2012  yamt sync with head
 1.2.4.1 06-Aug-2012  yamt file __aeabi_lcmp.c was added on branch yamt-pagecache on 2012-10-30 18:46:12 +0000
 1.13 06-May-2014  joerg branches: 1.13.24;
Make EHABI optional.
 1.12 19-Aug-2013  matt branches: 1.12.2;
Add cfi ops.
Thumbify
 1.11 13-Aug-2013  matt Use mvnge AHI, #0x80000000 instead of mvnge AHI, ALO, lsr #1
 1.10 13-Aug-2013  matt 0x800000000 -> 0x80000000 (one too many zeroes)
 1.9 13-Aug-2013  matt Fix movlt
 1.8 13-Aug-2013  matt andlt -> movlt
 1.7 09-May-2013  matt Don't worry about divide by 0 for kernel or standalone
 1.6 09-May-2013  skrll Don't use old syntax - it breaks the build. hi matt.
 1.5 08-May-2013  matt When using EABI, call __aeabi_{i,l}div0 when diving by 0.
 1.4 05-May-2013  skrll Whitespace.
 1.3 19-Apr-2013  skrll Fix logic inversion.
 1.2 14-Apr-2013  matt Fix calling of __qdivrem which the 3rd arg needs to passed on the stack.
 1.1 05-Aug-2012  matt branches: 1.1.2; 1.1.4;
Add another ARM EABI runtime routine for
combined quotient / remainder for signed long long.
 1.1.4.3 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.1.4.2 30-Oct-2012  yamt sync with head
 1.1.4.1 05-Aug-2012  yamt file __aeabi_ldivmod.S was added on branch yamt-pagecache on 2012-10-30 18:46:13 +0000
 1.1.2.2 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.1.2.1 23-Jun-2013  tls resync from head
 1.12.2.1 10-Aug-2014  tls Rebase.
 1.13.24.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.13.24.1 21-Apr-2020  martin Sync with HEAD
 1.2 06-Aug-2012  matt branches: 1.2.4; 1.2.36;
Change include to <arm/aeabi.h>
 1.1 06-Aug-2012  matt Add long long / unsigned long long comparision routines. These differ
from cmpdi2 in they return strcmp like values (-1, 0, 1).
 1.2.36.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.2.36.1 21-Apr-2020  martin Sync with HEAD
 1.2.4.2 30-Oct-2012  yamt sync with head
 1.2.4.1 06-Aug-2012  yamt file __aeabi_ulcmp.c was added on branch yamt-pagecache on 2012-10-30 18:46:13 +0000
 1.9 06-May-2014  joerg branches: 1.9.24;
Make EHABI optional.
 1.8 12-Dec-2013  matt branches: 1.8.2;
Fix a bug in the divby0 case.
 1.7 19-Aug-2013  matt Add .cfi ops
Thumbify
 1.6 09-May-2013  matt Don't worry about divide by 0 for kernel or standalone
 1.5 08-May-2013  matt When using EABI, call __aeabi_{i,l}div0 when diving by 0.
 1.4 14-Apr-2013  skrll Pretty sure we don't want to change instruction set here. hi matt.
 1.3 14-Apr-2013  matt Fix calling of __qdivrem which the 3rd arg needs to passed on the stack.
 1.2 05-Aug-2012  matt branches: 1.2.2; 1.2.4;
Add RCSID.
Simplify.
 1.1 05-Aug-2012  matt Add a routine for __aeabi_uldivmod which is just a wrapper around __qdivrem
but returns the 64-bit dividend and remainder in r0-r3.
 1.2.4.3 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.2.4.2 30-Oct-2012  yamt sync with head
 1.2.4.1 05-Aug-2012  yamt file __aeabi_uldivmod.S was added on branch yamt-pagecache on 2012-10-30 18:46:13 +0000
 1.2.2.2 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.2.2.1 23-Jun-2013  tls resync from head
 1.8.2.1 10-Aug-2014  tls Rebase.
 1.9.24.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.9.24.1 21-Apr-2020  martin Sync with HEAD
 1.9 17-May-2015  justin branches: 1.9.16;
It is just the __ffssi2 alias we do not want in rump kernel
 1.8 17-May-2015  justin Do not build arm toolchain symbols in the rump kernel

These symbols will be provided at link time and will be
duplicate symbols in rump kernel and libc if defined.

Many have been fixed previously, but these were missed
as did not have a test.
 1.7 23-Jan-2014  martin PR port-arm/48543: do provide __ffssi2 as strong alias (in case libgcc.a gets
not linked in) for now.
OK: skrll@
 1.6 30-Sep-2013  skrll More (stylistic) whitespace.
 1.5 30-Sep-2013  skrll Appease new gas
 1.4 19-Aug-2013  matt Add END() and clarify thumb/arm
 1.3 11-Aug-2013  matt Convert some more conditional instructions to unified syntax
 1.2 03-Jul-2011  matt branches: 1.2.2; 1.2.8;
Add a weak symbol definition for __ffssi2 so that __builtin_ffs will use
this if no strong defintion of __ffssi2 is available.
 1.1 20-Dec-2005  christos branches: 1.1.18;
Merge libkern + libc common files. As requested by core.
 1.1.18.1 28-Aug-2007  matt Add thumb versions (or force to arm32 mode when in thumb mode).
 1.2.8.1 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.2.2.1 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.9.16.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.9.16.1 21-Apr-2020  martin Sync with HEAD
 1.5 26-Mar-2015  justin branches: 1.5.16;
Don't build the __eabi_ aliases in the arm rump kernel.

These are provided by libc in userspace, so leave out of kernel
 1.4 02-Dec-2013  joerg Improve EABI handling of string functions. Most importantly, fix
__aeabi_memset, which has the arguments in the wrong order.
 1.3 19-Aug-2013  matt Add END()
 1.2 11-Aug-2013  matt Switch to unified syntax
use RET/RETc
use push/pop
 1.1 20-Dec-2005  christos branches: 1.1.18; 1.1.50; 1.1.56;
Merge libkern + libc common files. As requested by core.
 1.1.56.1 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.1.50.1 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.1.18.1 28-Aug-2007  matt Add thumb versions (or force to arm32 mode when in thumb mode).
 1.5.16.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.5.16.1 21-Apr-2020  martin Sync with HEAD
 1.8 07-Jun-2015  joerg branches: 1.8.16;
Add ARM EABI aliases for aligned arguments.
 1.7 26-Mar-2015  justin Don't build the __eabi_ aliases in the arm rump kernel.

These are provided by libc in userspace, so leave out of kernel
 1.6 02-Dec-2013  joerg branches: 1.6.4;
Improve EABI handling of string functions. Most importantly, fix
__aeabi_memset, which has the arguments in the wrong order.
 1.5 20-Aug-2013  matt Unless we are using an XSCALE, default to the normal arm version of memcpy.
 1.4 15-Feb-2013  matt #include <arm/cdefs.h> to get _ARM_ARCH_DWORD_OK
 1.3 08-Feb-2013  matt Fix typo.
 1.2 12-Dec-2012  matt Change __XSCALE__ to _ARM_ARCH_DWORD_OK so that any cpu with dword load/store
can use it.
 1.1 20-Dec-2005  christos branches: 1.1.50; 1.1.54; 1.1.56;
Merge libkern + libc common files. As requested by core.
 1.1.56.2 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.1.56.1 25-Feb-2013  tls resync with head
 1.1.54.3 15-Feb-2013  matt #include <arm/cdefs.h> to get _ARM_ARCH_DWORD_OK
 1.1.54.2 08-Feb-2013  matt Fix typo.
 1.1.54.1 07-Feb-2013  matt Sync with HEAD
 1.1.50.2 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.1.50.1 16-Jan-2013  yamt sync with (a bit old) head
 1.6.4.1 10-Jun-2015  snj Pull up following revision(s) (requested by joerg in ticket #834):
common/lib/libc/arch/arm/string/memcpy.S: revision 1.8
Add ARM EABI aliases for aligned arguments.
 1.8.16.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.8.16.1 21-Apr-2020  martin Sync with HEAD
 1.7 24-Jan-2023  mlelstv Fix two signed comparisons that were missed in the last patch.
Found be rillig@
 1.6 19-Jan-2023  mlelstv Use unsigned comparisons for pointers and size_t values.
 1.5 02-Dec-2013  joerg branches: 1.5.26; 1.5.40;
Improve EABI handling of string functions. Most importantly, fix
__aeabi_memset, which has the arguments in the wrong order.
 1.4 11-Aug-2013  matt Switch to unified syntax
use RET/RETc
use push/pop
 1.3 28-Jan-2013  matt Add aeabi strong aliases.
 1.2 28-Apr-2008  martin branches: 1.2.4; 1.2.24; 1.2.26;
Remove clause 3 and 4 from TNF licenses
 1.1 20-Dec-2005  christos branches: 1.1.18; 1.1.26;
Merge libkern + libc common files. As requested by core.
 1.1.26.1 18-May-2008  yamt sync with head.
 1.1.18.1 28-Aug-2007  matt Add thumb versions (or force to arm32 mode when in thumb mode).
 1.2.26.2 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.2.26.1 25-Feb-2013  tls resync with head
 1.2.24.1 07-Feb-2013  matt Sync with HEAD
 1.2.4.1 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.5.40.1 20-Jul-2024  martin Pull up following revision(s) (requested by rin in ticket #745):

common/lib/libc/arch/arm/string/memcpy_arm.S: revision 1.6
common/lib/libc/arch/arm/string/memcpy_arm.S: revision 1.7
common/lib/libc/arch/arm/string/memcpy_xscale.S: revision 1.6
common/lib/libc/arch/arm/string/memcpy_neon.S: revision 1.2
common/lib/libc/arch/arm/string/memset_naive.S: revision 1.2
common/lib/libc/arch/arm/string/memmove.S: revision 1.11
common/lib/libc/arch/arm/string/strlen_neon.S: revision 1.4
common/lib/libc/arch/arm/string/memset.S: revision 1.9

Use unsigned comparisons for pointers and size_t values.

Fix two signed comparisons that were missed in the last patch.

Found be rillig@
 1.5.26.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.5.26.1 21-Apr-2020  martin Sync with HEAD
 1.3 27-Feb-2025  andvar Fix various typos in comments.
 1.2 19-Jan-2023  mlelstv branches: 1.2.6;
Use unsigned comparisons for pointers and size_t values.
 1.1 03-Jan-2013  matt branches: 1.1.2; 1.1.4; 1.1.8; 1.1.38; 1.1.52;
This is a working version of memcpy implemented using NEON instructions.
Still needs tuning as it is still about 15% than the non-NEON version.
 1.1.52.1 20-Jul-2024  martin Pull up following revision(s) (requested by rin in ticket #745):

common/lib/libc/arch/arm/string/memcpy_arm.S: revision 1.6
common/lib/libc/arch/arm/string/memcpy_arm.S: revision 1.7
common/lib/libc/arch/arm/string/memcpy_xscale.S: revision 1.6
common/lib/libc/arch/arm/string/memcpy_neon.S: revision 1.2
common/lib/libc/arch/arm/string/memset_naive.S: revision 1.2
common/lib/libc/arch/arm/string/memmove.S: revision 1.11
common/lib/libc/arch/arm/string/strlen_neon.S: revision 1.4
common/lib/libc/arch/arm/string/memset.S: revision 1.9

Use unsigned comparisons for pointers and size_t values.

Fix two signed comparisons that were missed in the last patch.

Found be rillig@
 1.1.38.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.1.38.1 21-Apr-2020  martin Sync with HEAD
 1.1.8.2 25-Feb-2013  tls resync with head
 1.1.8.1 03-Jan-2013  tls file memcpy_neon.S was added on branch tls-maxphys on 2013-02-25 00:23:56 +0000
 1.1.4.2 07-Feb-2013  matt Sync with HEAD
 1.1.4.1 03-Jan-2013  matt file memcpy_neon.S was added on branch matt-nb6-plus on 2013-02-07 07:05:59 +0000
 1.1.2.2 23-Jan-2013  yamt sync with head
 1.1.2.1 03-Jan-2013  yamt file memcpy_neon.S was added on branch yamt-pagecache on 2013-01-23 00:04:06 +0000
 1.2.6.1 02-Aug-2025  perseant Sync with HEAD
 1.6 19-Jan-2023  mlelstv Use unsigned comparisons for pointers and size_t values.
 1.5 17-Dec-2013  joerg branches: 1.5.26; 1.5.40;
Write out register pairs for strd.
 1.4 19-Aug-2013  matt Add END(memcpy)
 1.3 11-Aug-2013  matt Switch to unified syntax
use RET/RETc
use push/pop
 1.2 21-Jun-2007  scw branches: 1.2.4; 1.2.34; 1.2.40;
Apply the patch, with some minor tweaks, supplied in PR/36513.
This prevents a possible prefetch past the end of the source buffer.

Note that the semantics of the pld instruction mean that it is unlikely
that this would have caused any problems except in very specific
circumstances in some types of device drivers.
 1.1 20-Dec-2005  christos branches: 1.1.6; 1.1.14;
Merge libkern + libc common files. As requested by core.
 1.1.14.1 03-Sep-2007  wrstuden Sync w/ NetBSD-4-RC_1
 1.1.6.1 22-Jun-2007  liamjfoy Pull up following revision(s) (requested by scw in ticket #741):
common/lib/libc/arch/arm/string/memcpy_xscale.S: revision 1.2
Apply the patch, with some minor tweaks, supplied in PR/36513.
This prevents a possible prefetch past the end of the source buffer.
Note that the semantics of the pld instruction mean that it is unlikely
that this would have caused any problems except in very specific
circumstances in some types of device drivers.
 1.2.40.1 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.2.34.1 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.2.4.1 28-Aug-2007  matt Add thumb versions (or force to arm32 mode when in thumb mode).
 1.5.40.1 20-Jul-2024  martin Pull up following revision(s) (requested by rin in ticket #745):

common/lib/libc/arch/arm/string/memcpy_arm.S: revision 1.6
common/lib/libc/arch/arm/string/memcpy_arm.S: revision 1.7
common/lib/libc/arch/arm/string/memcpy_xscale.S: revision 1.6
common/lib/libc/arch/arm/string/memcpy_neon.S: revision 1.2
common/lib/libc/arch/arm/string/memset_naive.S: revision 1.2
common/lib/libc/arch/arm/string/memmove.S: revision 1.11
common/lib/libc/arch/arm/string/strlen_neon.S: revision 1.4
common/lib/libc/arch/arm/string/memset.S: revision 1.9

Use unsigned comparisons for pointers and size_t values.

Fix two signed comparisons that were missed in the last patch.

Found be rillig@
 1.5.26.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.5.26.1 21-Apr-2020  martin Sync with HEAD
 1.11 19-Jan-2023  mlelstv Use unsigned comparisons for pointers and size_t values.
 1.10 13-Apr-2017  skrll branches: 1.10.12; 1.10.26;
Comment alignment. No functional change.
 1.9 26-Mar-2015  justin branches: 1.9.2; 1.9.4;
Don't build the __eabi_ aliases in the arm rump kernel.

These are provided by libc in userspace, so leave out of kernel
 1.8 02-Dec-2013  joerg Improve EABI handling of string functions. Most importantly, fix
__aeabi_memset, which has the arguments in the wrong order.
 1.7 30-Nov-2013  joerg Use PLT_SYM.
 1.6 19-Aug-2013  matt Add END()
 1.5 11-Aug-2013  matt Switch to unified syntax
use RET/RETc
use push/pop
 1.4 28-Jan-2013  matt Add aeabi strong aliases.
 1.3 28-Apr-2008  martin branches: 1.3.4; 1.3.24; 1.3.26;
Remove clause 3 and 4 from TNF licenses
 1.2 20-Jun-2007  scw branches: 1.2.4; 1.2.10;
Apply the patch supplied in PR/36512 to fix the buffer overlap check.
 1.1 20-Dec-2005  christos branches: 1.1.6; 1.1.14;
Merge libkern + libc common files. As requested by core.
 1.1.14.1 03-Sep-2007  wrstuden Sync w/ NetBSD-4-RC_1
 1.1.6.1 21-Jun-2007  liamjfoy Pull up following revision(s) (requested by scw in ticket #740):
common/lib/libc/arch/arm/string/memmove.S: revision 1.2
Apply the patch supplied in PR/36512 to fix the buffer overlap check.
 1.2.10.1 18-May-2008  yamt sync with head.
 1.2.4.1 28-Aug-2007  matt Add thumb versions (or force to arm32 mode when in thumb mode).
 1.3.26.2 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.3.26.1 25-Feb-2013  tls resync with head
 1.3.24.1 07-Feb-2013  matt Sync with HEAD
 1.3.4.1 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.9.4.1 21-Apr-2017  bouyer Sync with HEAD
 1.9.2.1 26-Apr-2017  pgoyette Sync with HEAD
 1.10.26.1 20-Jul-2024  martin Pull up following revision(s) (requested by rin in ticket #745):

common/lib/libc/arch/arm/string/memcpy_arm.S: revision 1.6
common/lib/libc/arch/arm/string/memcpy_arm.S: revision 1.7
common/lib/libc/arch/arm/string/memcpy_xscale.S: revision 1.6
common/lib/libc/arch/arm/string/memcpy_neon.S: revision 1.2
common/lib/libc/arch/arm/string/memset_naive.S: revision 1.2
common/lib/libc/arch/arm/string/memmove.S: revision 1.11
common/lib/libc/arch/arm/string/strlen_neon.S: revision 1.4
common/lib/libc/arch/arm/string/memset.S: revision 1.9

Use unsigned comparisons for pointers and size_t values.

Fix two signed comparisons that were missed in the last patch.

Found be rillig@
 1.10.12.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.10.12.1 21-Apr-2020  martin Sync with HEAD
 1.9 19-Jan-2023  mlelstv Use unsigned comparisons for pointers and size_t values.
 1.8 26-Mar-2015  justin branches: 1.8.16; 1.8.30;
Don't build the __eabi_ aliases in the arm rump kernel.

These are provided by libc in userspace, so leave out of kernel
 1.7 02-Dec-2013  joerg Improve EABI handling of string functions. Most importantly, fix
__aeabi_memset, which has the arguments in the wrong order.
 1.6 30-Nov-2013  joerg Use explicit form of register pair operations by specifying both.
 1.5 19-Aug-2013  matt Add END()
 1.4 11-Aug-2013  matt Switch to unified syntax
use RET/RETc
use push/pop
 1.3 28-Jan-2013  matt Add aeabi strong aliases.
 1.2 12-Dec-2012  matt Change __XSCALE__ to _ARM_ARCH_DWORD_OK so that any cpu with strd can use it.
 1.1 20-Dec-2005  christos branches: 1.1.18; 1.1.50; 1.1.54; 1.1.56;
Merge libkern + libc common files. As requested by core.
 1.1.56.2 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.1.56.1 25-Feb-2013  tls resync with head
 1.1.54.1 07-Feb-2013  matt Sync with HEAD
 1.1.50.2 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.1.50.1 16-Jan-2013  yamt sync with (a bit old) head
 1.1.18.1 28-Aug-2007  matt Add thumb versions (or force to arm32 mode when in thumb mode).
 1.8.30.1 20-Jul-2024  martin Pull up following revision(s) (requested by rin in ticket #745):

common/lib/libc/arch/arm/string/memcpy_arm.S: revision 1.6
common/lib/libc/arch/arm/string/memcpy_arm.S: revision 1.7
common/lib/libc/arch/arm/string/memcpy_xscale.S: revision 1.6
common/lib/libc/arch/arm/string/memcpy_neon.S: revision 1.2
common/lib/libc/arch/arm/string/memset_naive.S: revision 1.2
common/lib/libc/arch/arm/string/memmove.S: revision 1.11
common/lib/libc/arch/arm/string/strlen_neon.S: revision 1.4
common/lib/libc/arch/arm/string/memset.S: revision 1.9

Use unsigned comparisons for pointers and size_t values.

Fix two signed comparisons that were missed in the last patch.

Found be rillig@
 1.8.16.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.8.16.1 21-Apr-2020  martin Sync with HEAD
 1.2 14-Jan-2013  matt branches: 1.2.2; 1.2.4; 1.2.8; 1.2.38;
Fix two typos
 1.1 12-Jan-2013  matt A version of memset that can do NEON, VFP as well as normal arm instructions
 1.2.38.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.2.38.1 21-Apr-2020  martin Sync with HEAD
 1.2.8.2 25-Feb-2013  tls resync with head
 1.2.8.1 14-Jan-2013  tls file memset_arm.S was added on branch tls-maxphys on 2013-02-25 00:23:56 +0000
 1.2.4.2 07-Feb-2013  matt Sync with HEAD
 1.2.4.1 14-Jan-2013  matt file memset_arm.S was added on branch matt-nb6-plus on 2013-02-07 07:05:59 +0000
 1.2.2.2 23-Jan-2013  yamt sync with head
 1.2.2.1 14-Jan-2013  yamt file memset_arm.S was added on branch yamt-pagecache on 2013-01-23 00:04:06 +0000
 1.2 19-Jan-2023  mlelstv Use unsigned comparisons for pointers and size_t values.
 1.1 08-Jan-2013  matt branches: 1.1.2; 1.1.4; 1.1.8; 1.1.38; 1.1.52;
Add a simplier version of memset which is less than 1/2 the size of the
current one. On a Cortex-A9, this is about 15%-30% faster than the current
libc version. This is not a trivial implementation since that was an order
magnitude slower than the existing libc version.
 1.1.52.1 20-Jul-2024  martin Pull up following revision(s) (requested by rin in ticket #745):

common/lib/libc/arch/arm/string/memcpy_arm.S: revision 1.6
common/lib/libc/arch/arm/string/memcpy_arm.S: revision 1.7
common/lib/libc/arch/arm/string/memcpy_xscale.S: revision 1.6
common/lib/libc/arch/arm/string/memcpy_neon.S: revision 1.2
common/lib/libc/arch/arm/string/memset_naive.S: revision 1.2
common/lib/libc/arch/arm/string/memmove.S: revision 1.11
common/lib/libc/arch/arm/string/strlen_neon.S: revision 1.4
common/lib/libc/arch/arm/string/memset.S: revision 1.9

Use unsigned comparisons for pointers and size_t values.

Fix two signed comparisons that were missed in the last patch.

Found be rillig@
 1.1.38.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.1.38.1 21-Apr-2020  martin Sync with HEAD
 1.1.8.2 25-Feb-2013  tls resync with head
 1.1.8.1 08-Jan-2013  tls file memset_naive.S was added on branch tls-maxphys on 2013-02-25 00:23:56 +0000
 1.1.4.2 07-Feb-2013  matt Sync with HEAD
 1.1.4.1 08-Jan-2013  matt file memset_naive.S was added on branch matt-nb6-plus on 2013-02-07 07:05:59 +0000
 1.1.2.2 23-Jan-2013  yamt sync with head
 1.1.2.1 08-Jan-2013  yamt file memset_naive.S was added on branch yamt-pagecache on 2013-01-23 00:04:06 +0000
 1.1 13-Dec-2012  matt branches: 1.1.2; 1.1.4; 1.1.8; 1.1.38;
Add a NEON(only) implementation of memset.
This is a work in progress.
 1.1.38.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.1.38.1 21-Apr-2020  martin Sync with HEAD
 1.1.8.2 25-Feb-2013  tls resync with head
 1.1.8.1 13-Dec-2012  tls file memset_neon.S was added on branch tls-maxphys on 2013-02-25 00:23:56 +0000
 1.1.4.2 07-Feb-2013  matt Sync with HEAD
 1.1.4.1 13-Dec-2012  matt file memset_neon.S was added on branch matt-nb6-plus on 2013-02-07 07:06:00 +0000
 1.1.2.2 16-Jan-2013  yamt sync with (a bit old) head
 1.1.2.1 13-Dec-2012  yamt file memset_neon.S was added on branch yamt-pagecache on 2013-01-16 05:25:53 +0000
 1.1 23-Jan-2013  matt branches: 1.1.2; 1.1.6; 1.1.14; 1.1.38;
Switch to using ARM assembly versions of strcat, strchr, strrchr.
 1.1.38.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.1.38.1 21-Apr-2020  martin Sync with HEAD
 1.1.14.2 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.1.14.1 23-Jan-2013  yamt file strcat.S was added on branch yamt-pagecache on 2014-05-22 11:26:28 +0000
 1.1.6.2 25-Feb-2013  tls resync with head
 1.1.6.1 23-Jan-2013  tls file strcat.S was added on branch tls-maxphys on 2013-02-25 00:23:57 +0000
 1.1.2.2 07-Feb-2013  matt Sync with HEAD
 1.1.2.1 23-Jan-2013  matt file strcat.S was added on branch matt-nb6-plus on 2013-02-07 07:06:00 +0000
 1.6 06-May-2014  joerg branches: 1.6.24;
Make EHABI optional.
 1.5 05-Sep-2013  matt branches: 1.5.2;
Use __ARM_EABI__ and new __UNWIND_TABLES__ to decide when to use .cfi ops
 1.4 22-Aug-2013  matt Don't include .cfi info if _KERNEL || _STANDALONE
 1.3 19-Aug-2013  matt Add .cfi ops if EABI.
Thumbify.
 1.2 15-Jan-2013  matt branches: 1.2.2; 1.2.4; 1.2.8;
Add missing ! on str
 1.1 14-Jan-2013  matt Add a native version of strcat which uses the optimized strlen and strcpy
routines.
 1.2.8.3 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.2.8.2 25-Feb-2013  tls resync with head
 1.2.8.1 15-Jan-2013  tls file strcat_arm.S was added on branch tls-maxphys on 2013-02-25 00:23:57 +0000
 1.2.4.2 07-Feb-2013  matt Sync with HEAD
 1.2.4.1 15-Jan-2013  matt file strcat_arm.S was added on branch matt-nb6-plus on 2013-02-07 07:06:00 +0000
 1.2.2.3 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.2.2.2 23-Jan-2013  yamt sync with head
 1.2.2.1 15-Jan-2013  yamt file strcat_arm.S was added on branch yamt-pagecache on 2013-01-23 00:04:06 +0000
 1.5.2.1 10-Aug-2014  tls Rebase.
 1.6.24.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.6.24.1 21-Apr-2020  martin Sync with HEAD
 1.3 19-Aug-2013  matt branches: 1.3.26;
cbnz/cbz can not branch backwards so nuke 'em.
Use the same register usage in strlen as in strnlen
 1.2 19-Aug-2013  matt Thumbify
 1.1 14-Jan-2013  matt branches: 1.1.2; 1.1.4; 1.1.8;
Add a simple version of strcat.
 1.1.8.3 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.1.8.2 25-Feb-2013  tls resync with head
 1.1.8.1 14-Jan-2013  tls file strcat_naive.S was added on branch tls-maxphys on 2013-02-25 00:23:57 +0000
 1.1.4.2 07-Feb-2013  matt Sync with HEAD
 1.1.4.1 14-Jan-2013  matt file strcat_naive.S was added on branch matt-nb6-plus on 2013-02-07 07:06:00 +0000
 1.1.2.3 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.1.2.2 23-Jan-2013  yamt sync with head
 1.1.2.1 14-Jan-2013  yamt file strcat_naive.S was added on branch yamt-pagecache on 2013-01-23 00:04:06 +0000
 1.3.26.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.3.26.1 21-Apr-2020  martin Sync with HEAD
 1.4 20-Aug-2013  matt branches: 1.4.4; 1.4.28;
Use the arm versions of strlen/strchr/strrchr if compiling thumb2
 1.3 19-Aug-2013  matt For Thumb, use naive version
 1.2 23-Jan-2013  matt branches: 1.2.2; 1.2.6;
Add index/rindex strong aiases.
 1.1 23-Jan-2013  matt Switch to using ARM assembly versions of strcat, strchr, strrchr.
 1.2.6.3 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.2.6.2 25-Feb-2013  tls resync with head
 1.2.6.1 23-Jan-2013  tls file strchr.S was added on branch tls-maxphys on 2013-02-25 00:23:57 +0000
 1.2.2.2 07-Feb-2013  matt Sync with HEAD
 1.2.2.1 23-Jan-2013  matt file strchr.S was added on branch matt-nb6-plus on 2013-02-07 07:06:00 +0000
 1.4.28.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.4.28.1 21-Apr-2020  martin Sync with HEAD
 1.4.4.2 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.4.4.1 20-Aug-2013  yamt file strchr.S was added on branch yamt-pagecache on 2014-05-22 11:26:28 +0000
 1.8 19-Aug-2013  matt branches: 1.8.26;
Add two thumb2 bits.
 1.7 19-Aug-2013  matt Missing one teq -> cmp
 1.6 19-Aug-2013  matt Swap use of r1 and ip
teq -> cmp.
add s to few instructions
(thumbify part 1)
 1.5 08-Feb-2013  matt branches: 1.5.4;
Fix corner cases when searching for NUL.
 1.4 07-Feb-2013  matt branches: 1.4.2;
orrne wants 3 registers
 1.3 26-Jan-2013  matt Fix bug in detecting EOS/match on armv6+.
 1.2 15-Jan-2013  matt branches: 1.2.2;
Fix case when searching for NUL.
 1.1 15-Jan-2013  matt Add an ARM optimized version of strchr.
 1.2.2.3 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.2.2.2 23-Jan-2013  yamt sync with head
 1.2.2.1 15-Jan-2013  yamt file strchr_arm.S was added on branch yamt-pagecache on 2013-01-23 00:04:06 +0000
 1.4.2.3 08-Feb-2013  matt Sync with HEAD.
 1.4.2.2 07-Feb-2013  matt Sync with HEAD
 1.4.2.1 07-Feb-2013  matt file strchr_arm.S was added on branch matt-nb6-plus on 2013-02-07 07:06:00 +0000
 1.5.4.3 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.5.4.2 25-Feb-2013  tls resync with head
 1.5.4.1 08-Feb-2013  tls file strchr_arm.S was added on branch tls-maxphys on 2013-02-25 00:23:57 +0000
 1.8.26.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.8.26.1 21-Apr-2020  martin Sync with HEAD
 1.4 19-Aug-2013  matt branches: 1.4.26;
cbnz/cbz can not branch backwards so nuke 'em.
Use the same register usage in strlen as in strnlen
 1.3 19-Aug-2013  matt Thumbify
 1.2 19-Aug-2013  matt ip -> r2
teq -> cmp
 1.1 15-Jan-2013  matt branches: 1.1.2; 1.1.4; 1.1.8;
Add simple/small versions of strchr/strrchr for ARM.
 1.1.8.3 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.1.8.2 25-Feb-2013  tls resync with head
 1.1.8.1 15-Jan-2013  tls file strchr_naive.S was added on branch tls-maxphys on 2013-02-25 00:23:57 +0000
 1.1.4.2 07-Feb-2013  matt Sync with HEAD
 1.1.4.1 15-Jan-2013  matt file strchr_naive.S was added on branch matt-nb6-plus on 2013-02-07 07:06:00 +0000
 1.1.2.3 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.1.2.2 23-Jan-2013  yamt sync with head
 1.1.2.1 15-Jan-2013  yamt file strchr_naive.S was added on branch yamt-pagecache on 2013-01-23 00:04:06 +0000
 1.4.26.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.4.26.1 21-Apr-2020  martin Sync with HEAD
 1.2 19-Aug-2013  matt branches: 1.2.26;
Thumbify
 1.1 20-Dec-2005  christos branches: 1.1.18; 1.1.50; 1.1.56;
Merge libkern + libc common files. As requested by core.
 1.1.56.1 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.1.50.1 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.1.18.1 28-Aug-2007  matt Add thumb versions (or force to arm32 mode when in thumb mode).
 1.2.26.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.2.26.1 21-Apr-2020  martin Sync with HEAD
 1.5 20-Aug-2013  matt branches: 1.5.26;
If compiling standalone with Thumb, use the thumb version instead of the
naive version.
 1.4 10-Jan-2013  matt branches: 1.4.2; 1.4.4; 1.4.8;
use #if defined(xxx) instead of ifdef
 1.3 08-Jan-2013  matt Depending on _STANDALONE include the "naive" version or the normal arm version.
 1.2 02-Jan-2013  matt Deal with _LIBC (aliases, etc).
Add missing #endif.
 1.1 02-Jan-2013  matt Add an assembly version of strcpy/strncpy/strlcpy.
(they all use a common source with defines to determine which to build).
 1.4.8.3 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.4.8.2 25-Feb-2013  tls resync with head
 1.4.8.1 10-Jan-2013  tls file strcpy.S was added on branch tls-maxphys on 2013-02-25 00:23:57 +0000
 1.4.4.2 07-Feb-2013  matt Sync with HEAD
 1.4.4.1 10-Jan-2013  matt file strcpy.S was added on branch matt-nb6-plus on 2013-02-07 07:06:01 +0000
 1.4.2.3 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.4.2.2 23-Jan-2013  yamt sync with head
 1.4.2.1 10-Jan-2013  yamt file strcpy.S was added on branch yamt-pagecache on 2013-01-23 00:04:06 +0000
 1.5.26.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.5.26.1 21-Apr-2020  martin Sync with HEAD
 1.7 08-Feb-2024  andvar fix misplaced or missing "e" in words with "ment" ending (argument, implement,
increment, decrement, alignment), in comments, documentation, log messages.
 1.6 14-Jan-2017  christos branches: 1.6.14;
fix rump
 1.5 14-Jan-2017  christos fix weak symbols. More work needs to be done for memcpy/strlen etc.
 1.4 13-Jan-2017  christos No need to include namespace.h; no other assembly code does.
 1.3 11-Aug-2013  matt branches: 1.3.8;
Switch to unified syntax
use RET/RETc
use push/pop
 1.2 10-Jan-2013  matt branches: 1.2.2; 1.2.4; 1.2.8;
Fix a typo in strlcpy which caused to not deal with NULs predecing the
string properly.
 1.1 08-Jan-2013  matt Rename strlen.S and strcpy.S to strlen_arm.S and strcpy_arm.S
 1.2.8.3 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.2.8.2 25-Feb-2013  tls resync with head
 1.2.8.1 10-Jan-2013  tls file strcpy_arm.S was added on branch tls-maxphys on 2013-02-25 00:23:58 +0000
 1.2.4.2 07-Feb-2013  matt Sync with HEAD
 1.2.4.1 10-Jan-2013  matt file strcpy_arm.S was added on branch matt-nb6-plus on 2013-02-07 07:06:01 +0000
 1.2.2.3 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.2.2.2 23-Jan-2013  yamt sync with head
 1.2.2.1 10-Jan-2013  yamt file strcpy_arm.S was added on branch yamt-pagecache on 2013-01-23 00:04:06 +0000
 1.3.8.1 20-Mar-2017  pgoyette Sync with HEAD
 1.6.14.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.6.14.1 21-Apr-2020  martin Sync with HEAD
 1.8 14-Jan-2017  christos branches: 1.8.14;
fix standalone
 1.7 14-Jan-2017  christos fix standalone
 1.6 14-Jan-2017  christos fix rump
 1.5 13-Jan-2017  christos No need to include namespace.h; no other assembly code does.
 1.4 20-Aug-2013  matt branches: 1.4.8;
write of final NUL in strlcpy doesn't need to be post-incremented
 1.3 11-Aug-2013  matt Convert some more conditional instructions to unified syntax
 1.2 10-Jan-2013  matt branches: 1.2.2; 1.2.4; 1.2.8;
Add weak alias for strlcpy
 1.1 08-Jan-2013  matt Add simple/small versions of the str* functions. Suitable for libsa, etc.
 1.2.8.3 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.2.8.2 25-Feb-2013  tls resync with head
 1.2.8.1 10-Jan-2013  tls file strcpy_naive.S was added on branch tls-maxphys on 2013-02-25 00:23:58 +0000
 1.2.4.2 07-Feb-2013  matt Sync with HEAD
 1.2.4.1 10-Jan-2013  matt file strcpy_naive.S was added on branch matt-nb6-plus on 2013-02-07 07:06:01 +0000
 1.2.2.3 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.2.2.2 23-Jan-2013  yamt sync with head
 1.2.2.1 10-Jan-2013  yamt file strcpy_naive.S was added on branch yamt-pagecache on 2013-01-23 00:04:06 +0000
 1.4.8.1 20-Mar-2017  pgoyette Sync with HEAD
 1.8.14.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.8.14.1 21-Apr-2020  martin Sync with HEAD
 1.3 14-Jan-2017  christos branches: 1.3.14;
fix weak symbols. More work needs to be done for memcpy/strlen etc.
 1.2 13-Jan-2017  christos No need to include namespace.h; no other assembly code does.
 1.1 20-Aug-2013  matt branches: 1.1.4; 1.1.8; 1.1.12;
Thumb versions of strcpy/strlcpy/strncpy
 1.1.12.1 20-Mar-2017  pgoyette Sync with HEAD
 1.1.8.2 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.1.8.1 20-Aug-2013  tls file strcpy_thumb.S was added on branch tls-maxphys on 2014-08-19 23:45:12 +0000
 1.1.4.2 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.1.4.1 20-Aug-2013  yamt file strcpy_thumb.S was added on branch yamt-pagecache on 2014-05-22 11:26:28 +0000
 1.3.14.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.3.14.1 21-Apr-2020  martin Sync with HEAD
 1.4 14-Jan-2017  christos branches: 1.4.14;
fix weak symbols. More work needs to be done for memcpy/strlen etc.
 1.3 13-Jan-2017  christos No need to include namespace.h; no other assembly code does.
 1.2 20-Aug-2013  matt branches: 1.2.4; 1.2.10;
strlcat_arm.S is smaller than strlcat_naive.S so always use it.
 1.1 23-Jan-2013  matt branches: 1.1.2; 1.1.6;
Enable ARM assembly versions of strlcat and strnlen.
 1.1.6.3 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.1.6.2 25-Feb-2013  tls resync with head
 1.1.6.1 23-Jan-2013  tls file strlcat.S was added on branch tls-maxphys on 2013-02-25 00:23:58 +0000
 1.1.2.2 07-Feb-2013  matt Sync with HEAD
 1.1.2.1 23-Jan-2013  matt file strlcat.S was added on branch matt-nb6-plus on 2013-02-07 07:06:01 +0000
 1.2.10.1 20-Mar-2017  pgoyette Sync with HEAD
 1.2.4.2 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.2.4.1 20-Aug-2013  yamt file strlcat.S was added on branch yamt-pagecache on 2014-05-22 11:26:28 +0000
 1.4.14.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.4.14.1 21-Apr-2020  martin Sync with HEAD
 1.6 06-May-2014  joerg branches: 1.6.2; 1.6.26;
Make EHABI optional.
 1.5 05-Sep-2013  matt branches: 1.5.2;
Use __ARM_EABI__ and new __UNWIND_TABLES__ to decide when to use .cfi ops
 1.4 22-Aug-2013  matt Don't include .cfi info if _KERNEL || _STANDALONE
 1.3 19-Aug-2013  matt fix cfi_register -> cfi_offset
 1.2 19-Aug-2013  matt Add .cfi for __ARM_EABI__
Thumbify
 1.1 23-Jan-2013  matt branches: 1.1.2; 1.1.6;
Add ARM assembly of strlcat which is implemented using (the ARM optimized)
strnlen and strlcpy.
 1.1.6.3 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.1.6.2 25-Feb-2013  tls resync with head
 1.1.6.1 23-Jan-2013  tls file strlcat_arm.S was added on branch tls-maxphys on 2013-02-25 00:23:58 +0000
 1.1.2.2 07-Feb-2013  matt Sync with HEAD
 1.1.2.1 23-Jan-2013  matt file strlcat_arm.S was added on branch matt-nb6-plus on 2013-02-07 07:06:01 +0000
 1.5.2.1 10-Aug-2014  tls Rebase.
 1.6.26.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.6.26.1 21-Apr-2020  martin Sync with HEAD
 1.6.2.2 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.6.2.1 06-May-2014  yamt file strlcat_arm.S was added on branch yamt-pagecache on 2014-05-22 11:26:28 +0000
 1.3 22-Aug-2013  matt branches: 1.3.26;
Don't include .cfi info if _KERNEL || _STANDALONE
 1.2 11-Aug-2013  matt Convert some more conditional instructions to unified syntax
 1.1 18-Jan-2013  matt branches: 1.1.2; 1.1.4; 1.1.8;
Simple version of strlcat for ARM.
 1.1.8.3 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.1.8.2 25-Feb-2013  tls resync with head
 1.1.8.1 18-Jan-2013  tls file strlcat_naive.S was added on branch tls-maxphys on 2013-02-25 00:23:58 +0000
 1.1.4.2 07-Feb-2013  matt Sync with HEAD
 1.1.4.1 18-Jan-2013  matt file strlcat_naive.S was added on branch matt-nb6-plus on 2013-02-07 07:06:01 +0000
 1.1.2.3 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.1.2.2 23-Jan-2013  yamt sync with head
 1.1.2.1 18-Jan-2013  yamt file strlcat_naive.S was added on branch yamt-pagecache on 2013-01-23 00:04:06 +0000
 1.3.26.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.3.26.1 21-Apr-2020  martin Sync with HEAD
 1.5 20-Aug-2013  matt branches: 1.5.26;
If compiling standalone with Thumb, use the thumb version instead of the
naive version.
 1.4 10-Jan-2013  matt branches: 1.4.2; 1.4.4; 1.4.8;
Back out workaround.
 1.3 10-Jan-2013  matt Use the naive version of strlcpy until the longer one is fixed (it's still
faster than the C version).
 1.2 10-Jan-2013  matt use #if defined(xxx) instead of ifdef
 1.1 02-Jan-2013  matt Add an assembly version of strcpy/strncpy/strlcpy.
(they all use a common source with defines to determine which to build).
 1.4.8.3 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.4.8.2 25-Feb-2013  tls resync with head
 1.4.8.1 10-Jan-2013  tls file strlcpy.S was added on branch tls-maxphys on 2013-02-25 00:23:58 +0000
 1.4.4.2 07-Feb-2013  matt Sync with HEAD
 1.4.4.1 10-Jan-2013  matt file strlcpy.S was added on branch matt-nb6-plus on 2013-02-07 07:06:01 +0000
 1.4.2.3 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.4.2.2 23-Jan-2013  yamt sync with head
 1.4.2.1 10-Jan-2013  yamt file strlcpy.S was added on branch yamt-pagecache on 2013-01-23 00:04:06 +0000
 1.5.26.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.5.26.1 21-Apr-2020  martin Sync with HEAD
 1.4 20-Aug-2013  matt branches: 1.4.26;
Use the arm versions of strlen/strchr/strrchr if compiling thumb2
 1.3 19-Aug-2013  matt For Thumb, use naive version
 1.2 08-Jan-2013  matt branches: 1.2.2; 1.2.4; 1.2.8;
Depending on _STANDALONE include the "naive" version or the normal arm version.
 1.1 02-Jan-2013  matt Rename strlen_armv6.S to strlen.S since this is no longer armv6 dependent.
 1.2.8.3 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.2.8.2 25-Feb-2013  tls resync with head
 1.2.8.1 08-Jan-2013  tls file strlen.S was added on branch tls-maxphys on 2013-02-25 00:23:58 +0000
 1.2.4.2 07-Feb-2013  matt Sync with HEAD
 1.2.4.1 08-Jan-2013  matt file strlen.S was added on branch matt-nb6-plus on 2013-02-07 07:06:01 +0000
 1.2.2.3 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.2.2.2 23-Jan-2013  yamt sync with head
 1.2.2.1 08-Jan-2013  yamt file strlen.S was added on branch yamt-pagecache on 2013-01-23 00:04:06 +0000
 1.4.26.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.4.26.1 21-Apr-2020  martin Sync with HEAD
 1.11 15-Jan-2023  skrll Fix strnlen with a large maxlen argument by using unsigned comparison
conditions - from mlelstv.

I had a similar, but not quite as good patch.
 1.10 03-Dec-2022  skrll branches: 1.10.2;
Fix some comments
 1.9 06-May-2014  joerg branches: 1.9.24; 1.9.26;
Make EHABI optional.
 1.8 05-Sep-2013  matt branches: 1.8.2;
Use __ARM_EABI__ and new __UNWIND_TABLES__ to decide when to use .cfi ops
 1.7 22-Aug-2013  matt Don't include .cfi info if _KERNEL || _STANDALONE
 1.6 20-Aug-2013  matt Add a missing it gt before movgt for thumb
 1.5 20-Aug-2013  matt thumbify (part2)
 1.4 19-Aug-2013  matt For EABI, add .cfi ops
 1.3 23-Jan-2013  matt branches: 1.3.2; 1.3.6;
Add support for strnlen.
 1.2 09-Jan-2013  matt branches: 1.2.2;
Use movw on armv7 to fill uppper halfword.
 1.1 08-Jan-2013  matt Rename strlen.S and strcpy.S to strlen_arm.S and strcpy_arm.S
 1.2.2.3 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.2.2.2 23-Jan-2013  yamt sync with head
 1.2.2.1 09-Jan-2013  yamt file strlen_arm.S was added on branch yamt-pagecache on 2013-01-23 00:04:06 +0000
 1.3.6.3 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.3.6.2 25-Feb-2013  tls resync with head
 1.3.6.1 23-Jan-2013  tls file strlen_arm.S was added on branch tls-maxphys on 2013-02-25 00:23:59 +0000
 1.3.2.2 07-Feb-2013  matt Sync with HEAD
 1.3.2.1 23-Jan-2013  matt file strlen_arm.S was added on branch matt-nb6-plus on 2013-02-07 07:06:02 +0000
 1.8.2.1 10-Aug-2014  tls Rebase.
 1.9.26.1 19-Jan-2023  martin Pull up following revision(s) (requested by skrll in ticket #1567):

common/lib/libc/arch/arm/string/strlen_arm.S: revision 1.11

Fix strnlen with a large maxlen argument by using unsigned comparison
conditions - from mlelstv.

I had a similar, but not quite as good patch.
 1.9.24.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.9.24.1 21-Apr-2020  martin Sync with HEAD
 1.10.2.1 19-Jan-2023  martin Pull up following revision(s) (requested by skrll in ticket #51):

common/lib/libc/arch/arm/string/strlen_arm.S: revision 1.11

Fix strnlen with a large maxlen argument by using unsigned comparison
conditions - from mlelstv.

I had a similar, but not quite as good patch.
 1.4 02-Jan-2013  matt Rename strlen_armv6.S to strlen.S since this is no longer armv6 dependent.
 1.3 31-Dec-2012  matt Make this work on all ARMs but keep the armv6 optimizations. It as fast as
the existing strlen for small string and once strings are 8 bytes or more in
length it starts getting significantly faster. For really long strings,
compared to the existing strlen, this uses about 1/2 of the cycles for the
non-armv6 version and about 1/3 of the cycles for the armv6 version.
 1.2 29-Dec-2012  matt A few slight speedups (remove one instruction from the main loop).
 1.1 28-Dec-2012  matt strlen implementation for armv6 and later. Uses clz and uqadd8 to really
speed the search for NUL. as fast as normal strlen at about a length of
6 or 7 and 2-3 times faster starting around 10.
 1.9 03-Dec-2022  skrll improve a comment
 1.8 19-Aug-2013  matt branches: 1.8.26;
cbnz/cbz can not branch backwards so nuke 'em.
Use the same register usage in strlen as in strnlen
 1.7 19-Aug-2013  matt Thumbify
 1.6 19-Aug-2013  matt Use ip as a temporary
 1.5 19-Aug-2013  matt Change previous use of r2 to r3
 1.4 19-Aug-2013  matt teq -> cmp
ip -> r2
add/sub -> adds/subs
(thumbify part 1)
 1.3 23-Jan-2013  matt branches: 1.3.2; 1.3.6;
Add support for strnlen.
 1.2 08-Jan-2013  pgoyette branches: 1.2.2;
Add missing quote - fix build
 1.1 08-Jan-2013  matt Add simple/small versions of the str* functions. Suitable for libsa, etc.
 1.2.2.3 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.2.2.2 23-Jan-2013  yamt sync with head
 1.2.2.1 08-Jan-2013  yamt file strlen_naive.S was added on branch yamt-pagecache on 2013-01-23 00:04:06 +0000
 1.3.6.3 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.3.6.2 25-Feb-2013  tls resync with head
 1.3.6.1 23-Jan-2013  tls file strlen_naive.S was added on branch tls-maxphys on 2013-02-25 00:23:59 +0000
 1.3.2.2 07-Feb-2013  matt Sync with HEAD
 1.3.2.1 23-Jan-2013  matt file strlen_naive.S was added on branch matt-nb6-plus on 2013-02-07 07:06:02 +0000
 1.8.26.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.8.26.1 21-Apr-2020  martin Sync with HEAD
 1.4 19-Jan-2023  mlelstv Use unsigned comparisons for pointers and size_t values.
 1.3 28-Dec-2012  matt branches: 1.3.2; 1.3.4; 1.3.8; 1.3.38; 1.3.52;
Debug. This becomes faster than the normal strlen at about 80 characters.
 1.2 15-Dec-2012  matt Slighly improved (can deal with all 16 bytes being non-NUL and quickly
proceed to next qword).
 1.1 15-Dec-2012  matt Add a NEON implementation of strlen.
 1.3.52.1 20-Jul-2024  martin Pull up following revision(s) (requested by rin in ticket #745):

common/lib/libc/arch/arm/string/memcpy_arm.S: revision 1.6
common/lib/libc/arch/arm/string/memcpy_arm.S: revision 1.7
common/lib/libc/arch/arm/string/memcpy_xscale.S: revision 1.6
common/lib/libc/arch/arm/string/memcpy_neon.S: revision 1.2
common/lib/libc/arch/arm/string/memset_naive.S: revision 1.2
common/lib/libc/arch/arm/string/memmove.S: revision 1.11
common/lib/libc/arch/arm/string/strlen_neon.S: revision 1.4
common/lib/libc/arch/arm/string/memset.S: revision 1.9

Use unsigned comparisons for pointers and size_t values.

Fix two signed comparisons that were missed in the last patch.

Found be rillig@
 1.3.38.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.3.38.1 21-Apr-2020  martin Sync with HEAD
 1.3.8.2 25-Feb-2013  tls resync with head
 1.3.8.1 28-Dec-2012  tls file strlen_neon.S was added on branch tls-maxphys on 2013-02-25 00:23:59 +0000
 1.3.4.2 07-Feb-2013  matt Sync with HEAD
 1.3.4.1 28-Dec-2012  matt file strlen_neon.S was added on branch matt-nb6-plus on 2013-02-07 07:06:02 +0000
 1.3.2.2 23-Jan-2013  yamt sync with head
 1.3.2.1 28-Dec-2012  yamt file strlen_neon.S was added on branch yamt-pagecache on 2013-01-23 00:04:06 +0000
 1.2 19-Aug-2013  matt branches: 1.2.26;
Thumbify
 1.1 20-Dec-2005  christos branches: 1.1.18; 1.1.50; 1.1.56;
Merge libkern + libc common files. As requested by core.
 1.1.56.1 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.1.50.1 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.1.18.1 28-Aug-2007  matt Add thumb versions (or force to arm32 mode when in thumb mode).
 1.2.26.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.2.26.1 21-Apr-2020  martin Sync with HEAD
 1.3 20-Aug-2013  matt branches: 1.3.26;
If compiling standalone with Thumb, use the thumb version instead of the
naive version.
 1.2 10-Jan-2013  matt branches: 1.2.2; 1.2.4; 1.2.8;
use #if defined(xxx) instead of ifdef
 1.1 02-Jan-2013  matt Add an assembly version of strcpy/strncpy/strlcpy.
(they all use a common source with defines to determine which to build).
 1.2.8.3 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.2.8.2 25-Feb-2013  tls resync with head
 1.2.8.1 10-Jan-2013  tls file strncpy.S was added on branch tls-maxphys on 2013-02-25 00:23:59 +0000
 1.2.4.2 07-Feb-2013  matt Sync with HEAD
 1.2.4.1 10-Jan-2013  matt file strncpy.S was added on branch matt-nb6-plus on 2013-02-07 07:06:02 +0000
 1.2.2.3 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.2.2.2 23-Jan-2013  yamt sync with head
 1.2.2.1 10-Jan-2013  yamt file strncpy.S was added on branch yamt-pagecache on 2013-01-23 00:04:06 +0000
 1.3.26.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.3.26.1 21-Apr-2020  martin Sync with HEAD
 1.3 20-Aug-2013  matt branches: 1.3.4; 1.3.28;
Use the arm versions of strnlen if compiling thumb2
 1.2 19-Aug-2013  matt For Thumb, use naive version
 1.1 23-Jan-2013  matt branches: 1.1.2; 1.1.6;
Enable ARM assembly versions of strlcat and strnlen.
 1.1.6.3 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.1.6.2 25-Feb-2013  tls resync with head
 1.1.6.1 23-Jan-2013  tls file strnlen.S was added on branch tls-maxphys on 2013-02-25 00:23:59 +0000
 1.1.2.2 07-Feb-2013  matt Sync with HEAD
 1.1.2.1 23-Jan-2013  matt file strnlen.S was added on branch matt-nb6-plus on 2013-02-07 07:06:02 +0000
 1.3.28.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.3.28.1 21-Apr-2020  martin Sync with HEAD
 1.3.4.2 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.3.4.1 20-Aug-2013  yamt file strnlen.S was added on branch yamt-pagecache on 2014-05-22 11:26:28 +0000
 1.4 20-Aug-2013  matt branches: 1.4.4; 1.4.28;
Use the arm versions of strlen/strchr/strrchr if compiling thumb2
 1.3 19-Aug-2013  matt For Thumb, use naive version
 1.2 23-Jan-2013  matt branches: 1.2.2; 1.2.6;
Add index/rindex strong aiases.
 1.1 23-Jan-2013  matt Switch to using ARM assembly versions of strcat, strchr, strrchr.
 1.2.6.3 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.2.6.2 25-Feb-2013  tls resync with head
 1.2.6.1 23-Jan-2013  tls file strrchr.S was added on branch tls-maxphys on 2013-02-25 00:23:59 +0000
 1.2.2.2 07-Feb-2013  matt Sync with HEAD
 1.2.2.1 23-Jan-2013  matt file strrchr.S was added on branch matt-nb6-plus on 2013-02-07 07:06:02 +0000
 1.4.28.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.4.28.1 21-Apr-2020  martin Sync with HEAD
 1.4.4.2 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.4.4.1 20-Aug-2013  yamt file strrchr.S was added on branch yamt-pagecache on 2014-05-22 11:26:28 +0000
 1.6 25-Aug-2013  matt branches: 1.6.26;
move a misplaced #endif
 1.5 20-Aug-2013  matt swap r1 & ip
use adds, eors, etc.
teq -> cmp
 1.4 11-Aug-2013  matt Switch to unified syntax
use RET/RETc
use push/pop
 1.3 08-Feb-2013  matt branches: 1.3.4;
Fix corner cases when searching for NUL.
 1.2 28-Jan-2013  matt branches: 1.2.2;
Deal with an end-of-string condition properly.
 1.1 15-Jan-2013  matt branches: 1.1.2;
Add ARM optimized version of strrchr.
 1.1.2.3 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.1.2.2 23-Jan-2013  yamt sync with head
 1.1.2.1 15-Jan-2013  yamt file strrchr_arm.S was added on branch yamt-pagecache on 2013-01-23 00:04:06 +0000
 1.2.2.3 08-Feb-2013  matt Sync with HEAD.
 1.2.2.2 07-Feb-2013  matt Sync with HEAD
 1.2.2.1 28-Jan-2013  matt file strrchr_arm.S was added on branch matt-nb6-plus on 2013-02-07 07:06:02 +0000
 1.3.4.3 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.3.4.2 25-Feb-2013  tls resync with head
 1.3.4.1 08-Feb-2013  tls file strrchr_arm.S was added on branch tls-maxphys on 2013-02-25 00:23:59 +0000
 1.6.26.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.6.26.1 21-Apr-2020  martin Sync with HEAD
 1.4 19-Aug-2013  matt branches: 1.4.26;
cbnz/cbz can not branch backwards so nuke 'em.
Use the same register usage in strlen as in strnlen
 1.3 19-Aug-2013  matt Thumbify
 1.2 19-Aug-2013  matt ip -> r2
teq -> cmp
 1.1 15-Jan-2013  matt branches: 1.1.2; 1.1.4; 1.1.8;
Add simple/small versions of strchr/strrchr for ARM.
 1.1.8.3 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.1.8.2 25-Feb-2013  tls resync with head
 1.1.8.1 15-Jan-2013  tls file strrchr_naive.S was added on branch tls-maxphys on 2013-02-25 00:23:59 +0000
 1.1.4.2 07-Feb-2013  matt Sync with HEAD
 1.1.4.1 15-Jan-2013  matt file strrchr_naive.S was added on branch matt-nb6-plus on 2013-02-07 07:06:02 +0000
 1.1.2.3 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.1.2.2 23-Jan-2013  yamt sync with head
 1.1.2.1 15-Jan-2013  yamt file strrchr_naive.S was added on branch yamt-pagecache on 2013-01-23 00:04:06 +0000
 1.4.26.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.4.26.1 21-Apr-2020  martin Sync with HEAD
 1.16 29-Oct-2023  skrll Define __HAVE_ATOMIC_CAS_64_UP and provide __sync_val_compare_and_swap_8

The new santizer code in gcc12 needs this.
 1.15 30-Mar-2023  riastradh libc: Define __atomic_is_lock_free.

Limited to architectures where it is actually needed by gcc for any
calls to stdatomic.h atomic_is_lock_free for now.

We should also add it to other architectures too, along with lockful
atomic r/m/w operations for sizes that can't be handled natively, but
that's a lot more work. It is also necessary for -fno-inline-atomics
but we're missing a lot of other symbols for that too, to be fixed.
For now, this should enable the OpenSSL build to complete on these
architectures again after I reverted a local change.

XXX pullup-10
 1.14 28-Feb-2019  isaki branches: 1.14.14;
Add missing atomic_and_{8,16}_nv_cas.c for __sync_and_and_fetch_{1,2}.
XXX why is not only atomic_and_* symmetric unlike the others?
(in common/lib/libc/atomic/)
 1.13 14-Oct-2014  martin branches: 1.13.16;
Provide C++ 2011 <atomic> support functions for hppa and arm.
 1.12 22-Feb-2014  skrll branches: 1.12.4;
Add the missing __sync_* ops to libc
 1.11 22-Feb-2014  skrll Another layout change
 1.10 22-Feb-2014  skrll Layout changes - no functional change.
 1.9 29-Jan-2014  skrll Provide _atomic_cas_8_up and _atomic_cas_16_up functions
 1.8 17-Jan-2011  skrll branches: 1.8.6; 1.8.12;
Tweak layout - no functional change.
 1.7 06-Jan-2009  mjf The .if conditional crosses a line boundary, so add a backslash.

This fixes the kernel build for hppa.
 1.6 04-Jan-2009  pooka allow inclusion of atomic ops in librump
 1.5 29-Sep-2008  ad Allow atomic ops to be built as part of libpthread.
 1.4 11-Aug-2008  skrll Provide assembly _atomic_cas_up() as the compiler cannot be trusted to
generate fully restartable code sequences.
 1.3 11-Feb-2008  ad branches: 1.3.6;
Only build atomic ops for libkern/libc.
 1.2 10-Feb-2008  ad Enable the atomic ops in userspace.
 1.1 29-Nov-2007  ad branches: 1.1.4;
Atomic ops for hppa.
 1.1.4.3 23-Mar-2008  matt sync with HEAD
 1.1.4.2 09-Jan-2008  matt sync with HEAD
 1.1.4.1 29-Nov-2007  matt file Makefile.inc was added on branch matt-armv6 on 2008-01-09 01:20:52 +0000
 1.3.6.1 18-Sep-2008  wrstuden Sync with wrstuden-revivesa-base-2.
 1.8.12.1 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.8.6.1 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.12.4.1 12-Nov-2014  snj Pull up following revision(s) (requested by martin in ticket #218):
common/lib/libc/arch/arm/atomic/Makefile.inc: revision 1.24-1.26
common/lib/libc/arch/hppa/atomic/Makefile.inc: revision 1.13
common/lib/libc/arch/mips/atomic/Makefile.inc: revision 1.13
common/lib/libc/arch/sh3/atomic/Makefile.inc: revision 1.7
common/lib/libc/arch/sparc/atomic/Makefile.inc: revision 1.18
common/lib/libc/arch/vax/atomic/Makefile.inc: revision 1.7
common/lib/libc/atomic/atomic_and_16_nv_cas.c: revision 1.2
common/lib/libc/atomic/atomic_and_8_nv_cas.c: revision 1.2
common/lib/libc/atomic/atomic_c11_compare_exchange_cas_16.c: revision 1.1-1.2
common/lib/libc/atomic/atomic_c11_compare_exchange_cas_32.c: revision 1.1-1.2
common/lib/libc/atomic/atomic_c11_compare_exchange_cas_8.c: revision 1.1-1.2
common/lib/libc/atomic/atomic_cas_by_cas32.c: revision 1.4
common/lib/libc/atomic/atomic_op_namespace.h: revision 1.7
Add __sync_val_compare_and_swap_{1,2} aliases for _atomic_cas_{8,16}
--
Provide __atomic_compare_exchange_N (as needed for the C11 2011 <atomic>
ops) via the corresponding CAS.
--
Hook __atomic_compare_exchange_N into vax libc.
--
Provide __sync_and_and_fetch_2 and __sync_and_and_fetch_1 for pre-ARMv6,
they are needed for the C++ 2011 <atomic> stuff.
--
Add C++ 2011 <atomic> support functions.
--
Move the and_{16,8}_nv sources into the right (libc only) block.
--
Provide <atomic> C++ 2011 support functions for mips and sh3.
--
Provide C++ 2011 <atomic> support functions for hppa and arm.
--
Provide prototypes to fix build with clang.
 1.13.16.2 21-Apr-2020  martin Sync with HEAD
 1.13.16.1 10-Jun-2019  christos Sync with HEAD
 1.14.14.1 31-Jul-2023  martin Pull up following revision(s) (requested by riastradh in ticket #275):

common/lib/libc/arch/sparc/atomic/Makefile.inc: revision 1.24
common/lib/libc/arch/m68k/atomic/Makefile.inc: revision 1.16
common/lib/libc/arch/mips/atomic/Makefile.inc: revision 1.16
common/lib/libc/arch/hppa/atomic/Makefile.inc: revision 1.15
common/lib/libc/arch/vax/atomic/Makefile.inc: revision 1.9
common/lib/libc/atomic/atomic_is_lock_free.c: revision 1.1
common/lib/libc/arch/sh3/atomic/Makefile.inc: revision 1.9

libc: Define __atomic_is_lock_free.

Limited to architectures where it is actually needed by gcc for any
calls to stdatomic.h atomic_is_lock_free for now.

We should also add it to other architectures too, along with lockful
atomic r/m/w operations for sizes that can't be handled natively, but
that's a lot more work. It is also necessary for -fno-inline-atomics
but we're missing a lot of other symbols for that too, to be fixed.

For now, this should enable the OpenSSL build to complete on these
architectures again after I reverted a local change.
 1.4 29-Jan-2014  skrll branches: 1.4.26;
Copyright
 1.3 29-Jan-2014  skrll Provide _atomic_cas_8_up and _atomic_cas_16_up functions
 1.2 16-Jan-2011  skrll branches: 1.2.6; 1.2.12;
Fix RCSId.
 1.1 11-Aug-2008  skrll branches: 1.1.2;
Provide assembly _atomic_cas_up() as the compiler cannot be trusted to
generate fully restartable code sequences.
 1.1.2.2 18-Sep-2008  wrstuden Sync with wrstuden-revivesa-base-2.
 1.1.2.1 11-Aug-2008  wrstuden file atomic_cas_up.S was added on branch wrstuden-revivesa on 2008-09-18 04:54:18 +0000
 1.2.12.1 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.2.6.1 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.4.26.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.4.26.1 21-Apr-2020  martin Sync with HEAD
 1.3 09-Apr-2022  riastradh Introduce membar_acquire/release. Deprecate membar_enter/exit.

The names membar_enter/exit were unclear, and the documentation of
membar_enter has disagreed with the implementations on sparc,
powerpc, and even x86(!) for the entire time it has been in NetBSD.

The terms `acquire' and `release' are ubiquitous in the literature
today, and have been adopted in the C and C++ standards to mean
load-before-load/store and load/store-before-store, respectively,
which are exactly the orderings required by acquiring and releasing a
mutex, as well as other useful applications like decrementing a
reference count and then freeing the underlying object if it went to
zero.

Originally I proposed changing one word in the documentation for
membar_enter to make it load-before-load/store instead of
store-before-load/store, i.e., to make it an acquire barrier. I
proposed this on the grounds that

(a) all implementations guarantee load-before-load/store,
(b) some implementations fail to guarantee store-before-load/store,
and
(c) all uses in-tree assume load-before-load/store.

I verified parts (a) and (b) (except, for (a), powerpc didn't even
guarantee load-before-load/store -- isync isn't necessarily enough;
need lwsync in general -- but it _almost_ did, and it certainly didn't
guarantee store-before-load/store).

Part (c) might not be correct, however: under the mistaken assumption
that atomic-r/m/w then membar-w/rw is equivalent to atomic-r/m/w then
membar-r/rw, I only audited the cases of membar_enter that _aren't_
immediately after an atomic-r/m/w. All of those cases assume
load-before-load/store. But my assumption was wrong -- there are
cases of atomic-r/m/w then membar-w/rw that would be broken by
changing to atomic-r/m/w then membar-r/rw:

https://mail-index.netbsd.org/tech-kern/2022/03/29/msg028044.html

Furthermore, the name membar_enter has been adopted in other places
like OpenBSD where it actually does follow the documentation and
guarantee store-before-load/store, even if that order is not useful.
So the name membar_enter currently lives in a bad place where it
means either of two things -- r/rw or w/rw.

With this change, we deprecate membar_enter/exit, introduce
membar_acquire/release as better names for the useful pair (r/rw and
rw/w), and make sure the implementation of membar_enter guarantees
both what was documented _and_ what was implemented, making it an
alias for membar_sync.

While here, rework all of the membar_* definitions and aliases. The
new logic follows a rule to make it easier to audit:

membar_X is defined as an alias for membar_Y iff membar_X is
guaranteed by membar_Y.

The `no stronger than' relation is (the transitive closure of):

- membar_consumer (r/r) is guaranteed by membar_acquire (r/rw)
- membar_producer (w/w) is guaranteed by membar_release (rw/w)
- membar_acquire (r/rw) is guaranteed by membar_sync (rw/rw)
- membar_release (rw/w) is guaranteed by membar_sync (rw/rw)

And, for the deprecated membars:

- membar_enter (whether r/rw, w/rw, or rw/rw) is guaranteed by
membar_sync (rw/rw)
- membar_exit (rw/w) is guaranteed by membar_release (rw/w)

(membar_exit is identical to membar_release, but the name is
deprecated.)

Finally, while here, annotate some of the instructions with their
semantics. For powerpc, leave an essay with citations on the
unfortunate but -- as far as I can tell -- necessary decision to use
lwsync, not isync, for membar_acquire and membar_consumer.

Also add membar(3) and atomic(3) man page links.
 1.2 06-Apr-2022  riastradh Nix trailing whitespace in files of membars, atomics, and lock stubs.

Will be touching many of these files soon for functional changes.

No functional change intended.
 1.1 17-Jan-2011  skrll branches: 1.1.48;
Add an membar_ops.S - not used currently.
 1.1.48.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.1.48.1 21-Apr-2020  martin Sync with HEAD
 1.8 18-Feb-2014  martin branches: 1.8.26;
Provide most missing __sync_*64 primitives for i386
 1.7 04-Jan-2009  pooka branches: 1.7.8; 1.7.14;
allow inclusion of atomic ops in librump
 1.6 29-Sep-2008  ad Allow atomic ops to be built as part of libpthread.
 1.5 11-Feb-2008  ad Only build atomic ops for libkern/libc.
 1.4 10-Feb-2008  ad Enable the atomic ops in userspace.
 1.3 20-Dec-2007  ad branches: 1.3.2;
64-bit atomic ops for i386.
 1.2 28-Nov-2007  ad x86 atomic ops.
 1.1 17-Apr-2007  thorpej branches: 1.1.2;
file Makefile.inc was initially added on branch thorpej-atomic.
 1.1.2.1 17-Apr-2007  thorpej Add build glue for i386 atomic ops.
 1.3.2.3 23-Mar-2008  matt sync with HEAD
 1.3.2.2 09-Jan-2008  matt sync with HEAD
 1.3.2.1 20-Dec-2007  matt file Makefile.inc was added on branch matt-armv6 on 2008-01-09 01:20:52 +0000
 1.7.14.1 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.7.8.1 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.8.26.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.8.26.1 21-Apr-2020  martin Sync with HEAD
 1.38 06-Sep-2025  riastradh paravirt_membar_sync(9): New memory barrier.

For use in paravirtualized drivers which require store-before-load
ordering -- irrespective of whether the kernel is built for a single
processor, or whether the (virtual) machine is booted with a single
processor.

This is even required on architectures that don't even have a
store-before-load ordering barrier, like m68k; adding, e.g., a virtio
bus is _as if_ the architecture has been extended with relaxed memory
ordering when talking with that new bus. Such architectures need
some way to request the hypervisor enforce that ordering -- on m68k,
that's done by issuing a CASL instruction, which qemu maps to an
atomic r/m/w with sequential consistency ordering in the host.

PR kern/59618: occasional virtio block device lock ups/hangs
 1.37 16-Jul-2024  riastradh xen: Don't hotpatch away LOCK prefix in xen_mb, even on UP boots.

Both xen_mb and membar_sync are designed to provide store-before-load
ordering, but xen_mb has to provide it in synchronizing guest with
hypervisor, while membar_sync only has to provide it in synchronizing
one (guest) CPU with another (guest) CPU.

It is safe to hotpatch away the LOCK prefix in membar_sync on a
uniprocessor boot because membar_sync is only designed to coordinate
between normal memory on multiple CPUs, and is never necessary when
there's only one CPU involved.

But xen_mb is used to coordinate between the guest and the `device'
implemented by a hypervisor, which might be running on another
_physical_ CPU even if the NetBSD guest only sees one `CPU', i.e.,
one _virtual_ CPU. So even on `uniprocessor' boots, xen_mb must
still issue an instruction with store-before-load ordering on
multiprocessor systems, such as a LOCK ADD (or MFENCE, but MFENCE is
costlier for no benefit here).

No need to change xen_wmb (release ordering, load/store-before-store)
or xen_rmb (acquire ordering, load-before-load/store) because every
x86 store is a store-release and every x86 load is a load-acquire,
even on multiprocessor systems, so there's no hotpatching involved
anyway.

PR kern/57199
 1.36 30-Jul-2022  riastradh branches: 1.36.2; 1.36.8;
x86: Eliminate mfence hotpatch for membar_sync.

The more-compatible LOCK ADD $0,-N(%rsp) turns out to be cheaper
than MFENCE anyway. Let's save some space and maintenance and rip
out the hotpatching for it.
 1.35 09-Apr-2022  riastradh Introduce membar_acquire/release. Deprecate membar_enter/exit.

The names membar_enter/exit were unclear, and the documentation of
membar_enter has disagreed with the implementations on sparc,
powerpc, and even x86(!) for the entire time it has been in NetBSD.

The terms `acquire' and `release' are ubiquitous in the literature
today, and have been adopted in the C and C++ standards to mean
load-before-load/store and load/store-before-store, respectively,
which are exactly the orderings required by acquiring and releasing a
mutex, as well as other useful applications like decrementing a
reference count and then freeing the underlying object if it went to
zero.

Originally I proposed changing one word in the documentation for
membar_enter to make it load-before-load/store instead of
store-before-load/store, i.e., to make it an acquire barrier. I
proposed this on the grounds that

(a) all implementations guarantee load-before-load/store,
(b) some implementations fail to guarantee store-before-load/store,
and
(c) all uses in-tree assume load-before-load/store.

I verified parts (a) and (b) (except, for (a), powerpc didn't even
guarantee load-before-load/store -- isync isn't necessarily enough;
need lwsync in general -- but it _almost_ did, and it certainly didn't
guarantee store-before-load/store).

Part (c) might not be correct, however: under the mistaken assumption
that atomic-r/m/w then membar-w/rw is equivalent to atomic-r/m/w then
membar-r/rw, I only audited the cases of membar_enter that _aren't_
immediately after an atomic-r/m/w. All of those cases assume
load-before-load/store. But my assumption was wrong -- there are
cases of atomic-r/m/w then membar-w/rw that would be broken by
changing to atomic-r/m/w then membar-r/rw:

https://mail-index.netbsd.org/tech-kern/2022/03/29/msg028044.html

Furthermore, the name membar_enter has been adopted in other places
like OpenBSD where it actually does follow the documentation and
guarantee store-before-load/store, even if that order is not useful.
So the name membar_enter currently lives in a bad place where it
means either of two things -- r/rw or w/rw.

With this change, we deprecate membar_enter/exit, introduce
membar_acquire/release as better names for the useful pair (r/rw and
rw/w), and make sure the implementation of membar_enter guarantees
both what was documented _and_ what was implemented, making it an
alias for membar_sync.

While here, rework all of the membar_* definitions and aliases. The
new logic follows a rule to make it easier to audit:

membar_X is defined as an alias for membar_Y iff membar_X is
guaranteed by membar_Y.

The `no stronger than' relation is (the transitive closure of):

- membar_consumer (r/r) is guaranteed by membar_acquire (r/rw)
- membar_producer (w/w) is guaranteed by membar_release (rw/w)
- membar_acquire (r/rw) is guaranteed by membar_sync (rw/rw)
- membar_release (rw/w) is guaranteed by membar_sync (rw/rw)

And, for the deprecated membars:

- membar_enter (whether r/rw, w/rw, or rw/rw) is guaranteed by
membar_sync (rw/rw)
- membar_exit (rw/w) is guaranteed by membar_release (rw/w)

(membar_exit is identical to membar_release, but the name is
deprecated.)

Finally, while here, annotate some of the instructions with their
semantics. For powerpc, leave an essay with citations on the
unfortunate but -- as far as I can tell -- necessary decision to use
lwsync, not isync, for membar_acquire and membar_consumer.

Also add membar(3) and atomic(3) man page links.
 1.34 09-Apr-2022  riastradh i386/membar_ops: Upgrade membar_enter from R/RW to RW/RW.

This will be deprecated soon but let's avoid leaving rakes to trip on
with it arising from disagreement over the documentation (W/RW) and
implementation and usage (R/RW).
 1.33 09-Apr-2022  riastradh x86: Add a note on membar_sync and mfence.
 1.32 09-Apr-2022  riastradh x86: Omit needless store in membar_producer/exit.

On x86, every store is a store-release, so there is no need for any
barrier. But this wasn't a barrier anyway; it was just a store,
which was redundant with the store of the return address to the stack
implied by CALL even if issuing a store made a difference.
 1.31 09-Apr-2022  riastradh x86: Every load is a load-acquire, so membar_consumer is a noop.

lfence is only needed for MD logic, such as operations on I/O memory
rather than normal cacheable memory, or special instructions like
RDTSC -- never for MI synchronization between threads/CPUs. No need
for hot-patching to do lfence here.

(The x86_lfence function might reasonably be patched on i386 to do
lfence for MD logic, but it isn't now and this doesn't change that.)
 1.30 06-Apr-2022  riastradh Nix trailing whitespace in files of membars, atomics, and lock stubs.

Will be touching many of these files soon for functional changes.

No functional change intended.
 1.29 01-May-2020  maxv Use the hotpatch framework when patching _atomic_cas_64.
 1.28 26-Apr-2020  maxv Use the hotpatch framework for LFENCE/MFENCE.
 1.27 26-Apr-2020  maxv Remove unused argument in macro.
 1.26 26-Apr-2020  maxv Remove unused.
 1.25 26-Apr-2020  maxv Drop the hardcoded array, use the hotpatch section.
 1.24 25-Apr-2020  bouyer Merge the bouyer-xenpvh branch, bringing in Xen PV drivers support under HVM
guests in GENERIC.
Xen support can be disabled at runtime with
boot -c
disable hypervisor
 1.23 18-Jul-2018  bouyer branches: 1.23.8;
On Xen, always alias _atomic_cas_64 to _atomic_cas_cx8. AFAIK Xen doesn't
support CPUs that don't support cx8.
i386 XENPAE_DOMU boots again.
 1.22 23-May-2014  uebayasi branches: 1.22.22; 1.22.24;
Put missing END() markers to set ELF symbol size.
 1.21 22-Apr-2014  christos The kernel uses 64 bit atomic ops.
 1.20 18-Feb-2014  martin branches: 1.20.2;
Provide most missing __sync_*64 primitives for i386
 1.19 12-Jan-2011  joerg branches: 1.19.6; 1.19.12;
Allow use of traditional CPP to be set on a per platform base in sys.mk.
Honour this for dependency processing in bsd.dep.mk. Switch i386 and
amd64 assembly to use ISO C90 preprocessor concat and drop the
-traditional-cpp on this platform.
 1.18 26-Nov-2009  pooka Use strong alias within the kernel namespace regardless of if we're
dealing with a hard or soft kernel (kernel linker doesn't support
weak symbols).
 1.17 02-Apr-2009  enami So that profile kernel runs again,
- Adjust the size of functions used to patch.
- Fix the jump offset of mcount call when patching functions.

Approved by Andrew Doran.
 1.16 12-Jan-2009  pooka branches: 1.16.2;
include sys/param.h for _HARDKERNEL instead of homegrown def.
 1.15 04-Jan-2009  pooka Opt for libc versions in case of _KERNEL && !_RUMPKERNEL.
(kernel version uses sti/cli and is not PIC)
 1.14 19-Dec-2008  ad PR kern/40213 my i386 machine can't boot because of tsc

- Patch in atomic_cas_64() twice. The first patch is early and makes it
the MP-atomic version available if we have cmpxchg8b. The second patch
strips the lock prefix if ncpu==1.

- Fix the i486 atomic_cas_64() to not unconditionally enable interrupts.
 1.13 25-May-2008  chs branches: 1.13.4;
enable profiling of assembly functions.
 1.12 03-May-2008  yamt branches: 1.12.2;
whitespace.
 1.11 03-May-2008  yamt rename END to ENDLABEL. i'll use END for other purpose. ok by Andrew Doran.
 1.10 28-Apr-2008  martin Remove clause 3 and 4 from TNF licenses
 1.9 10-Feb-2008  ad branches: 1.9.4;
Add atomic_cas_foo_ni().
 1.8 09-Feb-2008  ad membar_enter was doing the wrong thing. For x86 we can alias:

membar_enter -> membar_consumer
membar_exit -> membar_producer
 1.7 20-Dec-2007  ad branches: 1.7.2;
- Make __cpu_simple_lock and similar real functions and patch at runtime.
- Remove old x86 atomic ops.
- Drop text alignment back to 16 on i386 (really, this time).
- Minor cleanup.
 1.6 20-Dec-2007  ad 64-bit atomic ops for i386.
 1.5 09-Dec-2007  ad Add missing strong aliases.
 1.4 29-Nov-2007  ad atomic_add_* takes signed integers, the others take unsigned.
 1.3 28-Nov-2007  ad A lock prefix on xchg is meaningless.
 1.2 28-Nov-2007  ad Fix up a few minor problems.
 1.1 28-Nov-2007  ad x86 atomic ops.
 1.7.2.3 23-Mar-2008  matt sync with HEAD
 1.7.2.2 09-Jan-2008  matt sync with HEAD
 1.7.2.1 20-Dec-2007  matt file atomic.S was added on branch matt-armv6 on 2008-01-09 01:20:53 +0000
 1.9.4.2 04-Jun-2008  yamt sync with head
 1.9.4.1 18-May-2008  yamt sync with head.
 1.12.2.1 23-Jun-2008  wrstuden Sync w/ -current. 34 merge conflicts to follow.
 1.13.4.2 03-Apr-2009  snj Pull up following revision(s) (requested by enami in ticket #645):
common/lib/libc/arch/i386/atomic/atomic.S: revision 1.17
sys/arch/amd64/amd64/spl.S: revision 1.21
sys/arch/x86/x86/patch.c: revision 1.17
So that profile kernel runs again,
- Adjust the size of functions used to patch.
- Fix the jump offset of mcount call when patching functions.
Approved by Andrew Doran.
 1.13.4.1 02-Feb-2009  snj Pull up following revision(s) (requested by ad in ticket #343):
common/lib/libc/arch/i386/atomic/atomic.S: revision 1.14
sys/arch/x86/include/cpufunc.h: revision 1.9
sys/arch/x86/x86/identcpu.c: revision 1.12
sys/arch/x86/x86/cpu.c: revision 1.60
sys/arch/x86/x86/patch.c: revision 1.15
PR kern/40213 my i386 machine can't boot because of tsc
- Patch in atomic_cas_64() twice. The first patch is early and makes it
the MP-atomic version available if we have cmpxchg8b. The second patch
strips the lock prefix if ncpu==1.
- Fix the i486 atomic_cas_64() to not unconditionally enable interrupts.
 1.16.2.1 13-May-2009  jym Sync with HEAD.

Commit is split, to avoid a "too many arguments" protocol error.
 1.19.12.1 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.19.6.1 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.20.2.1 10-Aug-2014  tls Rebase.
 1.22.24.3 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.22.24.2 21-Apr-2020  martin Sync with HEAD
 1.22.24.1 10-Jun-2019  christos Sync with HEAD
 1.22.22.1 28-Jul-2018  pgoyette Sync with HEAD
 1.23.8.1 14-Apr-2020  bouyer Force _atomic_cas_cx8 only for XENPV; x86_patch works fine for (PV)HVM
 1.36.8.1 02-Aug-2025  perseant Sync with HEAD
 1.36.2.1 20-Jul-2024  martin Pull up following revision(s) (requested by riastradh in ticket #764):

common/lib/libc/arch/i386/atomic/atomic.S: revision 1.37
sys/arch/xen/include/xenring.h: revision 1.8
sys/arch/i386/i386/cpufunc.S: revision 1.52
sys/arch/amd64/amd64/cpufunc.S: revision 1.68
sys/arch/xen/include/hypervisor.h: revision 1.60
common/lib/libc/arch/x86_64/atomic/atomic.S: revision 1.30

xen: Don't hotpatch away LOCK prefix in xen_mb, even on UP boots.

Both xen_mb and membar_sync are designed to provide store-before-load
ordering, but xen_mb has to provide it in synchronizing guest with
hypervisor, while membar_sync only has to provide it in synchronizing
one (guest) CPU with another (guest) CPU.

It is safe to hotpatch away the LOCK prefix in membar_sync on a
uniprocessor boot because membar_sync is only designed to coordinate
between normal memory on multiple CPUs, and is never necessary when
there's only one CPU involved.

But xen_mb is used to coordinate between the guest and the `device'
implemented by a hypervisor, which might be running on another
_physical_ CPU even if the NetBSD guest only sees one `CPU', i.e.,
one _virtual_ CPU. So even on `uniprocessor' boots, xen_mb must
still issue an instruction with store-before-load ordering on
multiprocessor systems, such as a LOCK ADD (or MFENCE, but MFENCE is
costlier for no benefit here).

No need to change xen_wmb (release ordering, load/store-before-store)
or xen_rmb (acquire ordering, load-before-load/store) because every
x86 store is a store-release and every x86 load is a load-acquire,
even on multiprocessor systems, so there's no hotpatching involved
anyway.

PR kern/57199
 1.1 16-Apr-2007  thorpej branches: 1.1.2;
file atomic_add.S was initially added on branch thorpej-atomic.
 1.1.2.2 22-Apr-2007  thorpej Make sure namespace-cleansed aliases are avaialble for all atomic ops.
 1.1.2.1 16-Apr-2007  thorpej Add atomic op implementations for x86.
 1.1 16-Apr-2007  thorpej branches: 1.1.2;
file atomic_and.S was initially added on branch thorpej-atomic.
 1.1.2.2 22-Apr-2007  thorpej Make sure namespace-cleansed aliases are avaialble for all atomic ops.
 1.1.2.1 16-Apr-2007  thorpej Add atomic op implementations for x86.
 1.1 16-Apr-2007  thorpej branches: 1.1.2;
file atomic_cas_32.S was initially added on branch thorpej-atomic.
 1.1.2.5 22-Apr-2007  thorpej Make sure namespace-cleansed aliases are avaialble for all atomic ops.
 1.1.2.4 17-Apr-2007  thorpej Fix the end-of-function padding so both versions of cas-32 end up the
same size.
 1.1.2.3 17-Apr-2007  thorpej Give the namespace treatment to _atomic_cas_32_486(), too.
 1.1.2.2 17-Apr-2007  thorpej Add the necessary aliasa for _atomic_cas_32().
 1.1.2.1 16-Apr-2007  thorpej Add atomic op implementations for x86.
 1.1 16-Apr-2007  thorpej branches: 1.1.2;
file atomic_dec.S was initially added on branch thorpej-atomic.
 1.1.2.2 22-Apr-2007  thorpej Make sure namespace-cleansed aliases are avaialble for all atomic ops.
 1.1.2.1 16-Apr-2007  thorpej Add atomic op implementations for x86.
 1.1 16-Apr-2007  thorpej branches: 1.1.2;
file atomic_inc.S was initially added on branch thorpej-atomic.
 1.1.2.2 22-Apr-2007  thorpej Make sure namespace-cleansed aliases are avaialble for all atomic ops.
 1.1.2.1 16-Apr-2007  thorpej Add atomic op implementations for x86.
 1.1 16-Apr-2007  thorpej branches: 1.1.2;
file atomic_op_asm.h was initially added on branch thorpej-atomic.
 1.1.2.1 16-Apr-2007  thorpej Add atomic op implementations for x86.
 1.1 16-Apr-2007  thorpej branches: 1.1.2;
file atomic_or.S was initially added on branch thorpej-atomic.
 1.1.2.3 22-Apr-2007  thorpej Make sure namespace-cleansed aliases are avaialble for all atomic ops.
 1.1.2.2 17-Apr-2007  thorpej Fix copy-and-pasto.
 1.1.2.1 16-Apr-2007  thorpej Add atomic op implementations for x86.
 1.1 16-Apr-2007  thorpej branches: 1.1.2;
file atomic_swap.S was initially added on branch thorpej-atomic.
 1.1.2.2 22-Apr-2007  thorpej Make sure namespace-cleansed aliases are avaialble for all atomic ops.
 1.1.2.1 16-Apr-2007  thorpej Add atomic op implementations for x86.
 1.1 16-Apr-2007  thorpej branches: 1.1.2;
file membar_ops.S was initially added on branch thorpej-atomic.
 1.1.2.2 22-Apr-2007  thorpej Make sure namespace-cleansed aliases are avaialble for all atomic ops.
 1.1.2.1 16-Apr-2007  thorpej Add atomic op implementations for x86.
 1.4 23-May-2014  uebayasi branches: 1.4.24;
Put missing END() markers to set ELF symbol size.
 1.3 28-Nov-2007  ad branches: 1.3.34; 1.3.48;
Use xchgb, bswapl and not rotate/shift.
 1.2 27-Dec-2005  yamt branches: 1.2.18;
don't rename bswap{16,32} if defined(_KERNEL) || defined(_STANDALONE).
(they are hidden by gcc-optimized versions, though.)
 1.1 20-Dec-2005  christos Merge libkern + libc common files. As requested by core.
 1.2.18.1 09-Jan-2008  matt sync with HEAD
 1.3.48.1 10-Aug-2014  tls Rebase.
 1.3.34.1 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.4.24.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.4.24.1 21-Apr-2020  martin Sync with HEAD
 1.4 23-May-2014  uebayasi branches: 1.4.24;
Put missing END() markers to set ELF symbol size.
 1.3 28-Nov-2007  ad branches: 1.3.34; 1.3.48;
Use xchgb, bswapl and not rotate/shift.
 1.2 27-Dec-2005  yamt branches: 1.2.18;
don't rename bswap{16,32} if defined(_KERNEL) || defined(_STANDALONE).
(they are hidden by gcc-optimized versions, though.)
 1.1 20-Dec-2005  christos Merge libkern + libc common files. As requested by core.
 1.2.18.1 09-Jan-2008  matt sync with HEAD
 1.3.48.1 10-Aug-2014  tls Rebase.
 1.3.34.1 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.4.24.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.4.24.1 21-Apr-2020  martin Sync with HEAD
 1.2 22-Mar-2014  jakllsch branches: 1.2.26;
For all i386 string assembly functions that don't overlap use END() so
that symbol size information is available.
 1.1 20-Dec-2005  christos branches: 1.1.50; 1.1.56;
Merge libkern + libc common files. As requested by core.
 1.1.56.1 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.1.50.1 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.2.26.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.2.26.1 21-Apr-2020  martin Sync with HEAD
 1.2 22-Mar-2014  jakllsch branches: 1.2.26;
For all i386 string assembly functions that don't overlap use END() so
that symbol size information is available.
 1.1 20-Dec-2005  christos branches: 1.1.50; 1.1.56;
Merge libkern + libc common files. As requested by core.
 1.1.56.1 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.1.50.1 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.2.26.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.2.26.1 21-Apr-2020  martin Sync with HEAD
 1.4 27-Jan-2020  ad x86 uses the C versions of bcmp() and memcmp() now.
 1.3 22-Mar-2014  jakllsch branches: 1.3.26; 1.3.30;
For all i386 string assembly functions that don't overlap use END() so
that symbol size information is available.
 1.2 12-Nov-2007  ad branches: 1.2.28; 1.2.34;
Don't unconditionally clear the direction flag. The ABI says it must always
be clear when making a function call, and 'cld' takes about 50 clock cyles
on the P4.
 1.1 20-Dec-2005  christos branches: 1.1.18;
Merge libkern + libc common files. As requested by core.
 1.1.18.1 09-Jan-2008  matt sync with HEAD
 1.2.34.1 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.2.28.1 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.3.30.1 29-Feb-2020  ad Sync with head.
 1.3.26.1 08-Apr-2020  martin Merge changes from current as of 20200406
 1.4 22-Mar-2014  jakllsch branches: 1.4.26;
For all i386 string assembly functions that don't overlap use END() so
that symbol size information is available.
 1.3 12-Nov-2007  ad branches: 1.3.28; 1.3.34;
Don't unconditionally clear the direction flag. The ABI says it must always
be clear when making a function call, and 'cld' takes about 50 clock cyles
on the P4.
 1.2 07-Apr-2006  yamt branches: 1.2.18;
prefix local labels with ".L".
 1.1 20-Dec-2005  christos Merge libkern + libc common files. As requested by core.
 1.2.18.1 09-Jan-2008  matt sync with HEAD
 1.3.34.1 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.3.28.1 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.4.26.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.4.26.1 21-Apr-2020  martin Sync with HEAD
 1.1 20-Dec-2005  christos branches: 1.1.94;
Merge libkern + libc common files. As requested by core.
 1.1.94.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.1.94.1 21-Apr-2020  martin Sync with HEAD
 1.5 23-May-2014  uebayasi branches: 1.5.24;
Put missing END() markers to set ELF symbol size.
 1.4 29-Apr-2008  martin branches: 1.4.26; 1.4.40;
Convert to new 2 clause license
 1.3 12-Nov-2007  ad branches: 1.3.6;
Don't unconditionally clear the direction flag. The ABI says it must always
be clear when making a function call, and 'cld' takes about 50 clock cyles
on the P4.
 1.2 07-Apr-2006  yamt branches: 1.2.18;
prefix local labels with ".L".
 1.1 20-Dec-2005  christos Merge libkern + libc common files. As requested by core.
 1.2.18.1 09-Jan-2008  matt sync with HEAD
 1.3.6.1 18-May-2008  yamt sync with head.
 1.4.40.1 10-Aug-2014  tls Rebase.
 1.4.26.1 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.5.24.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.5.24.1 21-Apr-2020  martin Sync with HEAD
 1.2 22-Mar-2014  jakllsch branches: 1.2.26;
For all i386 string assembly functions that don't overlap use END() so
that symbol size information is available.
 1.1 20-Dec-2005  christos branches: 1.1.50; 1.1.56;
Merge libkern + libc common files. As requested by core.
 1.1.56.1 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.1.50.1 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.2.26.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.2.26.1 21-Apr-2020  martin Sync with HEAD
 1.3 22-Mar-2014  jakllsch branches: 1.3.26;
For all i386 string assembly functions that don't overlap use END() so
that symbol size information is available.
 1.2 17-Jul-2009  dsl branches: 1.2.6; 1.2.12;
Change all archs so that strchr.[cS] and strrchr.[cS] exist and generate
duplicate symbols for index() and rindex().
 1.1 20-Dec-2005  christos branches: 1.1.36;
Merge libkern + libc common files. As requested by core.
 1.1.36.1 23-Jul-2009  jym Sync with HEAD.
 1.2.12.1 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.2.6.1 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.3.26.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.3.26.1 21-Apr-2020  martin Sync with HEAD
 1.2 22-Mar-2014  jakllsch branches: 1.2.26;
For all i386 string assembly functions that don't overlap use END() so
that symbol size information is available.
 1.1 20-Dec-2005  christos branches: 1.1.50; 1.1.56;
Merge libkern + libc common files. As requested by core.
 1.1.56.1 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.1.50.1 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.2.26.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.2.26.1 21-Apr-2020  martin Sync with HEAD
 1.2 22-Mar-2014  jakllsch branches: 1.2.26;
For all i386 string assembly functions that don't overlap use END() so
that symbol size information is available.
 1.1 20-Dec-2005  christos branches: 1.1.50; 1.1.56;
Merge libkern + libc common files. As requested by core.
 1.1.56.1 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.1.50.1 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.2.26.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.2.26.1 21-Apr-2020  martin Sync with HEAD
 1.5 30-Mar-2024  andvar s/Westley/Wesley/ in a book reference (in comments).
 1.4 08-Dec-2021  andvar s/efficent/efficient/ in comments.
 1.3 04-Oct-2021  andvar remove duplicate the article in comments.
 1.2 22-Mar-2014  jakllsch branches: 1.2.26;
For all i386 string assembly functions that don't overlap use END() so
that symbol size information is available.
 1.1 20-Dec-2005  christos branches: 1.1.50; 1.1.56;
Merge libkern + libc common files. As requested by core.
 1.1.56.1 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.1.50.1 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.2.26.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.2.26.1 21-Apr-2020  martin Sync with HEAD
 1.3 22-Mar-2014  jakllsch branches: 1.3.26;
For all i386 string assembly functions that don't overlap use END() so
that symbol size information is available.
 1.2 17-Jul-2009  dsl branches: 1.2.6; 1.2.12;
Change all archs so that strchr.[cS] and strrchr.[cS] exist and generate
duplicate symbols for index() and rindex().
 1.1 20-Dec-2005  christos branches: 1.1.36;
Merge libkern + libc common files. As requested by core.
 1.1.36.1 23-Jul-2009  jym Sync with HEAD.
 1.2.12.1 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.2.6.1 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.3.26.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.3.26.1 21-Apr-2020  martin Sync with HEAD
 1.2 22-Mar-2014  jakllsch branches: 1.2.26;
For all i386 string assembly functions that don't overlap use END() so
that symbol size information is available.
 1.1 16-Jun-2011  joerg branches: 1.1.2; 1.1.8;
Add support for size optimised versions of the assembler functions in
the small subdirectory. Provide more compact versions of the functions
used by bootxx in x86.
 1.1.8.1 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.1.2.1 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.2.26.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.2.26.1 21-Apr-2020  martin Sync with HEAD
 1.2 23-May-2014  uebayasi branches: 1.2.24;
Put missing END() markers to set ELF symbol size.
 1.1 16-Jun-2011  joerg branches: 1.1.8; 1.1.20;
Add support for size optimised versions of the assembler functions in
the small subdirectory. Provide more compact versions of the functions
used by bootxx in x86.
 1.1.20.1 10-Aug-2014  tls Rebase.
 1.1.8.1 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.2.24.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.2.24.1 21-Apr-2020  martin Sync with HEAD
 1.1 16-Jun-2011  joerg branches: 1.1.44;
Add support for size optimised versions of the assembler functions in
the small subdirectory. Provide more compact versions of the functions
used by bootxx in x86.
 1.1.44.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.1.44.1 21-Apr-2020  martin Sync with HEAD
 1.2 22-Mar-2014  jakllsch branches: 1.2.26;
For all i386 string assembly functions that don't overlap use END() so
that symbol size information is available.
 1.1 16-Jun-2011  joerg branches: 1.1.2; 1.1.8;
Add support for size optimised versions of the assembler functions in
the small subdirectory. Provide more compact versions of the functions
used by bootxx in x86.
 1.1.8.1 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.1.2.1 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.2.26.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.2.26.1 21-Apr-2020  martin Sync with HEAD
 1.3 22-Sep-2014  khorben branches: 1.3.16;
Look for the character to locate before checking for the NUL character

As documented in PR port-i386/49208, this fixes strchr(s, '\0'), as used by
the FAT first-stage bootloader on x86 (bootxx_msdos).
strchr(s, '\0') is otherwise equivalent to strlen(string), which would
probably look nicer in the original file, dosfs.c from libsa.

Confirmed working in qemu and on real hardware.
ok joerg@

XXX pull-up to netbsd-6 and netbsd-7
 1.2 22-Mar-2014  jakllsch branches: 1.2.4;
For all i386 string assembly functions that don't overlap use END() so
that symbol size information is available.
 1.1 16-Jun-2011  joerg branches: 1.1.2; 1.1.4; 1.1.8;
Add support for size optimised versions of the assembler functions in
the small subdirectory. Provide more compact versions of the functions
used by bootxx in x86.
 1.1.8.1 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.1.4.1 27-Oct-2014  msaitoh Pull up following revision(s) (requested by khorben in ticket #1158):
common/lib/libc/arch/i386/string/small/strchr.S: revision 1.3
Look for the character to locate before checking for the NUL character
As documented in PR port-i386/49208, this fixes strchr(s, '\0'), as used by
the FAT first-stage bootloader on x86 (bootxx_msdos).
strchr(s, '\0') is otherwise equivalent to strlen(string), which would
probably look nicer in the original file, dosfs.c from libsa.
Confirmed working in qemu and on real hardware.
ok joerg@
XXX pull-up to netbsd-6 and netbsd-7
 1.1.2.1 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.2.4.1 29-Sep-2014  msaitoh Pull up following revision(s) (requested by khorben in ticket #118):
common/lib/libc/arch/i386/string/small/strchr.S: revision 1.3
Look for the character to locate before checking for the NUL character
As documented in PR port-i386/49208, this fixes strchr(s, '\0'), as used by
the FAT first-stage bootloader on x86 (bootxx_msdos).
strchr(s, '\0') is otherwise equivalent to strlen(string), which would
probably look nicer in the original file, dosfs.c from libsa.
Confirmed working in qemu and on real hardware.
ok joerg@
XXX pull-up to netbsd-6 and netbsd-7
 1.3.16.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.3.16.1 21-Apr-2020  martin Sync with HEAD
 1.2 22-Mar-2014  jakllsch branches: 1.2.26;
For all i386 string assembly functions that don't overlap use END() so
that symbol size information is available.
 1.1 16-Jun-2011  joerg branches: 1.1.2; 1.1.8;
Add support for size optimised versions of the assembler functions in
the small subdirectory. Provide more compact versions of the functions
used by bootxx in x86.
 1.1.8.1 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.1.2.1 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.2.26.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.2.26.1 21-Apr-2020  martin Sync with HEAD
 1.2 22-Mar-2014  jakllsch branches: 1.2.26;
For all i386 string assembly functions that don't overlap use END() so
that symbol size information is available.
 1.1 16-Jun-2011  joerg branches: 1.1.2; 1.1.8;
Add support for size optimised versions of the assembler functions in
the small subdirectory. Provide more compact versions of the functions
used by bootxx in x86.
 1.1.8.1 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.1.2.1 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.2.26.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.2.26.1 21-Apr-2020  martin Sync with HEAD
 1.2 22-Mar-2014  jakllsch branches: 1.2.26;
For all i386 string assembly functions that don't overlap use END() so
that symbol size information is available.
 1.1 16-Jun-2011  joerg branches: 1.1.2; 1.1.8;
Add support for size optimised versions of the assembler functions in
the small subdirectory. Provide more compact versions of the functions
used by bootxx in x86.
 1.1.8.1 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.1.2.1 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.2.26.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.2.26.1 21-Apr-2020  martin Sync with HEAD
 1.3 04-Jan-2009  pooka branches: 1.3.50;
allow inclusion of atomic ops in librump
 1.2 29-Sep-2008  ad Allow atomic ops to be built as part of libpthread.
 1.1 20-Mar-2008  kochi branches: 1.1.2; 1.1.4;
Add ia64 MI atomic ops.
(Not tested at all, just for reference)
 1.1.4.2 24-Mar-2008  keiichi sync with head.
 1.1.4.1 20-Mar-2008  keiichi file Makefile.inc was added on branch keiichi-mipv6 on 2008-03-24 07:14:29 +0000
 1.1.2.2 23-Mar-2008  matt sync with HEAD
 1.1.2.1 20-Mar-2008  matt file Makefile.inc was added on branch matt-armv6 on 2008-03-23 00:12:41 +0000
 1.3.50.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.3.50.1 21-Apr-2020  martin Sync with HEAD
 1.7 09-Apr-2022  riastradh Introduce membar_acquire/release. Deprecate membar_enter/exit.

The names membar_enter/exit were unclear, and the documentation of
membar_enter has disagreed with the implementations on sparc,
powerpc, and even x86(!) for the entire time it has been in NetBSD.

The terms `acquire' and `release' are ubiquitous in the literature
today, and have been adopted in the C and C++ standards to mean
load-before-load/store and load/store-before-store, respectively,
which are exactly the orderings required by acquiring and releasing a
mutex, as well as other useful applications like decrementing a
reference count and then freeing the underlying object if it went to
zero.

Originally I proposed changing one word in the documentation for
membar_enter to make it load-before-load/store instead of
store-before-load/store, i.e., to make it an acquire barrier. I
proposed this on the grounds that

(a) all implementations guarantee load-before-load/store,
(b) some implementations fail to guarantee store-before-load/store,
and
(c) all uses in-tree assume load-before-load/store.

I verified parts (a) and (b) (except, for (a), powerpc didn't even
guarantee load-before-load/store -- isync isn't necessarily enough;
need lwsync in general -- but it _almost_ did, and it certainly didn't
guarantee store-before-load/store).

Part (c) might not be correct, however: under the mistaken assumption
that atomic-r/m/w then membar-w/rw is equivalent to atomic-r/m/w then
membar-r/rw, I only audited the cases of membar_enter that _aren't_
immediately after an atomic-r/m/w. All of those cases assume
load-before-load/store. But my assumption was wrong -- there are
cases of atomic-r/m/w then membar-w/rw that would be broken by
changing to atomic-r/m/w then membar-r/rw:

https://mail-index.netbsd.org/tech-kern/2022/03/29/msg028044.html

Furthermore, the name membar_enter has been adopted in other places
like OpenBSD where it actually does follow the documentation and
guarantee store-before-load/store, even if that order is not useful.
So the name membar_enter currently lives in a bad place where it
means either of two things -- r/rw or w/rw.

With this change, we deprecate membar_enter/exit, introduce
membar_acquire/release as better names for the useful pair (r/rw and
rw/w), and make sure the implementation of membar_enter guarantees
both what was documented _and_ what was implemented, making it an
alias for membar_sync.

While here, rework all of the membar_* definitions and aliases. The
new logic follows a rule to make it easier to audit:

membar_X is defined as an alias for membar_Y iff membar_X is
guaranteed by membar_Y.

The `no stronger than' relation is (the transitive closure of):

- membar_consumer (r/r) is guaranteed by membar_acquire (r/rw)
- membar_producer (w/w) is guaranteed by membar_release (rw/w)
- membar_acquire (r/rw) is guaranteed by membar_sync (rw/rw)
- membar_release (rw/w) is guaranteed by membar_sync (rw/rw)

And, for the deprecated membars:

- membar_enter (whether r/rw, w/rw, or rw/rw) is guaranteed by
membar_sync (rw/rw)
- membar_exit (rw/w) is guaranteed by membar_release (rw/w)

(membar_exit is identical to membar_release, but the name is
deprecated.)

Finally, while here, annotate some of the instructions with their
semantics. For powerpc, leave an essay with citations on the
unfortunate but -- as far as I can tell -- necessary decision to use
lwsync, not isync, for membar_acquire and membar_consumer.

Also add membar(3) and atomic(3) man page links.
 1.6 06-Apr-2022  riastradh Nix trailing whitespace in files of membars, atomics, and lock stubs.

Will be touching many of these files soon for functional changes.

No functional change intended.
 1.5 09-Nov-2009  skrll branches: 1.5.48;
New binutils seems to want end of functions marked.
 1.4 25-May-2008  chs enable profiling of assembly functions.
 1.3 28-Apr-2008  martin branches: 1.3.2;
Remove clause 3 and 4 from TNF licenses
 1.2 24-Mar-2008  kochi branches: 1.2.2;
Add atomic_cas_foo_ni alias.
 1.1 20-Mar-2008  kochi branches: 1.1.2; 1.1.4;
Add ia64 MI atomic ops.
(Not tested at all, just for reference)
 1.1.4.2 24-Mar-2008  keiichi sync with head.
 1.1.4.1 20-Mar-2008  keiichi file atomic.S was added on branch keiichi-mipv6 on 2008-03-24 07:14:29 +0000
 1.1.2.2 23-Mar-2008  matt sync with HEAD
 1.1.2.1 20-Mar-2008  matt file atomic.S was added on branch matt-armv6 on 2008-03-23 00:12:41 +0000
 1.2.2.2 04-Jun-2008  yamt sync with head
 1.2.2.1 18-May-2008  yamt sync with head.
 1.3.2.1 23-Jun-2008  wrstuden Sync w/ -current. 34 merge conflicts to follow.
 1.5.48.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.5.48.1 21-Apr-2020  martin Sync with HEAD
 1.1 05-Aug-2016  scole branches: 1.1.2; 1.1.16;
PR port-ia64/51261

Imported from FreeBSD with a minor change or two
 1.1.16.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.1.16.1 21-Apr-2020  martin Sync with HEAD
 1.1.2.2 06-Aug-2016  pgoyette Sync with HEAD
 1.1.2.1 05-Aug-2016  pgoyette file bcopy.S was added on branch pgoyette-localcount on 2016-08-06 00:18:38 +0000
 1.1 05-Aug-2016  scole branches: 1.1.2; 1.1.16;
PR port-ia64/51261

Imported from FreeBSD with a minor change or two
 1.1.16.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.1.16.1 21-Apr-2020  martin Sync with HEAD
 1.1.2.2 06-Aug-2016  pgoyette Sync with HEAD
 1.1.2.1 05-Aug-2016  pgoyette file bzero.S was added on branch pgoyette-localcount on 2016-08-06 00:18:38 +0000
 1.1 05-Aug-2016  scole branches: 1.1.2; 1.1.16;
PR port-ia64/51261

Imported from FreeBSD with a minor change or two
 1.1.16.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.1.16.1 21-Apr-2020  martin Sync with HEAD
 1.1.2.2 06-Aug-2016  pgoyette Sync with HEAD
 1.1.2.1 05-Aug-2016  pgoyette file ffs.S was added on branch pgoyette-localcount on 2016-08-06 00:18:38 +0000
 1.1 05-Aug-2016  scole branches: 1.1.2; 1.1.16;
PR port-ia64/51261

Imported from FreeBSD with a minor change or two
 1.1.16.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.1.16.1 21-Apr-2020  martin Sync with HEAD
 1.1.2.2 06-Aug-2016  pgoyette Sync with HEAD
 1.1.2.1 05-Aug-2016  pgoyette file memcpy.S was added on branch pgoyette-localcount on 2016-08-06 00:18:38 +0000
 1.1 05-Aug-2016  scole branches: 1.1.2; 1.1.16;
PR port-ia64/51261

Imported from FreeBSD with a minor change or two
 1.1.16.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.1.16.1 21-Apr-2020  martin Sync with HEAD
 1.1.2.2 06-Aug-2016  pgoyette Sync with HEAD
 1.1.2.1 05-Aug-2016  pgoyette file memmove.S was added on branch pgoyette-localcount on 2016-08-06 00:18:38 +0000
 1.16 30-Mar-2023  riastradh libc: Define __atomic_is_lock_free.

Limited to architectures where it is actually needed by gcc for any
calls to stdatomic.h atomic_is_lock_free for now.

We should also add it to other architectures too, along with lockful
atomic r/m/w operations for sizes that can't be handled natively, but
that's a lot more work. It is also necessary for -fno-inline-atomics
but we're missing a lot of other symbols for that too, to be fixed.
For now, this should enable the OpenSSL build to complete on these
architectures again after I reverted a local change.

XXX pullup-10
 1.15 28-Feb-2019  isaki branches: 1.15.14;
Add missing atomic_and_{8,16}_nv_cas.c for __sync_and_and_fetch_{1,2}.
XXX why is not only atomic_and_* symmetric unlike the others?
(in common/lib/libc/atomic/)
 1.14 20-Nov-2014  matt branches: 1.14.16;
Add atomic_c11_compare_exchange_cas_{8,16,32}.c
 1.13 22-Feb-2014  martin Add missing __sunc_* ops for sun2
 1.12 20-Feb-2014  martin Add missing __sync_* ops
 1.11 29-Jan-2014  martin Provide _atomic_cas_8_up and _atomic_cas_16_up as assembler functions
 1.10 18-Jul-2013  matt invert tests ${MACHINE} != "m68000" -> ${MACHINE} == "m68k"
 1.9 04-Jan-2009  pooka branches: 1.9.8; 1.9.14; 1.9.18;
allow inclusion of atomic ops in librump
 1.8 29-Sep-2008  ad Allow atomic ops to be built as part of libpthread.
 1.7 29-Apr-2008  scw Implement _atomic_cas_up() in assembly code as the compiler cannot be
trusted to generate fully restartable code sequences.

Addresses lib/38482 for ARM and m68000.
 1.6 06-Apr-2008  tsutsui branches: 1.6.2;
Fix pasto.
 1.5 05-Apr-2008  tsutsui Use .c version atomic ops for m68010.
 1.4 11-Feb-2008  ad Only build atomic ops for libkern/libc.
 1.3 10-Feb-2008  ad Enable the atomic ops in userspace.
 1.2 29-Nov-2007  ad branches: 1.2.4;
Pull in m68k atomic ops from the thorpej-atomic branch.
 1.1 22-Apr-2007  thorpej branches: 1.1.2;
file Makefile.inc was initially added on branch thorpej-atomic.
 1.1.2.1 22-Apr-2007  thorpej Atomic ops for 68020 and better processors. 68010 will need to use
something else for anything that uses the CAS instruction.
 1.2.4.3 23-Mar-2008  matt sync with HEAD
 1.2.4.2 09-Jan-2008  matt sync with HEAD
 1.2.4.1 29-Nov-2007  matt file Makefile.inc was added on branch matt-armv6 on 2008-01-09 01:20:54 +0000
 1.6.2.1 18-May-2008  yamt sync with head.
 1.9.18.1 23-Jul-2013  riastradh sync with HEAD
 1.9.14.1 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.9.8.1 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.14.16.2 21-Apr-2020  martin Sync with HEAD
 1.14.16.1 10-Jun-2019  christos Sync with HEAD
 1.15.14.1 31-Jul-2023  martin Pull up following revision(s) (requested by riastradh in ticket #275):

common/lib/libc/arch/sparc/atomic/Makefile.inc: revision 1.24
common/lib/libc/arch/m68k/atomic/Makefile.inc: revision 1.16
common/lib/libc/arch/mips/atomic/Makefile.inc: revision 1.16
common/lib/libc/arch/hppa/atomic/Makefile.inc: revision 1.15
common/lib/libc/arch/vax/atomic/Makefile.inc: revision 1.9
common/lib/libc/atomic/atomic_is_lock_free.c: revision 1.1
common/lib/libc/arch/sh3/atomic/Makefile.inc: revision 1.9

libc: Define __atomic_is_lock_free.

Limited to architectures where it is actually needed by gcc for any
calls to stdatomic.h atomic_is_lock_free for now.

We should also add it to other architectures too, along with lockful
atomic r/m/w operations for sizes that can't be handled natively, but
that's a lot more work. It is also necessary for -fno-inline-atomics
but we're missing a lot of other symbols for that too, to be fixed.

For now, this should enable the OpenSSL build to complete on these
architectures again after I reverted a local change.
 1.11 06-Apr-2022  riastradh Nix trailing whitespace in files of membars, atomics, and lock stubs.

Will be touching many of these files soon for functional changes.

No functional change intended.
 1.10 23-Jun-2014  joerg branches: 1.10.24;
Add aliases for the builtins used to implement C11/C++11 atomics.
 1.9 22-Feb-2014  martin branches: 1.9.2;
Try to hide the C runtime implementation specific __sync_* ops from librump,
to avoid duplicates.
 1.8 20-Feb-2014  martin Add missing __sync_* ops
 1.7 18-Feb-2014  martin Add a few __sync_* and atomic functions.
 1.6 16-Jul-2013  matt Add END(func)
Fix some straggling %r@<eol> to (%r)
 1.5 16-Jul-2013  matt Convert to normal Motorola syntax for addressing.
%r@ -> (%r)
%r@- -> -(%r)
%r@+ -> (%r)+
%r@(n) n(%r)
etc.
No object differences.
 1.4 25-May-2008  chs branches: 1.4.2; 1.4.24;
enable profiling of assembly functions.
 1.3 28-Apr-2008  martin branches: 1.3.2;
Remove clause 3 and 4 from TNF licenses
 1.2 29-Nov-2007  ad branches: 1.2.4; 1.2.8;
Pull in m68k atomic ops from the thorpej-atomic branch.
 1.1 22-Apr-2007  thorpej branches: 1.1.2;
file atomic_add.S was initially added on branch thorpej-atomic.
 1.1.2.2 22-Apr-2007  thorpej Make sure namespace-cleansed aliases are avaialble for all atomic ops.
 1.1.2.1 22-Apr-2007  thorpej Atomic ops for 68020 and better processors. 68010 will need to use
something else for anything that uses the CAS instruction.
 1.2.8.2 04-Jun-2008  yamt sync with head
 1.2.8.1 18-May-2008  yamt sync with head.
 1.2.4.2 09-Jan-2008  matt sync with HEAD
 1.2.4.1 29-Nov-2007  matt file atomic_add.S was added on branch matt-armv6 on 2008-01-09 01:20:55 +0000
 1.3.2.1 23-Jun-2008  wrstuden Sync w/ -current. 34 merge conflicts to follow.
 1.4.24.1 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.4.2.1 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.9.2.1 10-Aug-2014  tls Rebase.
 1.10.24.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.10.24.1 21-Apr-2020  martin Sync with HEAD
 1.12 06-Apr-2022  riastradh Nix trailing whitespace in files of membars, atomics, and lock stubs.

Will be touching many of these files soon for functional changes.

No functional change intended.
 1.11 23-Jun-2014  joerg branches: 1.11.24;
Add aliases for the builtins used to implement C11/C++11 atomics.
 1.10 22-Feb-2014  martin branches: 1.10.2;
Try to hide the C runtime implementation specific __sync_* ops from librump,
to avoid duplicates.
 1.9 20-Feb-2014  martin Add missing __sync_* ops
 1.8 16-Jul-2013  matt Add END(func)
Fix some straggling %r@<eol> to (%r)
 1.7 16-Jul-2013  matt Convert to normal Motorola syntax for addressing.
%r@ -> (%r)
%r@- -> -(%r)
%r@+ -> (%r)+
%r@(n) n(%r)
etc.
No object differences.
 1.6 25-May-2008  chs branches: 1.6.2; 1.6.24;
enable profiling of assembly functions.
 1.5 28-Apr-2008  martin branches: 1.5.2;
Remove clause 3 and 4 from TNF licenses
 1.4 12-Jan-2008  ad branches: 1.4.4;
atomic_and/atomic_or don't operate on pointers, so don't copy the return
to a0. From thorpej@.
 1.3 29-Nov-2007  ad branches: 1.3.4;
Apparently pointers are also returned in %a0.
 1.2 29-Nov-2007  ad Pull in m68k atomic ops from the thorpej-atomic branch.
 1.1 22-Apr-2007  thorpej branches: 1.1.2;
file atomic_and.S was initially added on branch thorpej-atomic.
 1.1.2.2 22-Apr-2007  thorpej Make sure namespace-cleansed aliases are avaialble for all atomic ops.
 1.1.2.1 22-Apr-2007  thorpej Atomic ops for 68020 and better processors. 68010 will need to use
something else for anything that uses the CAS instruction.
 1.3.4.3 23-Mar-2008  matt sync with HEAD
 1.3.4.2 09-Jan-2008  matt sync with HEAD
 1.3.4.1 29-Nov-2007  matt file atomic_and.S was added on branch matt-armv6 on 2008-01-09 01:20:55 +0000
 1.4.4.2 04-Jun-2008  yamt sync with head
 1.4.4.1 18-May-2008  yamt sync with head.
 1.5.2.1 23-Jun-2008  wrstuden Sync w/ -current. 34 merge conflicts to follow.
 1.6.24.1 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.6.2.1 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.10.2.1 10-Aug-2014  tls Rebase.
 1.11.24.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.11.24.1 21-Apr-2020  martin Sync with HEAD
 1.14 06-Apr-2022  riastradh Nix trailing whitespace in files of membars, atomics, and lock stubs.

Will be touching many of these files soon for functional changes.

No functional change intended.
 1.13 10-Jul-2015  mlelstv branches: 1.13.16;
fix comment
 1.12 04-Jul-2015  isaki Improve the code in __sync_bool_compare_and_swap_{1,2,4}.
- bccs is smaller and faster than bcc(.w) in this case.
- it can be used movql in this case (assembler optimise it though).
 1.11 04-Jul-2015  isaki atomic_cas_{8,16}:
- Correct the wrong offset in stack.
__sync_bool_compare_and_swap_{1,2,4}:
- Correct the wrong offset in stack.
- D3 must be preserved in subroutines.
PR/49995.
 1.10 22-Feb-2014  martin branches: 1.10.4;
Try to hide the C runtime implementation specific __sync_* ops from librump,
to avoid duplicates.
 1.9 20-Feb-2014  martin Simplify the variants returning a bool
 1.8 18-Feb-2014  martin Add a few __sync_* and atomic functions.
 1.7 16-Jul-2013  matt Add END(func)
Fix some straggling %r@<eol> to (%r)
 1.6 16-Jul-2013  matt Convert to normal Motorola syntax for addressing.
%r@ -> (%r)
%r@- -> -(%r)
%r@+ -> (%r)+
%r@(n) n(%r)
etc.
No object differences.
 1.5 25-May-2008  chs branches: 1.5.2; 1.5.24;
enable profiling of assembly functions.
 1.4 28-Apr-2008  martin branches: 1.4.2;
Remove clause 3 and 4 from TNF licenses
 1.3 10-Feb-2008  ad branches: 1.3.4;
Add atomic_cas_foo_ni().
 1.2 29-Nov-2007  ad branches: 1.2.4;
Pull in m68k atomic ops from the thorpej-atomic branch.
 1.1 22-Apr-2007  thorpej branches: 1.1.2;
file atomic_cas.S was initially added on branch thorpej-atomic.
 1.1.2.2 22-Apr-2007  thorpej Make sure namespace-cleansed aliases are avaialble for all atomic ops.
 1.1.2.1 22-Apr-2007  thorpej Atomic ops for 68020 and better processors. 68010 will need to use
something else for anything that uses the CAS instruction.
 1.2.4.3 23-Mar-2008  matt sync with HEAD
 1.2.4.2 09-Jan-2008  matt sync with HEAD
 1.2.4.1 29-Nov-2007  matt file atomic_cas.S was added on branch matt-armv6 on 2008-01-09 01:20:56 +0000
 1.3.4.2 04-Jun-2008  yamt sync with head
 1.3.4.1 18-May-2008  yamt sync with head.
 1.4.2.1 23-Jun-2008  wrstuden Sync w/ -current. 34 merge conflicts to follow.
 1.5.24.1 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.5.2.1 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.10.4.1 16-Jul-2015  snj Pull up following revision(s) (requested by isaki in ticket #865):
common/lib/libc/arch/m68k/atomic/atomic_cas.S: revisions 1.11, 1.12
atomic_cas_{8,16}:
- Correct the wrong offset in stack.
__sync_bool_compare_and_swap_{1,2,4}:
- Correct the wrong offset in stack.
- D3 must be preserved in subroutines.
PR/49995.
--
Improve the code in __sync_bool_compare_and_swap_{1,2,4}.
- bccs is smaller and faster than bcc(.w) in this case.
- it can be used movql in this case (assembler optimise it though).
 1.13.16.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.13.16.1 21-Apr-2020  martin Sync with HEAD
 1.7 06-Apr-2022  riastradh Nix trailing whitespace in files of membars, atomics, and lock stubs.

Will be touching many of these files soon for functional changes.

No functional change intended.
 1.6 29-Jan-2014  martin branches: 1.6.26;
Provide _atomic_cas_8_up and _atomic_cas_16_up as assembler functions
 1.5 16-Jul-2013  matt Add END(func)
Fix some straggling %r@<eol> to (%r)
 1.4 16-Jul-2013  matt Convert to normal Motorola syntax for addressing.
%r@ -> (%r)
%r@- -> -(%r)
%r@+ -> (%r)+
%r@(n) n(%r)
etc.
No object differences.
 1.3 25-May-2008  chs branches: 1.3.2; 1.3.24;
enable profiling of assembly functions.
 1.2 30-Apr-2008  tsutsui branches: 1.2.2; 1.2.4;
Make this compile (missing register prefix).
 1.1 29-Apr-2008  scw Implement _atomic_cas_up() in assembly code as the compiler cannot be
trusted to generate fully restartable code sequences.

Addresses lib/38482 for ARM and m68000.
 1.2.4.3 04-Jun-2008  yamt sync with head
 1.2.4.2 18-May-2008  yamt sync with head.
 1.2.4.1 30-Apr-2008  yamt file atomic_cas_68000.S was added on branch yamt-pf42 on 2008-05-18 12:28:45 +0000
 1.2.2.1 23-Jun-2008  wrstuden Sync w/ -current. 34 merge conflicts to follow.
 1.3.24.1 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.3.2.1 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.6.26.1 21-Apr-2020  martin Sync with HEAD
 1.7 06-Apr-2022  riastradh Nix trailing whitespace in files of membars, atomics, and lock stubs.

Will be touching many of these files soon for functional changes.

No functional change intended.
 1.6 16-Jul-2013  matt branches: 1.6.28;
Add END(func)
Fix some straggling %r@<eol> to (%r)
 1.5 16-Jul-2013  matt Convert to normal Motorola syntax for addressing.
%r@ -> (%r)
%r@- -> -(%r)
%r@+ -> (%r)+
%r@(n) n(%r)
etc.
No object differences.
 1.4 25-May-2008  chs branches: 1.4.2; 1.4.24;
enable profiling of assembly functions.
 1.3 28-Apr-2008  martin branches: 1.3.2;
Remove clause 3 and 4 from TNF licenses
 1.2 29-Nov-2007  ad branches: 1.2.4; 1.2.8;
Pull in m68k atomic ops from the thorpej-atomic branch.
 1.1 22-Apr-2007  thorpej branches: 1.1.2;
file atomic_dec.S was initially added on branch thorpej-atomic.
 1.1.2.2 22-Apr-2007  thorpej Make sure namespace-cleansed aliases are avaialble for all atomic ops.
 1.1.2.1 22-Apr-2007  thorpej Atomic ops for 68020 and better processors. 68010 will need to use
something else for anything that uses the CAS instruction.
 1.2.8.2 04-Jun-2008  yamt sync with head
 1.2.8.1 18-May-2008  yamt sync with head.
 1.2.4.2 09-Jan-2008  matt sync with HEAD
 1.2.4.1 29-Nov-2007  matt file atomic_dec.S was added on branch matt-armv6 on 2008-01-09 01:20:56 +0000
 1.3.2.1 23-Jun-2008  wrstuden Sync w/ -current. 34 merge conflicts to follow.
 1.4.24.1 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.4.2.1 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.6.28.1 21-Apr-2020  martin Sync with HEAD
 1.7 06-Apr-2022  riastradh Nix trailing whitespace in files of membars, atomics, and lock stubs.

Will be touching many of these files soon for functional changes.

No functional change intended.
 1.6 16-Jul-2013  matt branches: 1.6.28;
Add END(func)
Fix some straggling %r@<eol> to (%r)
 1.5 16-Jul-2013  matt Convert to normal Motorola syntax for addressing.
%r@ -> (%r)
%r@- -> -(%r)
%r@+ -> (%r)+
%r@(n) n(%r)
etc.
No object differences.
 1.4 25-May-2008  chs branches: 1.4.2; 1.4.24;
enable profiling of assembly functions.
 1.3 28-Apr-2008  martin branches: 1.3.2;
Remove clause 3 and 4 from TNF licenses
 1.2 29-Nov-2007  ad branches: 1.2.4; 1.2.8;
Pull in m68k atomic ops from the thorpej-atomic branch.
 1.1 22-Apr-2007  thorpej branches: 1.1.2;
file atomic_inc.S was initially added on branch thorpej-atomic.
 1.1.2.2 22-Apr-2007  thorpej Make sure namespace-cleansed aliases are avaialble for all atomic ops.
 1.1.2.1 22-Apr-2007  thorpej Atomic ops for 68020 and better processors. 68010 will need to use
something else for anything that uses the CAS instruction.
 1.2.8.2 04-Jun-2008  yamt sync with head
 1.2.8.1 18-May-2008  yamt sync with head.
 1.2.4.2 09-Jan-2008  matt sync with HEAD
 1.2.4.1 29-Nov-2007  matt file atomic_inc.S was added on branch matt-armv6 on 2008-01-09 01:20:57 +0000
 1.3.2.1 23-Jun-2008  wrstuden Sync w/ -current. 34 merge conflicts to follow.
 1.4.24.1 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.4.2.1 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.6.28.1 21-Apr-2020  martin Sync with HEAD
 1.3 06-Apr-2022  riastradh Nix trailing whitespace in files of membars, atomics, and lock stubs.

Will be touching many of these files soon for functional changes.

No functional change intended.
 1.2 22-Feb-2014  martin branches: 1.2.4; 1.2.8; 1.2.30;
Try to hide the C runtime implementation specific __sync_* ops from librump,
to avoid duplicates.
 1.1 20-Feb-2014  martin Add missing __sync_* ops
 1.2.30.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.2.30.1 21-Apr-2020  martin Sync with HEAD
 1.2.8.2 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.2.8.1 22-Feb-2014  tls file atomic_nand.S was added on branch tls-maxphys on 2014-08-19 23:45:13 +0000
 1.2.4.2 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.2.4.1 22-Feb-2014  yamt file atomic_nand.S was added on branch yamt-pagecache on 2014-05-22 11:26:29 +0000
 1.6 06-Apr-2022  riastradh Nix trailing whitespace in files of membars, atomics, and lock stubs.

Will be touching many of these files soon for functional changes.

No functional change intended.
 1.5 23-Feb-2014  martin branches: 1.5.26;
Remove misplaced #endif
 1.4 22-Feb-2014  martin Try to hide the C runtime implementation specific __sync_* ops from librump,
to avoid duplicates.
 1.3 28-Apr-2008  martin branches: 1.3.4; 1.3.26;
Remove clause 3 and 4 from TNF licenses
 1.2 29-Nov-2007  ad branches: 1.2.4; 1.2.8;
Pull in m68k atomic ops from the thorpej-atomic branch.
 1.1 22-Apr-2007  thorpej branches: 1.1.2;
file atomic_op_asm.h was initially added on branch thorpej-atomic.
 1.1.2.1 22-Apr-2007  thorpej Atomic ops for 68020 and better processors. 68010 will need to use
something else for anything that uses the CAS instruction.
 1.2.8.1 18-May-2008  yamt sync with head.
 1.2.4.2 09-Jan-2008  matt sync with HEAD
 1.2.4.1 29-Nov-2007  matt file atomic_op_asm.h was added on branch matt-armv6 on 2008-01-09 01:20:57 +0000
 1.3.26.1 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.3.4.1 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.5.26.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.5.26.1 21-Apr-2020  martin Sync with HEAD
 1.12 06-Apr-2022  riastradh Nix trailing whitespace in files of membars, atomics, and lock stubs.

Will be touching many of these files soon for functional changes.

No functional change intended.
 1.11 23-Jun-2014  joerg branches: 1.11.24;
Add aliases for the builtins used to implement C11/C++11 atomics.
 1.10 22-Feb-2014  martin branches: 1.10.2;
Try to hide the C runtime implementation specific __sync_* ops from librump,
to avoid duplicates.
 1.9 20-Feb-2014  martin Add missing __sync_* ops
 1.8 16-Jul-2013  matt Add END(func)
Fix some straggling %r@<eol> to (%r)
 1.7 16-Jul-2013  matt Convert to normal Motorola syntax for addressing.
%r@ -> (%r)
%r@- -> -(%r)
%r@+ -> (%r)+
%r@(n) n(%r)
etc.
No object differences.
 1.6 25-May-2008  chs branches: 1.6.2; 1.6.24;
enable profiling of assembly functions.
 1.5 28-Apr-2008  martin branches: 1.5.2;
Remove clause 3 and 4 from TNF licenses
 1.4 12-Jan-2008  ad branches: 1.4.4;
atomic_and/atomic_or don't operate on pointers, so don't copy the return
to a0. From thorpej@.
 1.3 29-Nov-2007  ad branches: 1.3.4;
Apparently pointers are also returned in %a0.
 1.2 29-Nov-2007  ad Pull in m68k atomic ops from the thorpej-atomic branch.
 1.1 22-Apr-2007  thorpej branches: 1.1.2;
file atomic_or.S was initially added on branch thorpej-atomic.
 1.1.2.2 22-Apr-2007  thorpej Make sure namespace-cleansed aliases are avaialble for all atomic ops.
 1.1.2.1 22-Apr-2007  thorpej Atomic ops for 68020 and better processors. 68010 will need to use
something else for anything that uses the CAS instruction.
 1.3.4.3 23-Mar-2008  matt sync with HEAD
 1.3.4.2 09-Jan-2008  matt sync with HEAD
 1.3.4.1 29-Nov-2007  matt file atomic_or.S was added on branch matt-armv6 on 2008-01-09 01:20:57 +0000
 1.4.4.2 04-Jun-2008  yamt sync with head
 1.4.4.1 18-May-2008  yamt sync with head.
 1.5.2.1 23-Jun-2008  wrstuden Sync w/ -current. 34 merge conflicts to follow.
 1.6.24.1 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.6.2.1 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.10.2.1 10-Aug-2014  tls Rebase.
 1.11.24.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.11.24.1 21-Apr-2020  martin Sync with HEAD
 1.4 06-Apr-2022  riastradh Nix trailing whitespace in files of membars, atomics, and lock stubs.

Will be touching many of these files soon for functional changes.

No functional change intended.
 1.3 23-Jun-2014  joerg branches: 1.3.4; 1.3.26;
Add aliases for the builtins used to implement C11/C++11 atomics.
 1.2 22-Feb-2014  martin branches: 1.2.2; 1.2.4;
Try to hide the C runtime implementation specific __sync_* ops from librump,
to avoid duplicates.
 1.1 20-Feb-2014  martin Add missing __sync_* ops
 1.2.4.2 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.2.4.1 22-Feb-2014  yamt file atomic_sub.S was added on branch yamt-pagecache on 2014-05-22 11:26:29 +0000
 1.2.2.1 10-Aug-2014  tls Rebase.
 1.3.26.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.3.26.1 21-Apr-2020  martin Sync with HEAD
 1.3.4.2 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.3.4.1 23-Jun-2014  tls file atomic_sub.S was added on branch tls-maxphys on 2014-08-19 23:45:13 +0000
 1.10 06-Apr-2022  riastradh Nix trailing whitespace in files of membars, atomics, and lock stubs.

Will be touching many of these files soon for functional changes.

No functional change intended.
 1.9 28-Jun-2014  joerg branches: 1.9.24;
Add aliases for the C11/C++11 spelling of the CAS primitives.
 1.8 22-Feb-2014  martin branches: 1.8.2;
Try to hide the C runtime implementation specific __sync_* ops from librump,
to avoid duplicates.
 1.7 18-Feb-2014  martin Add a few __sync_* and atomic functions.
 1.6 16-Jul-2013  matt Add END(func)
Fix some straggling %r@<eol> to (%r)
 1.5 16-Jul-2013  matt Convert to normal Motorola syntax for addressing.
%r@ -> (%r)
%r@- -> -(%r)
%r@+ -> (%r)+
%r@(n) n(%r)
etc.
No object differences.
 1.4 25-May-2008  chs branches: 1.4.2; 1.4.24;
enable profiling of assembly functions.
 1.3 28-Apr-2008  martin branches: 1.3.2;
Remove clause 3 and 4 from TNF licenses
 1.2 29-Nov-2007  ad branches: 1.2.4; 1.2.8;
Pull in m68k atomic ops from the thorpej-atomic branch.
 1.1 22-Apr-2007  thorpej branches: 1.1.2;
file atomic_swap.S was initially added on branch thorpej-atomic.
 1.1.2.2 22-Apr-2007  thorpej Make sure namespace-cleansed aliases are avaialble for all atomic ops.
 1.1.2.1 22-Apr-2007  thorpej Atomic ops for 68020 and better processors. 68010 will need to use
something else for anything that uses the CAS instruction.
 1.2.8.2 04-Jun-2008  yamt sync with head
 1.2.8.1 18-May-2008  yamt sync with head.
 1.2.4.2 09-Jan-2008  matt sync with HEAD
 1.2.4.1 29-Nov-2007  matt file atomic_swap.S was added on branch matt-armv6 on 2008-01-09 01:20:58 +0000
 1.3.2.1 23-Jun-2008  wrstuden Sync w/ -current. 34 merge conflicts to follow.
 1.4.24.1 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.4.2.1 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.8.2.1 10-Aug-2014  tls Rebase.
 1.9.24.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.9.24.1 21-Apr-2020  martin Sync with HEAD
 1.4 06-Apr-2022  riastradh Nix trailing whitespace in files of membars, atomics, and lock stubs.

Will be touching many of these files soon for functional changes.

No functional change intended.
 1.3 23-Jun-2014  joerg branches: 1.3.4; 1.3.26;
Add aliases for the builtins used to implement C11/C++11 atomics.
 1.2 22-Feb-2014  martin branches: 1.2.2; 1.2.4;
Try to hide the C runtime implementation specific __sync_* ops from librump,
to avoid duplicates.
 1.1 20-Feb-2014  martin Add missing __sync_* ops
 1.2.4.2 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.2.4.1 22-Feb-2014  yamt file atomic_xor.S was added on branch yamt-pagecache on 2014-05-22 11:26:29 +0000
 1.2.2.1 10-Aug-2014  tls Rebase.
 1.3.26.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.3.26.1 21-Apr-2020  martin Sync with HEAD
 1.3.4.2 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.3.4.1 23-Jun-2014  tls file atomic_xor.S was added on branch tls-maxphys on 2014-08-19 23:45:13 +0000
 1.6 18-Mar-2014  riastradh branches: 1.6.26;
Merge riastradh-drm2 to HEAD.
 1.5 18-Jul-2013  matt Adjust for Coldfire
 1.4 16-Jul-2013  matt branches: 1.4.2;
Add END(func)
Fix some straggling %r@<eol> to (%r)
 1.3 16-Jul-2013  matt Convert to normal Motorola syntax for addressing.
%r@ -> (%r)
%r@- -> -(%r)
%r@+ -> (%r)+
%r@(n) n(%r)
etc.
No object differences.
 1.2 19-Sep-2007  he branches: 1.2.28; 1.2.34;
Improve consistency between <sys/bswap.h> and what's actually
defined by the C library for the various m68k ports by borrowing
some #ifs from the i386 port.

Also, align sun2 with the other m68k ports as to whether they
define __bswap{16,32} or bswap{16,32} in their C library (all
now define the __ variants).

This should make the m68k ports build sys/rump again, except
for sun2 which hits another problem later on.
 1.1 20-Dec-2005  christos branches: 1.1.18;
Merge libkern + libc common files. As requested by core.
 1.1.18.1 06-Nov-2007  matt sync with HEAD
 1.2.34.1 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.2.28.1 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.4.2.1 23-Jul-2013  riastradh sync with HEAD
 1.6.26.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.6.26.1 21-Apr-2020  martin Sync with HEAD
 1.6 18-Mar-2014  riastradh branches: 1.6.26;
Merge riastradh-drm2 to HEAD.
 1.5 18-Jul-2013  matt Adjust for Coldfire
 1.4 16-Jul-2013  matt branches: 1.4.2;
Add END(func)
Fix some straggling %r@<eol> to (%r)
 1.3 16-Jul-2013  matt Convert to normal Motorola syntax for addressing.
%r@ -> (%r)
%r@- -> -(%r)
%r@+ -> (%r)+
%r@(n) n(%r)
etc.
No object differences.
 1.2 19-Sep-2007  he branches: 1.2.28; 1.2.34;
Improve consistency between <sys/bswap.h> and what's actually
defined by the C library for the various m68k ports by borrowing
some #ifs from the i386 port.

Also, align sun2 with the other m68k ports as to whether they
define __bswap{16,32} or bswap{16,32} in their C library (all
now define the __ variants).

This should make the m68k ports build sys/rump again, except
for sun2 which hits another problem later on.
 1.1 20-Dec-2005  christos branches: 1.1.18;
Merge libkern + libc common files. As requested by core.
 1.1.18.1 06-Nov-2007  matt sync with HEAD
 1.2.34.1 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.2.28.1 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.4.2.1 23-Jul-2013  riastradh sync with HEAD
 1.6.26.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.6.26.1 21-Apr-2020  martin Sync with HEAD
 1.5 18-Mar-2014  riastradh branches: 1.5.26;
Merge riastradh-drm2 to HEAD.
 1.4 18-Jul-2013  matt Adjust for Coldfire
 1.3 16-Jul-2013  matt branches: 1.3.2;
Add END(func)
Fix some straggling %r@<eol> to (%r)
 1.2 16-Jul-2013  matt Convert to normal Motorola syntax for addressing.
%r@ -> (%r)
%r@- -> -(%r)
%r@+ -> (%r)+
%r@(n) n(%r)
etc.
No object differences.
 1.1 20-Dec-2005  christos branches: 1.1.50; 1.1.56;
Merge libkern + libc common files. As requested by core.
 1.1.56.1 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.1.50.1 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.3.2.1 23-Jul-2013  riastradh sync with HEAD
 1.5.26.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.5.26.1 21-Apr-2020  martin Sync with HEAD
 1.6 22-Apr-2020  rin Restrict usage of m68k assembler versions of {,u}divsi3 and {,u}divsi3 to
kernel and bootloader for 68010.

They requires a special calling convention to udivsi3, and cannot to be
mixed up in normal routines provided by libgcc or compiler_rt. Although,
there's no problem for using them in a controlled situation, i.e., kernel
and standalone programs.

Note that this does not affect at all m68k ports other than sun2, since
codes generated by gcc do not call these routines.

Assembler files are moved from common/lib/libc/arch/m68k/gen to
sys/lib/libkern/arch/m68k in order not to be compiled in libc.

Revert hack introduced to lib/libc/compiler_rt/Makefile.inc rev 1.37:
http://cvsweb.netbsd.org/bsdweb.cgi/src/lib/libc/compiler_rt/Makefile.inc#rev1.37

Proposed on port-sun2@ with no response...
(Again, this does not affect m68k ports other than sun2.)
http://mail-index.netbsd.org/port-sun2/2020/03/10/msg000102.html
 1.5 07-Sep-2013  chs branches: 1.5.26; 1.5.34;
switch to ELF naming for local labels.
 1.4 16-Jul-2013  matt Add END(func)
Fix some straggling %r@<eol> to (%r)
 1.3 16-Jul-2013  matt Convert to normal Motorola syntax for addressing.
%r@ -> (%r)
%r@- -> -(%r)
%r@+ -> (%r)+
%r@(n) n(%r)
etc.
No object differences.
 1.2 05-Jan-2006  he branches: 1.2.50; 1.2.56;
Add the m68010 versions of the integer multiply divide, modulo,
and multiply functions from the old libkern. The m68010 versions
are only compiled when __mc68010__ is defined, presently that's
only done when building for sun2.

Reviewed by christos.
 1.1 20-Dec-2005  christos Merge libkern + libc common files. As requested by core.
 1.2.56.1 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.2.50.1 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.5.34.1 25-Apr-2020  bouyer Sync with HEAD
 1.5.26.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.5.26.1 21-Apr-2020  martin Sync with HEAD
 1.8 22-Apr-2020  rin Restrict usage of m68k assembler versions of {,u}divsi3 and {,u}divsi3 to
kernel and bootloader for 68010.

They requires a special calling convention to udivsi3, and cannot to be
mixed up in normal routines provided by libgcc or compiler_rt. Although,
there's no problem for using them in a controlled situation, i.e., kernel
and standalone programs.

Note that this does not affect at all m68k ports other than sun2, since
codes generated by gcc do not call these routines.

Assembler files are moved from common/lib/libc/arch/m68k/gen to
sys/lib/libkern/arch/m68k in order not to be compiled in libc.

Revert hack introduced to lib/libc/compiler_rt/Makefile.inc rev 1.37:
http://cvsweb.netbsd.org/bsdweb.cgi/src/lib/libc/compiler_rt/Makefile.inc#rev1.37

Proposed on port-sun2@ with no response...
(Again, this does not affect m68k ports other than sun2.)
http://mail-index.netbsd.org/port-sun2/2020/03/10/msg000102.html
 1.7 04-Mar-2014  matt branches: 1.7.26; 1.7.34;
Make this coldfire compatible
 1.6 07-Sep-2013  chs switch to ELF naming for local labels.
 1.5 16-Jul-2013  matt Add END(func)
Fix some straggling %r@<eol> to (%r)
 1.4 16-Jul-2013  matt Convert to normal Motorola syntax for addressing.
%r@ -> (%r)
%r@- -> -(%r)
%r@+ -> (%r)+
%r@(n) n(%r)
etc.
No object differences.
 1.3 13-Jan-2006  christos branches: 1.3.50; 1.3.56;
fix the mc68010 case.
 1.2 05-Jan-2006  he Add the m68010 versions of the integer multiply divide, modulo,
and multiply functions from the old libkern. The m68010 versions
are only compiled when __mc68010__ is defined, presently that's
only done when building for sun2.

Reviewed by christos.
 1.1 20-Dec-2005  christos Merge libkern + libc common files. As requested by core.
 1.3.56.1 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.3.50.1 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.7.34.1 25-Apr-2020  bouyer Sync with HEAD
 1.7.26.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.7.26.1 21-Apr-2020  martin Sync with HEAD
 1.2 31-May-2020  rin Fix typo in comment.
 1.1 31-May-2020  rin Add m68k assembler version of __muldi3().

This is intended for 68060:
- GCC does not emit __muldi3() for 68020-40, that have 32 * 32 --> 64 mulul
- mulsl (and moveml), used in this code, are not implemented for 68010

In comparison with that from compiler_rt, this version saves:
- 12% of processing time
- 12 bytes of stack
- 50 bytes of code size
Also, slightly faster, memory saving, and smaller than libgcc version.

By examining with evcnt(9), __muldi3() is invoked more than 1000 times per
sec by kernel, which should justify to introduce assembler version of this
function.
 1.5 09-Mar-2020  rin Remove wrong comment (copy-paste from somewhere);
__mulsi3 does not depend on __udivsi3.
 1.4 16-Jul-2013  matt branches: 1.4.28;
Add END(func)
Fix some straggling %r@<eol> to (%r)
 1.3 16-Jul-2013  matt Convert to normal Motorola syntax for addressing.
%r@ -> (%r)
%r@- -> -(%r)
%r@+ -> (%r)+
%r@(n) n(%r)
etc.
No object differences.
 1.2 05-Jan-2006  he branches: 1.2.50; 1.2.56;
Add the m68010 versions of the integer multiply divide, modulo,
and multiply functions from the old libkern. The m68010 versions
are only compiled when __mc68010__ is defined, presently that's
only done when building for sun2.

Reviewed by christos.
 1.1 20-Dec-2005  christos Merge libkern + libc common files. As requested by core.
 1.2.56.1 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.2.50.1 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.4.28.3 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.4.28.2 21-Apr-2020  martin Sync with HEAD
 1.4.28.1 08-Apr-2020  martin Merge changes from current as of 20200406
 1.5 22-Apr-2020  rin Restrict usage of m68k assembler versions of {,u}divsi3 and {,u}divsi3 to
kernel and bootloader for 68010.

They requires a special calling convention to udivsi3, and cannot to be
mixed up in normal routines provided by libgcc or compiler_rt. Although,
there's no problem for using them in a controlled situation, i.e., kernel
and standalone programs.

Note that this does not affect at all m68k ports other than sun2, since
codes generated by gcc do not call these routines.

Assembler files are moved from common/lib/libc/arch/m68k/gen to
sys/lib/libkern/arch/m68k in order not to be compiled in libc.

Revert hack introduced to lib/libc/compiler_rt/Makefile.inc rev 1.37:
http://cvsweb.netbsd.org/bsdweb.cgi/src/lib/libc/compiler_rt/Makefile.inc#rev1.37

Proposed on port-sun2@ with no response...
(Again, this does not affect m68k ports other than sun2.)
http://mail-index.netbsd.org/port-sun2/2020/03/10/msg000102.html
 1.4 16-Jul-2013  matt branches: 1.4.28; 1.4.36;
Add END(func)
Fix some straggling %r@<eol> to (%r)
 1.3 16-Jul-2013  matt Convert to normal Motorola syntax for addressing.
%r@ -> (%r)
%r@- -> -(%r)
%r@+ -> (%r)+
%r@(n) n(%r)
etc.
No object differences.
 1.2 05-Jan-2006  he branches: 1.2.50; 1.2.56;
Add the m68010 versions of the integer multiply divide, modulo,
and multiply functions from the old libkern. The m68010 versions
are only compiled when __mc68010__ is defined, presently that's
only done when building for sun2.

Reviewed by christos.
 1.1 20-Dec-2005  christos Merge libkern + libc common files. As requested by core.
 1.2.56.1 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.2.50.1 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.4.36.1 25-Apr-2020  bouyer Sync with HEAD
 1.4.28.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.4.28.1 21-Apr-2020  martin Sync with HEAD
 1.6 22-Apr-2020  rin Restrict usage of m68k assembler versions of {,u}divsi3 and {,u}divsi3 to
kernel and bootloader for 68010.

They requires a special calling convention to udivsi3, and cannot to be
mixed up in normal routines provided by libgcc or compiler_rt. Although,
there's no problem for using them in a controlled situation, i.e., kernel
and standalone programs.

Note that this does not affect at all m68k ports other than sun2, since
codes generated by gcc do not call these routines.

Assembler files are moved from common/lib/libc/arch/m68k/gen to
sys/lib/libkern/arch/m68k in order not to be compiled in libc.

Revert hack introduced to lib/libc/compiler_rt/Makefile.inc rev 1.37:
http://cvsweb.netbsd.org/bsdweb.cgi/src/lib/libc/compiler_rt/Makefile.inc#rev1.37

Proposed on port-sun2@ with no response...
(Again, this does not affect m68k ports other than sun2.)
http://mail-index.netbsd.org/port-sun2/2020/03/10/msg000102.html
 1.5 04-Mar-2014  matt branches: 1.5.26; 1.5.34;
Make this work with coldfire.
 1.4 16-Jul-2013  matt Add END(func)
Fix some straggling %r@<eol> to (%r)
 1.3 16-Jul-2013  matt Convert to normal Motorola syntax for addressing.
%r@ -> (%r)
%r@- -> -(%r)
%r@+ -> (%r)+
%r@(n) n(%r)
etc.
No object differences.
 1.2 05-Jan-2006  he branches: 1.2.50; 1.2.56;
Add the m68010 versions of the integer multiply divide, modulo,
and multiply functions from the old libkern. The m68010 versions
are only compiled when __mc68010__ is defined, presently that's
only done when building for sun2.

Reviewed by christos.
 1.1 20-Dec-2005  christos Merge libkern + libc common files. As requested by core.
 1.2.56.1 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.2.50.1 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.5.34.1 25-Apr-2020  bouyer Sync with HEAD
 1.5.26.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.5.26.1 21-Apr-2020  martin Sync with HEAD
 1.3 16-Jul-2013  matt branches: 1.3.28;
Add END(func)
Fix some straggling %r@<eol> to (%r)
 1.2 16-Jul-2013  matt Convert to normal Motorola syntax for addressing.
%r@ -> (%r)
%r@- -> -(%r)
%r@+ -> (%r)+
%r@(n) n(%r)
etc.
No object differences.
 1.1 20-Dec-2005  christos branches: 1.1.50; 1.1.56;
Merge libkern + libc common files. As requested by core.
 1.1.56.1 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.1.50.1 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.3.28.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.3.28.1 21-Apr-2020  martin Sync with HEAD
 1.3 16-Jul-2013  matt branches: 1.3.28;
Add END(func)
Fix some straggling %r@<eol> to (%r)
 1.2 16-Jul-2013  matt Convert to normal Motorola syntax for addressing.
%r@ -> (%r)
%r@- -> -(%r)
%r@+ -> (%r)+
%r@(n) n(%r)
etc.
No object differences.
 1.1 20-Dec-2005  christos branches: 1.1.50; 1.1.56;
Merge libkern + libc common files. As requested by core.
 1.1.56.1 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.1.50.1 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.3.28.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.3.28.1 21-Apr-2020  martin Sync with HEAD
 1.3 16-Jul-2013  matt branches: 1.3.28;
Add END(func)
Fix some straggling %r@<eol> to (%r)
 1.2 16-Jul-2013  matt Convert to normal Motorola syntax for addressing.
%r@ -> (%r)
%r@- -> -(%r)
%r@+ -> (%r)+
%r@(n) n(%r)
etc.
No object differences.
 1.1 20-Dec-2005  christos branches: 1.1.50; 1.1.56;
Merge libkern + libc common files. As requested by core.
 1.1.56.1 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.1.50.1 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.3.28.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.3.28.1 21-Apr-2020  martin Sync with HEAD
 1.3 16-Jul-2013  matt branches: 1.3.28;
Add END(func)
Fix some straggling %r@<eol> to (%r)
 1.2 16-Jul-2013  matt Convert to normal Motorola syntax for addressing.
%r@ -> (%r)
%r@- -> -(%r)
%r@+ -> (%r)+
%r@(n) n(%r)
etc.
No object differences.
 1.1 20-Dec-2005  christos branches: 1.1.50; 1.1.56;
Merge libkern + libc common files. As requested by core.
 1.1.56.1 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.1.50.1 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.3.28.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.3.28.1 21-Apr-2020  martin Sync with HEAD
 1.6 07-Sep-2013  chs branches: 1.6.26;
switch to ELF naming for local labels.
 1.5 18-Jul-2013  matt Adjust for Coldfire
 1.4 16-Jul-2013  matt branches: 1.4.2;
Add END(func)
Fix some straggling %r@<eol> to (%r)
 1.3 16-Jul-2013  matt Convert to normal Motorola syntax for addressing.
%r@ -> (%r)
%r@- -> -(%r)
%r@+ -> (%r)+
%r@(n) n(%r)
etc.
No object differences.
 1.2 28-Apr-2008  martin branches: 1.2.4; 1.2.26;
Remove clause 3 and 4 from TNF licenses
 1.1 20-Dec-2005  christos branches: 1.1.26;
Merge libkern + libc common files. As requested by core.
 1.1.26.1 18-May-2008  yamt sync with head.
 1.2.26.1 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.2.4.1 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.4.2.1 23-Jul-2013  riastradh sync with HEAD
 1.6.26.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.6.26.1 21-Apr-2020  martin Sync with HEAD
 1.6 07-Sep-2013  chs branches: 1.6.26;
switch to ELF naming for local labels.
 1.5 18-Jul-2013  matt Adjust for Coldfire
 1.4 16-Jul-2013  matt branches: 1.4.2;
Add END(func)
Fix some straggling %r@<eol> to (%r)
 1.3 16-Jul-2013  matt Convert to normal Motorola syntax for addressing.
%r@ -> (%r)
%r@- -> -(%r)
%r@+ -> (%r)+
%r@(n) n(%r)
etc.
No object differences.
 1.2 28-Apr-2008  martin branches: 1.2.4; 1.2.26;
Remove clause 3 and 4 from TNF licenses
 1.1 20-Dec-2005  christos branches: 1.1.26;
Merge libkern + libc common files. As requested by core.
 1.1.26.1 18-May-2008  yamt sync with head.
 1.2.26.1 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.2.4.1 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.4.2.1 23-Jul-2013  riastradh sync with HEAD
 1.6.26.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.6.26.1 21-Apr-2020  martin Sync with HEAD
 1.6 07-Sep-2013  chs branches: 1.6.26;
switch to ELF naming for local labels.
 1.5 18-Jul-2013  matt Adjust for Coldfire
 1.4 16-Jul-2013  matt branches: 1.4.2;
Add END(func)
Fix some straggling %r@<eol> to (%r)
 1.3 16-Jul-2013  matt Convert to normal Motorola syntax for addressing.
%r@ -> (%r)
%r@- -> -(%r)
%r@+ -> (%r)+
%r@(n) n(%r)
etc.
No object differences.
 1.2 28-Apr-2008  martin branches: 1.2.4; 1.2.26;
Remove clause 3 and 4 from TNF licenses
 1.1 20-Dec-2005  christos branches: 1.1.26;
Merge libkern + libc common files. As requested by core.
 1.1.26.1 18-May-2008  yamt sync with head.
 1.2.26.1 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.2.4.1 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.4.2.1 23-Jul-2013  riastradh sync with HEAD
 1.6.26.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.6.26.1 21-Apr-2020  martin Sync with HEAD
 1.8 10-Mar-2020  rin For kernel, rename ffs to __ffssi2 rather than having a weak symbol.
This enables us to load modules depended to __ffssi2.

It is difficult to deal with weak symbols consistently in in-kernel
linker. See explanation by pgoyette on tech-kern:

http://mail-index.netbsd.org/tech-kern/2020/03/09/msg026148.html

Also, we do not currently provide ffs(9) as a kernel routine.
 1.7 09-Mar-2020  rin Add missing END() for coldfire.
 1.6 07-Sep-2013  chs branches: 1.6.26;
switch to ELF naming for local labels.
 1.5 18-Jul-2013  matt Adjust for Coldfire
 1.4 16-Jul-2013  matt branches: 1.4.2;
Add END(func)
Fix some straggling %r@<eol> to (%r)
 1.3 16-Jul-2013  matt Convert to normal Motorola syntax for addressing.
%r@ -> (%r)
%r@- -> -(%r)
%r@+ -> (%r)+
%r@(n) n(%r)
etc.
No object differences.
 1.2 09-Jul-2011  mrg branches: 1.2.2; 1.2.8;
add a weak alias for __ffssi2
 1.1 20-Dec-2005  christos Merge libkern + libc common files. As requested by core.
 1.2.8.1 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.2.2.1 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.4.2.1 23-Jul-2013  riastradh sync with HEAD
 1.6.26.3 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.6.26.2 21-Apr-2020  martin Sync with HEAD
 1.6.26.1 08-Apr-2020  martin Merge changes from current as of 20200406
 1.2 17-Jul-2009  dsl Delete files that are no longer needed.
 1.1 20-Dec-2005  christos branches: 1.1.36;
Merge libkern + libc common files. As requested by core.
 1.1.36.1 23-Jul-2009  jym Sync with HEAD.
 1.5 07-Sep-2013  chs branches: 1.5.26;
switch to ELF naming for local labels.
 1.4 18-Jul-2013  matt Adjust for Coldfire
 1.3 16-Jul-2013  matt branches: 1.3.2;
Add END(func)
Fix some straggling %r@<eol> to (%r)
 1.2 16-Jul-2013  matt Convert to normal Motorola syntax for addressing.
%r@ -> (%r)
%r@- -> -(%r)
%r@+ -> (%r)+
%r@(n) n(%r)
etc.
No object differences.
 1.1 20-Dec-2005  christos branches: 1.1.50; 1.1.56;
Merge libkern + libc common files. As requested by core.
 1.1.56.1 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.1.50.1 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.3.2.1 23-Jul-2013  riastradh sync with HEAD
 1.5.26.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.5.26.1 21-Apr-2020  martin Sync with HEAD
 1.1 20-Dec-2005  christos branches: 1.1.94;
Merge libkern + libc common files. As requested by core.
 1.1.94.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.1.94.1 21-Apr-2020  martin Sync with HEAD
 1.1 20-Dec-2005  christos branches: 1.1.94;
Merge libkern + libc common files. As requested by core.
 1.1.94.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.1.94.1 21-Apr-2020  martin Sync with HEAD
 1.7 07-Sep-2013  chs branches: 1.7.26;
switch to ELF naming for local labels.
 1.6 19-Jul-2013  matt With these changes, they now produce identical binary as before the coldfire
changes.
 1.5 18-Jul-2013  matt Adjust for Coldfire
 1.4 16-Jul-2013  matt branches: 1.4.2;
Add END(func)
Fix some straggling %r@<eol> to (%r)
 1.3 16-Jul-2013  matt Convert to normal Motorola syntax for addressing.
%r@ -> (%r)
%r@- -> -(%r)
%r@+ -> (%r)+
%r@(n) n(%r)
etc.
No object differences.
 1.2 28-Apr-2008  martin branches: 1.2.4; 1.2.26;
Remove clause 3 and 4 from TNF licenses
 1.1 20-Dec-2005  christos branches: 1.1.26;
Merge libkern + libc common files. As requested by core.
 1.1.26.1 18-May-2008  yamt sync with head.
 1.2.26.1 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.2.4.1 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.4.2.1 23-Jul-2013  riastradh sync with HEAD
 1.7.26.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.7.26.1 21-Apr-2020  martin Sync with HEAD
 1.2 17-Jul-2009  dsl Delete files that are no longer needed.
 1.1 20-Dec-2005  christos branches: 1.1.36;
Merge libkern + libc common files. As requested by core.
 1.1.36.1 23-Jul-2009  jym Sync with HEAD.
 1.4 07-Sep-2013  chs branches: 1.4.26;
switch to ELF naming for local labels.
 1.3 16-Jul-2013  matt Add END(func)
Fix some straggling %r@<eol> to (%r)
 1.2 16-Jul-2013  matt Convert to normal Motorola syntax for addressing.
%r@ -> (%r)
%r@- -> -(%r)
%r@+ -> (%r)+
%r@(n) n(%r)
etc.
No object differences.
 1.1 20-Dec-2005  christos branches: 1.1.50; 1.1.56;
Merge libkern + libc common files. As requested by core.
 1.1.56.1 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.1.50.1 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.4.26.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.4.26.1 21-Apr-2020  martin Sync with HEAD
 1.4 16-Jul-2013  matt branches: 1.4.28;
Add END(func)
Fix some straggling %r@<eol> to (%r)
 1.3 16-Jul-2013  matt Convert to normal Motorola syntax for addressing.
%r@ -> (%r)
%r@- -> -(%r)
%r@+ -> (%r)+
%r@(n) n(%r)
etc.
No object differences.
 1.2 17-Jul-2009  dsl branches: 1.2.6; 1.2.12;
Change all archs so that strchr.[cS] and strrchr.[cS] exist and generate
duplicate symbols for index() and rindex().
 1.1 20-Dec-2005  christos branches: 1.1.36;
Merge libkern + libc common files. As requested by core.
 1.1.36.1 23-Jul-2009  jym Sync with HEAD.
 1.2.12.1 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.2.6.1 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.4.28.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.4.28.1 21-Apr-2020  martin Sync with HEAD
 1.7 07-Sep-2013  chs branches: 1.7.26;
switch to ELF naming for local labels.
 1.6 19-Jul-2013  matt With these changes, they now produce identical binary as before the coldfire
changes.
 1.5 18-Jul-2013  matt Adjust for Coldfire
 1.4 16-Jul-2013  matt branches: 1.4.2;
Add END(func)
Fix some straggling %r@<eol> to (%r)
 1.3 16-Jul-2013  matt Convert to normal Motorola syntax for addressing.
%r@ -> (%r)
%r@- -> -(%r)
%r@+ -> (%r)+
%r@(n) n(%r)
etc.
No object differences.
 1.2 28-Apr-2008  martin branches: 1.2.4; 1.2.26;
Remove clause 3 and 4 from TNF licenses
 1.1 20-Dec-2005  christos branches: 1.1.26;
Merge libkern + libc common files. As requested by core.
 1.1.26.1 18-May-2008  yamt sync with head.
 1.2.26.1 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.2.4.1 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.4.2.1 23-Jul-2013  riastradh sync with HEAD
 1.7.26.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.7.26.1 21-Apr-2020  martin Sync with HEAD
 1.4 07-Sep-2013  chs branches: 1.4.26;
switch to ELF naming for local labels.
 1.3 16-Jul-2013  matt Add END(func)
Fix some straggling %r@<eol> to (%r)
 1.2 16-Jul-2013  matt Convert to normal Motorola syntax for addressing.
%r@ -> (%r)
%r@- -> -(%r)
%r@+ -> (%r)+
%r@(n) n(%r)
etc.
No object differences.
 1.1 20-Dec-2005  christos branches: 1.1.50; 1.1.56;
Merge libkern + libc common files. As requested by core.
 1.1.56.1 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.1.50.1 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.4.26.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.4.26.1 21-Apr-2020  martin Sync with HEAD
 1.4 07-Sep-2013  chs branches: 1.4.26;
switch to ELF naming for local labels.
 1.3 16-Jul-2013  matt Add END(func)
Fix some straggling %r@<eol> to (%r)
 1.2 16-Jul-2013  matt Convert to normal Motorola syntax for addressing.
%r@ -> (%r)
%r@- -> -(%r)
%r@+ -> (%r)+
%r@(n) n(%r)
etc.
No object differences.
 1.1 20-Dec-2005  christos branches: 1.1.50; 1.1.56;
Merge libkern + libc common files. As requested by core.
 1.1.56.1 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.1.50.1 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.4.26.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.4.26.1 21-Apr-2020  martin Sync with HEAD
 1.7 07-Sep-2013  chs branches: 1.7.26;
switch to ELF naming for local labels.
 1.6 19-Jul-2013  matt With these changes, they now produce identical binary as before the coldfire
changes.
 1.5 18-Jul-2013  matt Adjust for Coldfire
 1.4 16-Jul-2013  matt branches: 1.4.2;
Add END(func)
Fix some straggling %r@<eol> to (%r)
 1.3 16-Jul-2013  matt Convert to normal Motorola syntax for addressing.
%r@ -> (%r)
%r@- -> -(%r)
%r@+ -> (%r)+
%r@(n) n(%r)
etc.
No object differences.
 1.2 28-Apr-2008  martin branches: 1.2.4; 1.2.26;
Remove clause 3 and 4 from TNF licenses
 1.1 20-Dec-2005  christos branches: 1.1.26;
Merge libkern + libc common files. As requested by core.
 1.1.26.1 18-May-2008  yamt sync with head.
 1.2.26.1 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.2.4.1 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.4.2.1 23-Jul-2013  riastradh sync with HEAD
 1.7.26.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.7.26.1 21-Apr-2020  martin Sync with HEAD
 1.5 07-Sep-2013  chs branches: 1.5.26;
switch to ELF naming for local labels.
 1.4 18-Jul-2013  matt Adjust for Coldfire
 1.3 16-Jul-2013  matt branches: 1.3.2;
Add END(func)
Fix some straggling %r@<eol> to (%r)
 1.2 16-Jul-2013  matt Convert to normal Motorola syntax for addressing.
%r@ -> (%r)
%r@- -> -(%r)
%r@+ -> (%r)+
%r@(n) n(%r)
etc.
No object differences.
 1.1 20-Dec-2005  christos branches: 1.1.50; 1.1.56;
Merge libkern + libc common files. As requested by core.
 1.1.56.1 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.1.50.1 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.3.2.1 23-Jul-2013  riastradh sync with HEAD
 1.5.26.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.5.26.1 21-Apr-2020  martin Sync with HEAD
 1.4 16-Jul-2013  matt branches: 1.4.28;
Add END(func)
Fix some straggling %r@<eol> to (%r)
 1.3 16-Jul-2013  matt Convert to normal Motorola syntax for addressing.
%r@ -> (%r)
%r@- -> -(%r)
%r@+ -> (%r)+
%r@(n) n(%r)
etc.
No object differences.
 1.2 17-Jul-2009  dsl branches: 1.2.6; 1.2.12;
Change all archs so that strchr.[cS] and strrchr.[cS] exist and generate
duplicate symbols for index() and rindex().
 1.1 20-Dec-2005  christos branches: 1.1.36;
Merge libkern + libc common files. As requested by core.
 1.1.36.1 23-Jul-2009  jym Sync with HEAD.
 1.2.12.1 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.2.6.1 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.4.28.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.4.28.1 21-Apr-2020  martin Sync with HEAD
 1.16 30-Mar-2023  riastradh libc: Define __atomic_is_lock_free.

Limited to architectures where it is actually needed by gcc for any
calls to stdatomic.h atomic_is_lock_free for now.

We should also add it to other architectures too, along with lockful
atomic r/m/w operations for sizes that can't be handled natively, but
that's a lot more work. It is also necessary for -fno-inline-atomics
but we're missing a lot of other symbols for that too, to be fixed.
For now, this should enable the OpenSSL build to complete on these
architectures again after I reverted a local change.

XXX pullup-10
 1.15 25-Apr-2021  christos branches: 1.15.6;
use ${MACHINE_MIPS64}
 1.14 28-Feb-2019  isaki Add missing atomic_and_{8,16}_nv_cas.c for __sync_and_and_fetch_{1,2}.
XXX why is not only atomic_and_* symmetric unlike the others?
(in common/lib/libc/atomic/)
 1.13 13-Oct-2014  martin branches: 1.13.16;
Provide <atomic> C++ 2011 support functions for mips and sh3.
 1.12 24-Feb-2014  martin branches: 1.12.4;
Provide cas_16 and cas_8 emulation via cas_32 and use that for mips64
 1.11 21-Feb-2014  martin Provide all __sync_* ops in libc.
 1.10 14-Dec-2009  matt branches: 1.10.6; 1.10.12;
Merge from matt-nb5-mips64
 1.9 04-Jan-2009  pooka allow inclusion of atomic ops in librump
 1.8 29-Sep-2008  ad branches: 1.8.8;
Allow atomic ops to be built as part of libpthread.
 1.7 30-Apr-2008  ad Assembly _atomic_cas_up() for mips. PR lib/38482.
 1.6 11-Feb-2008  ad branches: 1.6.4;
Only build atomic ops for libkern/libc.
 1.5 10-Feb-2008  ad Enable the atomic ops in userspace.
 1.4 30-Nov-2007  ad branches: 1.4.4;
Memory barriers for MIPS.
 1.3 29-Nov-2007  ad Use the CAS-based inc/dec variants, since these CPUs don't have atomic
add in hardware (does arm?).
 1.2 29-Nov-2007  ad Make the 64-bit operations available when possible.
 1.1 29-Nov-2007  ad Atomic ops for MIPS. Use the CAS functions already provided by the kernel,
and use the generic C code to provide the rest. Unfortunatley the C code
assembles up pretty badly on MIPS but at least it will work.
 1.4.4.3 23-Mar-2008  matt sync with HEAD
 1.4.4.2 09-Jan-2008  matt sync with HEAD
 1.4.4.1 30-Nov-2007  matt file Makefile.inc was added on branch matt-armv6 on 2008-01-09 01:20:58 +0000
 1.6.4.1 18-May-2008  yamt sync with head.
 1.8.8.2 05-Sep-2009  matt Resolve some conflicts.
 1.8.8.1 05-Sep-2009  matt Enable the new atomic op routines on mips64e[bl].
 1.10.12.1 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.10.6.1 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.12.4.1 12-Nov-2014  snj Pull up following revision(s) (requested by martin in ticket #218):
common/lib/libc/arch/arm/atomic/Makefile.inc: revision 1.24-1.26
common/lib/libc/arch/hppa/atomic/Makefile.inc: revision 1.13
common/lib/libc/arch/mips/atomic/Makefile.inc: revision 1.13
common/lib/libc/arch/sh3/atomic/Makefile.inc: revision 1.7
common/lib/libc/arch/sparc/atomic/Makefile.inc: revision 1.18
common/lib/libc/arch/vax/atomic/Makefile.inc: revision 1.7
common/lib/libc/atomic/atomic_and_16_nv_cas.c: revision 1.2
common/lib/libc/atomic/atomic_and_8_nv_cas.c: revision 1.2
common/lib/libc/atomic/atomic_c11_compare_exchange_cas_16.c: revision 1.1-1.2
common/lib/libc/atomic/atomic_c11_compare_exchange_cas_32.c: revision 1.1-1.2
common/lib/libc/atomic/atomic_c11_compare_exchange_cas_8.c: revision 1.1-1.2
common/lib/libc/atomic/atomic_cas_by_cas32.c: revision 1.4
common/lib/libc/atomic/atomic_op_namespace.h: revision 1.7
Add __sync_val_compare_and_swap_{1,2} aliases for _atomic_cas_{8,16}
--
Provide __atomic_compare_exchange_N (as needed for the C11 2011 <atomic>
ops) via the corresponding CAS.
--
Hook __atomic_compare_exchange_N into vax libc.
--
Provide __sync_and_and_fetch_2 and __sync_and_and_fetch_1 for pre-ARMv6,
they are needed for the C++ 2011 <atomic> stuff.
--
Add C++ 2011 <atomic> support functions.
--
Move the and_{16,8}_nv sources into the right (libc only) block.
--
Provide <atomic> C++ 2011 support functions for mips and sh3.
--
Provide C++ 2011 <atomic> support functions for hppa and arm.
--
Provide prototypes to fix build with clang.
 1.13.16.2 21-Apr-2020  martin Sync with HEAD
 1.13.16.1 10-Jun-2019  christos Sync with HEAD
 1.15.6.1 31-Jul-2023  martin Pull up following revision(s) (requested by riastradh in ticket #275):

common/lib/libc/arch/sparc/atomic/Makefile.inc: revision 1.24
common/lib/libc/arch/m68k/atomic/Makefile.inc: revision 1.16
common/lib/libc/arch/mips/atomic/Makefile.inc: revision 1.16
common/lib/libc/arch/hppa/atomic/Makefile.inc: revision 1.15
common/lib/libc/arch/vax/atomic/Makefile.inc: revision 1.9
common/lib/libc/atomic/atomic_is_lock_free.c: revision 1.1
common/lib/libc/arch/sh3/atomic/Makefile.inc: revision 1.9

libc: Define __atomic_is_lock_free.

Limited to architectures where it is actually needed by gcc for any
calls to stdatomic.h atomic_is_lock_free for now.

We should also add it to other architectures too, along with lockful
atomic r/m/w operations for sizes that can't be handled natively, but
that's a lot more work. It is also necessary for -fno-inline-atomics
but we're missing a lot of other symbols for that too, to be fixed.

For now, this should enable the OpenSSL build to complete on these
architectures again after I reverted a local change.
 1.7 06-Aug-2020  skrll Centralise SYNC/BDSYNC in asm.h and introduce a new LLCSCSYNC and use it
before any ll/sc sequences.

Define LLSCSYNC as syncw; syncw for cnMIPS - issue two as early cnMIPS
has errat{um,a} that means the first can fail.
 1.6 01-Aug-2020  skrll Trailing whitespace
 1.5 01-Jun-2015  matt branches: 1.5.16;
Include OCTEON support for syncw and saa/saad (Store Atomic Add).
 1.4 14-Mar-2012  christos don't include <sys/cdefs.h> from assembly.
 1.3 27-Aug-2011  bouyer branches: 1.3.2;
loongson2f support:
- Add some loongson2 definitions to cpuregs.h, from OpenBSD
- Make sure that the at register is useable before every jump register
instruction (exept when register is k0 or k1) because -mfix-loongson2f-btb
needs the at register for its workaround
- add code to mips_fixup.c to handle the instructions added by
-mfix-loongson2f-btb
- Add a ls2-specific tlb miss handler: it doesn't have separate handler
for the xtlbmiss exeption.
- Fixes for some #ifdef MIPS3_LOONGSON2 assembly code (using the wrong
register)
 1.2 14-Dec-2009  matt Merge from matt-nb5-mips64
 1.1 05-Sep-2009  matt branches: 1.1.2;
file atomic_add.S was initially added on branch matt-nb5-mips64.
 1.1.2.1 05-Sep-2009  matt Add native ll/sc or lld/scd versions of the atomic ops.
 1.3.2.1 17-Apr-2012  yamt sync with head
 1.5.16.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.5.16.1 21-Apr-2020  martin Sync with HEAD
 1.6 06-Aug-2020  skrll Centralise SYNC/BDSYNC in asm.h and introduce a new LLCSCSYNC and use it
before any ll/sc sequences.

Define LLSCSYNC as syncw; syncw for cnMIPS - issue two as early cnMIPS
has errat{um,a} that means the first can fail.
 1.5 01-Aug-2020  skrll Trailing whitespace
 1.4 14-Mar-2012  christos branches: 1.4.34;
don't include <sys/cdefs.h> from assembly.
 1.3 27-Aug-2011  bouyer branches: 1.3.2;
loongson2f support:
- Add some loongson2 definitions to cpuregs.h, from OpenBSD
- Make sure that the at register is useable before every jump register
instruction (exept when register is k0 or k1) because -mfix-loongson2f-btb
needs the at register for its workaround
- add code to mips_fixup.c to handle the instructions added by
-mfix-loongson2f-btb
- Add a ls2-specific tlb miss handler: it doesn't have separate handler
for the xtlbmiss exeption.
- Fixes for some #ifdef MIPS3_LOONGSON2 assembly code (using the wrong
register)
 1.2 14-Dec-2009  matt Merge from matt-nb5-mips64
 1.1 05-Sep-2009  matt branches: 1.1.2;
file atomic_and.S was initially added on branch matt-nb5-mips64.
 1.1.2.1 05-Sep-2009  matt Add native ll/sc or lld/scd versions of the atomic ops.
 1.3.2.1 17-Apr-2012  yamt sync with head
 1.4.34.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.4.34.1 21-Apr-2020  martin Sync with HEAD
 1.9 27-Feb-2022  riastradh mips: Membar audit.

This change should be safe because it doesn't remove or weaken any
memory barriers, but does add, clarify, or strengthen barriers.

Goals:

- Make sure mutex_enter/exit and mutex_spin_enter/exit have
acquire/release semantics.

- New macros make maintenance easier and purpose clearer:

. SYNC_ACQ is for load-before-load/store barrier, and BDSYNC_ACQ
for a branch delay slot -- currently defined as plain sync for MP
and nothing, or nop, for UP; thus it is no weaker than SYNC and
BDSYNC as currently defined, which is syncw on Octeon, plain sync
on non-Octeon MP, and nothing/nop on UP.

It is not clear to me whether load-then-syncw or ll/sc-then-syncw
or even bare load provides load-acquire semantics on Octeon -- if
no, this will fix bugs; if yes (like it is on SPARC PSO), we can
relax SYNC_ACQ to be syncw or nothing later.

. SYNC_REL is for load/store-before-store barrier -- currently
defined as plain sync for MP and nothing for UP.

It is not clear to me whether syncw-then-store is enough for
store-release on Octeon -- if no, we can leave this as is; if
yes, we can relax SYNC_REL to be syncw on Octeon.

. SYNC_PLUNGER is there to flush clogged Cavium store buffers, and
BDSYNC_PLUNGER for a branch delay slot -- syncw on Octeon,
nothing or nop on non-Octeon.

=> This is not necessary (or, as far as I'm aware, sufficient)
for acquire semantics -- it serves only to flush store buffers
where stores might otherwise linger for hundreds of thousands
of cycles, which would, e.g., cause spin locks to be held for
unreasonably long durations.

Newerish revisions of the MIPS ISA also have finer-grained sync
variants that could be plopped in here.

Mechanism:

Insert these barriers in the right places, replacing only those where
the definition is currently equivalent, so this change is safe.

- Replace #ifdef _MIPS_ARCH_OCTEONP / syncw / #endif at the end of
atomic_cas_* by SYNC_PLUNGER, which is `sync 4' (a.k.a. syncw) if
__OCTEON__ and empty otherwise.

=> From what I can tell, __OCTEON__ is defined in at least as many
contexts as _MIPS_ARCH_OCTEONP -- i.e., there are some Octeons
with no _MIPS_ARCH_OCTEONP, but I don't know if any of them are
relevant to us or ever saw the light of day outside Cavium; we
seem to buid with `-march=octeonp' so this is unlikely to make a
difference. If it turns out that we do care, well, now there's
a central place to make the distinction for sync instructions.

- Replace post-ll/sc SYNC by SYNC_ACQ in _atomic_cas_*, which are
internal kernel versions used in sys/arch/mips/include/lock.h where
it assumes they have load-acquire semantics. Should move this to
lock.h later, since we _don't_ define __HAVE_ATOMIC_AS_MEMBAR on
MIPS and so the extra barrier might be costly.

- Insert SYNC_REL before ll/sc, and replace post-ll/sc SYNC by
SYNC_ACQ, in _ucas_*, which is used without any barriers in futex
code and doesn't mention barriers in the man page so I have to
assume it is required to be a release/acquire barrier.

- Change BDSYNC to BDSYNC_ACQ in mutex_enter and mutex_spin_enter.
This is necessary to provide load-acquire semantics -- unclear if
it was provided already by syncw on Octeon, but it seems more
likely that either (a) no sync or syncw is needed at all, or (b)
syncw is not enough and sync is needed, since syncw is only a
store-before-store ordering barrier.

- Insert SYNC_REL before ll/sc in mutex_exit and mutex_spin_exit.
This is currently redundant with the SYNC already there, but
SYNC_REL more clearly identifies the necessary semantics in case we
want to define it differently on different systems, and having a
sync in the middle of an ll/sc is a bit weird and possibly not a
good idea, so I intend to (carefully) remove the redundant SYNC in
a later change.

- Change BDSYNC to BDSYNC_PLUNGER at the end of mutex_exit. This has
no semantic change right now -- it's syncw on Octeon, sync on
non-Octeon MP, nop on UP -- but we can relax it later to nop on
non-Cavium MP.

- Leave LLSCSYNC in for now -- it is apparently there for a Cavium
erratum, but I'm not sure what the erratum is, exactly, and I have
no reference for it. I suspect these can be safely removed, but we
might have to double up some other syncw instructions -- Linux uses
it only in store-release sequences, not at the head of every ll/sc.
 1.8 06-Aug-2020  skrll Centralise SYNC/BDSYNC in asm.h and introduce a new LLCSCSYNC and use it
before any ll/sc sequences.

Define LLSCSYNC as syncw; syncw for cnMIPS - issue two as early cnMIPS
has errat{um,a} that means the first can fail.
 1.7 01-Aug-2020  skrll Trailing whitespace
 1.6 20-Feb-2019  rin Export atomic_cas_32_ni in a similar manner to its 64-bit counterpart.

Compile test only, but seems trivial enough for me.

Fix build error due to test/lib/libc/atomic/t_atomic_cas.

Note that mips32 does not use atomic_cas.S.
 1.5 19-Feb-2019  martin Add atomic_cas_64_ni alias
 1.4 01-Jun-2015  matt branches: 1.4.16;
Include OCTEON support for syncw and saa/saad (Store Atomic Add).
 1.3 14-Mar-2012  christos don't include <sys/cdefs.h> from assembly.
 1.2 14-Dec-2009  matt branches: 1.2.6;
Merge from matt-nb5-mips64
 1.1 05-Sep-2009  matt branches: 1.1.2;
file atomic_cas.S was initially added on branch matt-nb5-mips64.
 1.1.2.1 05-Sep-2009  matt Add native ll/sc or lld/scd versions of the atomic ops.
 1.2.6.1 17-Apr-2012  yamt sync with head
 1.4.16.3 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.4.16.2 21-Apr-2020  martin Sync with HEAD
 1.4.16.1 10-Jun-2019  christos Sync with HEAD
 1.3 01-Aug-2020  skrll Trailing whitespace
 1.2 25-May-2008  chs branches: 1.2.62;
enable profiling of assembly functions.
 1.1 30-Apr-2008  ad branches: 1.1.2; 1.1.4;
Assembly _atomic_cas_up() for mips. PR lib/38482.
 1.1.4.3 04-Jun-2008  yamt sync with head
 1.1.4.2 18-May-2008  yamt sync with head.
 1.1.4.1 30-Apr-2008  yamt file atomic_cas_up.S was added on branch yamt-pf42 on 2008-05-18 12:28:45 +0000
 1.1.2.1 23-Jun-2008  wrstuden Sync w/ -current. 34 merge conflicts to follow.
 1.2.62.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.2.62.1 21-Apr-2020  martin Sync with HEAD
 1.7 06-Aug-2020  skrll Centralise SYNC/BDSYNC in asm.h and introduce a new LLCSCSYNC and use it
before any ll/sc sequences.

Define LLSCSYNC as syncw; syncw for cnMIPS - issue two as early cnMIPS
has errat{um,a} that means the first can fail.
 1.6 01-Aug-2020  skrll Trailing whitespace
 1.5 01-Jun-2015  matt branches: 1.5.16;
Include OCTEON support for syncw and saa/saad (Store Atomic Add).
 1.4 14-Mar-2012  christos don't include <sys/cdefs.h> from assembly.
 1.3 27-Aug-2011  bouyer branches: 1.3.2;
loongson2f support:
- Add some loongson2 definitions to cpuregs.h, from OpenBSD
- Make sure that the at register is useable before every jump register
instruction (exept when register is k0 or k1) because -mfix-loongson2f-btb
needs the at register for its workaround
- add code to mips_fixup.c to handle the instructions added by
-mfix-loongson2f-btb
- Add a ls2-specific tlb miss handler: it doesn't have separate handler
for the xtlbmiss exeption.
- Fixes for some #ifdef MIPS3_LOONGSON2 assembly code (using the wrong
register)
 1.2 14-Dec-2009  matt Merge from matt-nb5-mips64
 1.1 05-Sep-2009  matt branches: 1.1.2;
file atomic_dec.S was initially added on branch matt-nb5-mips64.
 1.1.2.1 05-Sep-2009  matt Add native ll/sc or lld/scd versions of the atomic ops.
 1.3.2.1 17-Apr-2012  yamt sync with head
 1.5.16.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.5.16.1 21-Apr-2020  martin Sync with HEAD
 1.7 06-Aug-2020  skrll Centralise SYNC/BDSYNC in asm.h and introduce a new LLCSCSYNC and use it
before any ll/sc sequences.

Define LLSCSYNC as syncw; syncw for cnMIPS - issue two as early cnMIPS
has errat{um,a} that means the first can fail.
 1.6 01-Aug-2020  skrll Trailing whitespace
 1.5 01-Jun-2015  matt branches: 1.5.16;
Include OCTEON support for syncw and saa/saad (Store Atomic Add).
 1.4 14-Mar-2012  christos don't include <sys/cdefs.h> from assembly.
 1.3 27-Aug-2011  bouyer branches: 1.3.2;
loongson2f support:
- Add some loongson2 definitions to cpuregs.h, from OpenBSD
- Make sure that the at register is useable before every jump register
instruction (exept when register is k0 or k1) because -mfix-loongson2f-btb
needs the at register for its workaround
- add code to mips_fixup.c to handle the instructions added by
-mfix-loongson2f-btb
- Add a ls2-specific tlb miss handler: it doesn't have separate handler
for the xtlbmiss exeption.
- Fixes for some #ifdef MIPS3_LOONGSON2 assembly code (using the wrong
register)
 1.2 14-Dec-2009  matt Merge from matt-nb5-mips64
 1.1 05-Sep-2009  matt branches: 1.1.2;
file atomic_inc.S was initially added on branch matt-nb5-mips64.
 1.1.2.1 05-Sep-2009  matt Add native ll/sc or lld/scd versions of the atomic ops.
 1.3.2.1 17-Apr-2012  yamt sync with head
 1.5.16.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.5.16.1 21-Apr-2020  martin Sync with HEAD
 1.5 27-Feb-2022  riastradh mips: Membar audit.

This change should be safe because it doesn't remove or weaken any
memory barriers, but does add, clarify, or strengthen barriers.

Goals:

- Make sure mutex_enter/exit and mutex_spin_enter/exit have
acquire/release semantics.

- New macros make maintenance easier and purpose clearer:

. SYNC_ACQ is for load-before-load/store barrier, and BDSYNC_ACQ
for a branch delay slot -- currently defined as plain sync for MP
and nothing, or nop, for UP; thus it is no weaker than SYNC and
BDSYNC as currently defined, which is syncw on Octeon, plain sync
on non-Octeon MP, and nothing/nop on UP.

It is not clear to me whether load-then-syncw or ll/sc-then-syncw
or even bare load provides load-acquire semantics on Octeon -- if
no, this will fix bugs; if yes (like it is on SPARC PSO), we can
relax SYNC_ACQ to be syncw or nothing later.

. SYNC_REL is for load/store-before-store barrier -- currently
defined as plain sync for MP and nothing for UP.

It is not clear to me whether syncw-then-store is enough for
store-release on Octeon -- if no, we can leave this as is; if
yes, we can relax SYNC_REL to be syncw on Octeon.

. SYNC_PLUNGER is there to flush clogged Cavium store buffers, and
BDSYNC_PLUNGER for a branch delay slot -- syncw on Octeon,
nothing or nop on non-Octeon.

=> This is not necessary (or, as far as I'm aware, sufficient)
for acquire semantics -- it serves only to flush store buffers
where stores might otherwise linger for hundreds of thousands
of cycles, which would, e.g., cause spin locks to be held for
unreasonably long durations.

Newerish revisions of the MIPS ISA also have finer-grained sync
variants that could be plopped in here.

Mechanism:

Insert these barriers in the right places, replacing only those where
the definition is currently equivalent, so this change is safe.

- Replace #ifdef _MIPS_ARCH_OCTEONP / syncw / #endif at the end of
atomic_cas_* by SYNC_PLUNGER, which is `sync 4' (a.k.a. syncw) if
__OCTEON__ and empty otherwise.

=> From what I can tell, __OCTEON__ is defined in at least as many
contexts as _MIPS_ARCH_OCTEONP -- i.e., there are some Octeons
with no _MIPS_ARCH_OCTEONP, but I don't know if any of them are
relevant to us or ever saw the light of day outside Cavium; we
seem to buid with `-march=octeonp' so this is unlikely to make a
difference. If it turns out that we do care, well, now there's
a central place to make the distinction for sync instructions.

- Replace post-ll/sc SYNC by SYNC_ACQ in _atomic_cas_*, which are
internal kernel versions used in sys/arch/mips/include/lock.h where
it assumes they have load-acquire semantics. Should move this to
lock.h later, since we _don't_ define __HAVE_ATOMIC_AS_MEMBAR on
MIPS and so the extra barrier might be costly.

- Insert SYNC_REL before ll/sc, and replace post-ll/sc SYNC by
SYNC_ACQ, in _ucas_*, which is used without any barriers in futex
code and doesn't mention barriers in the man page so I have to
assume it is required to be a release/acquire barrier.

- Change BDSYNC to BDSYNC_ACQ in mutex_enter and mutex_spin_enter.
This is necessary to provide load-acquire semantics -- unclear if
it was provided already by syncw on Octeon, but it seems more
likely that either (a) no sync or syncw is needed at all, or (b)
syncw is not enough and sync is needed, since syncw is only a
store-before-store ordering barrier.

- Insert SYNC_REL before ll/sc in mutex_exit and mutex_spin_exit.
This is currently redundant with the SYNC already there, but
SYNC_REL more clearly identifies the necessary semantics in case we
want to define it differently on different systems, and having a
sync in the middle of an ll/sc is a bit weird and possibly not a
good idea, so I intend to (carefully) remove the redundant SYNC in
a later change.

- Change BDSYNC to BDSYNC_PLUNGER at the end of mutex_exit. This has
no semantic change right now -- it's syncw on Octeon, sync on
non-Octeon MP, nop on UP -- but we can relax it later to nop on
non-Cavium MP.

- Leave LLSCSYNC in for now -- it is apparently there for a Cavium
erratum, but I'm not sure what the erratum is, exactly, and I have
no reference for it. I suspect these can be safely removed, but we
might have to double up some other syncw instructions -- Linux uses
it only in store-release sequences, not at the head of every ll/sc.
 1.4 01-Aug-2020  skrll Trailing whitespace
 1.3 01-Jun-2015  matt branches: 1.3.16;
Include OCTEON support for syncw and saa/saad (Store Atomic Add).
 1.2 28-Apr-2008  martin Remove clause 3 and 4 from TNF licenses
 1.1 30-Nov-2007  ad branches: 1.1.4; 1.1.8;
Memory barriers for MIPS.
 1.1.8.1 18-May-2008  yamt sync with head.
 1.1.4.2 09-Jan-2008  matt sync with HEAD
 1.1.4.1 30-Nov-2007  matt file atomic_op_asm.h was added on branch matt-armv6 on 2008-01-09 01:20:58 +0000
 1.3.16.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.3.16.1 21-Apr-2020  martin Sync with HEAD
 1.6 06-Aug-2020  skrll Centralise SYNC/BDSYNC in asm.h and introduce a new LLCSCSYNC and use it
before any ll/sc sequences.

Define LLSCSYNC as syncw; syncw for cnMIPS - issue two as early cnMIPS
has errat{um,a} that means the first can fail.
 1.5 01-Aug-2020  skrll Trailing whitespace
 1.4 14-Mar-2012  christos branches: 1.4.34;
don't include <sys/cdefs.h> from assembly.
 1.3 27-Aug-2011  bouyer branches: 1.3.2;
loongson2f support:
- Add some loongson2 definitions to cpuregs.h, from OpenBSD
- Make sure that the at register is useable before every jump register
instruction (exept when register is k0 or k1) because -mfix-loongson2f-btb
needs the at register for its workaround
- add code to mips_fixup.c to handle the instructions added by
-mfix-loongson2f-btb
- Add a ls2-specific tlb miss handler: it doesn't have separate handler
for the xtlbmiss exeption.
- Fixes for some #ifdef MIPS3_LOONGSON2 assembly code (using the wrong
register)
 1.2 14-Dec-2009  matt Merge from matt-nb5-mips64
 1.1 05-Sep-2009  matt branches: 1.1.2;
file atomic_or.S was initially added on branch matt-nb5-mips64.
 1.1.2.1 05-Sep-2009  matt Add native ll/sc or lld/scd versions of the atomic ops.
 1.3.2.1 17-Apr-2012  yamt sync with head
 1.4.34.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.4.34.1 21-Apr-2020  martin Sync with HEAD
 1.8 27-Feb-2022  riastradh mips: Membar audit.

This change should be safe because it doesn't remove or weaken any
memory barriers, but does add, clarify, or strengthen barriers.

Goals:

- Make sure mutex_enter/exit and mutex_spin_enter/exit have
acquire/release semantics.

- New macros make maintenance easier and purpose clearer:

. SYNC_ACQ is for load-before-load/store barrier, and BDSYNC_ACQ
for a branch delay slot -- currently defined as plain sync for MP
and nothing, or nop, for UP; thus it is no weaker than SYNC and
BDSYNC as currently defined, which is syncw on Octeon, plain sync
on non-Octeon MP, and nothing/nop on UP.

It is not clear to me whether load-then-syncw or ll/sc-then-syncw
or even bare load provides load-acquire semantics on Octeon -- if
no, this will fix bugs; if yes (like it is on SPARC PSO), we can
relax SYNC_ACQ to be syncw or nothing later.

. SYNC_REL is for load/store-before-store barrier -- currently
defined as plain sync for MP and nothing for UP.

It is not clear to me whether syncw-then-store is enough for
store-release on Octeon -- if no, we can leave this as is; if
yes, we can relax SYNC_REL to be syncw on Octeon.

. SYNC_PLUNGER is there to flush clogged Cavium store buffers, and
BDSYNC_PLUNGER for a branch delay slot -- syncw on Octeon,
nothing or nop on non-Octeon.

=> This is not necessary (or, as far as I'm aware, sufficient)
for acquire semantics -- it serves only to flush store buffers
where stores might otherwise linger for hundreds of thousands
of cycles, which would, e.g., cause spin locks to be held for
unreasonably long durations.

Newerish revisions of the MIPS ISA also have finer-grained sync
variants that could be plopped in here.

Mechanism:

Insert these barriers in the right places, replacing only those where
the definition is currently equivalent, so this change is safe.

- Replace #ifdef _MIPS_ARCH_OCTEONP / syncw / #endif at the end of
atomic_cas_* by SYNC_PLUNGER, which is `sync 4' (a.k.a. syncw) if
__OCTEON__ and empty otherwise.

=> From what I can tell, __OCTEON__ is defined in at least as many
contexts as _MIPS_ARCH_OCTEONP -- i.e., there are some Octeons
with no _MIPS_ARCH_OCTEONP, but I don't know if any of them are
relevant to us or ever saw the light of day outside Cavium; we
seem to buid with `-march=octeonp' so this is unlikely to make a
difference. If it turns out that we do care, well, now there's
a central place to make the distinction for sync instructions.

- Replace post-ll/sc SYNC by SYNC_ACQ in _atomic_cas_*, which are
internal kernel versions used in sys/arch/mips/include/lock.h where
it assumes they have load-acquire semantics. Should move this to
lock.h later, since we _don't_ define __HAVE_ATOMIC_AS_MEMBAR on
MIPS and so the extra barrier might be costly.

- Insert SYNC_REL before ll/sc, and replace post-ll/sc SYNC by
SYNC_ACQ, in _ucas_*, which is used without any barriers in futex
code and doesn't mention barriers in the man page so I have to
assume it is required to be a release/acquire barrier.

- Change BDSYNC to BDSYNC_ACQ in mutex_enter and mutex_spin_enter.
This is necessary to provide load-acquire semantics -- unclear if
it was provided already by syncw on Octeon, but it seems more
likely that either (a) no sync or syncw is needed at all, or (b)
syncw is not enough and sync is needed, since syncw is only a
store-before-store ordering barrier.

- Insert SYNC_REL before ll/sc in mutex_exit and mutex_spin_exit.
This is currently redundant with the SYNC already there, but
SYNC_REL more clearly identifies the necessary semantics in case we
want to define it differently on different systems, and having a
sync in the middle of an ll/sc is a bit weird and possibly not a
good idea, so I intend to (carefully) remove the redundant SYNC in
a later change.

- Change BDSYNC to BDSYNC_PLUNGER at the end of mutex_exit. This has
no semantic change right now -- it's syncw on Octeon, sync on
non-Octeon MP, nop on UP -- but we can relax it later to nop on
non-Cavium MP.

- Leave LLSCSYNC in for now -- it is apparently there for a Cavium
erratum, but I'm not sure what the erratum is, exactly, and I have
no reference for it. I suspect these can be safely removed, but we
might have to double up some other syncw instructions -- Linux uses
it only in store-release sequences, not at the head of every ll/sc.
 1.7 06-Aug-2020  skrll Centralise SYNC/BDSYNC in asm.h and introduce a new LLCSCSYNC and use it
before any ll/sc sequences.

Define LLSCSYNC as syncw; syncw for cnMIPS - issue two as early cnMIPS
has errat{um,a} that means the first can fail.
 1.6 01-Aug-2020  skrll Trailing whitespace
 1.5 01-Jun-2015  matt branches: 1.5.16;
Include OCTEON support for syncw and saa/saad (Store Atomic Add).
 1.4 14-Mar-2012  christos don't include <sys/cdefs.h> from assembly.
 1.3 27-Aug-2011  bouyer branches: 1.3.2;
loongson2f support:
- Add some loongson2 definitions to cpuregs.h, from OpenBSD
- Make sure that the at register is useable before every jump register
instruction (exept when register is k0 or k1) because -mfix-loongson2f-btb
needs the at register for its workaround
- add code to mips_fixup.c to handle the instructions added by
-mfix-loongson2f-btb
- Add a ls2-specific tlb miss handler: it doesn't have separate handler
for the xtlbmiss exeption.
- Fixes for some #ifdef MIPS3_LOONGSON2 assembly code (using the wrong
register)
 1.2 14-Dec-2009  matt Merge from matt-nb5-mips64
 1.1 05-Sep-2009  matt branches: 1.1.2;
file atomic_swap.S was initially added on branch matt-nb5-mips64.
 1.1.2.1 05-Sep-2009  matt Add native ll/sc or lld/scd versions of the atomic ops.
 1.3.2.1 17-Apr-2012  yamt sync with head
 1.5.16.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.5.16.1 21-Apr-2020  martin Sync with HEAD
 1.13 21-Apr-2022  riastradh mips/cavium: Take advantage of Octeon's guaranteed r/rw ordering.
 1.12 09-Apr-2022  riastradh Introduce membar_acquire/release. Deprecate membar_enter/exit.

The names membar_enter/exit were unclear, and the documentation of
membar_enter has disagreed with the implementations on sparc,
powerpc, and even x86(!) for the entire time it has been in NetBSD.

The terms `acquire' and `release' are ubiquitous in the literature
today, and have been adopted in the C and C++ standards to mean
load-before-load/store and load/store-before-store, respectively,
which are exactly the orderings required by acquiring and releasing a
mutex, as well as other useful applications like decrementing a
reference count and then freeing the underlying object if it went to
zero.

Originally I proposed changing one word in the documentation for
membar_enter to make it load-before-load/store instead of
store-before-load/store, i.e., to make it an acquire barrier. I
proposed this on the grounds that

(a) all implementations guarantee load-before-load/store,
(b) some implementations fail to guarantee store-before-load/store,
and
(c) all uses in-tree assume load-before-load/store.

I verified parts (a) and (b) (except, for (a), powerpc didn't even
guarantee load-before-load/store -- isync isn't necessarily enough;
need lwsync in general -- but it _almost_ did, and it certainly didn't
guarantee store-before-load/store).

Part (c) might not be correct, however: under the mistaken assumption
that atomic-r/m/w then membar-w/rw is equivalent to atomic-r/m/w then
membar-r/rw, I only audited the cases of membar_enter that _aren't_
immediately after an atomic-r/m/w. All of those cases assume
load-before-load/store. But my assumption was wrong -- there are
cases of atomic-r/m/w then membar-w/rw that would be broken by
changing to atomic-r/m/w then membar-r/rw:

https://mail-index.netbsd.org/tech-kern/2022/03/29/msg028044.html

Furthermore, the name membar_enter has been adopted in other places
like OpenBSD where it actually does follow the documentation and
guarantee store-before-load/store, even if that order is not useful.
So the name membar_enter currently lives in a bad place where it
means either of two things -- r/rw or w/rw.

With this change, we deprecate membar_enter/exit, introduce
membar_acquire/release as better names for the useful pair (r/rw and
rw/w), and make sure the implementation of membar_enter guarantees
both what was documented _and_ what was implemented, making it an
alias for membar_sync.

While here, rework all of the membar_* definitions and aliases. The
new logic follows a rule to make it easier to audit:

membar_X is defined as an alias for membar_Y iff membar_X is
guaranteed by membar_Y.

The `no stronger than' relation is (the transitive closure of):

- membar_consumer (r/r) is guaranteed by membar_acquire (r/rw)
- membar_producer (w/w) is guaranteed by membar_release (rw/w)
- membar_acquire (r/rw) is guaranteed by membar_sync (rw/rw)
- membar_release (rw/w) is guaranteed by membar_sync (rw/rw)

And, for the deprecated membars:

- membar_enter (whether r/rw, w/rw, or rw/rw) is guaranteed by
membar_sync (rw/rw)
- membar_exit (rw/w) is guaranteed by membar_release (rw/w)

(membar_exit is identical to membar_release, but the name is
deprecated.)

Finally, while here, annotate some of the instructions with their
semantics. For powerpc, leave an essay with citations on the
unfortunate but -- as far as I can tell -- necessary decision to use
lwsync, not isync, for membar_acquire and membar_consumer.

Also add membar(3) and atomic(3) man page links.
 1.11 12-Feb-2022  riastradh mips: Brush up __cpu_simple_lock.

- Eradicate last vestiges of mb_* barriers.

- In __cpu_simple_lock_init, omit needless barrier. It is the
caller's responsibility to ensure __cpu_simple_lock_init happens
before other operations on it anyway, so there was never any need
for a barrier here.

- In __cpu_simple_lock_try, leave comments about memory ordering
guarantees of the kernel's _atomic_cas_uint, which are inexplicably
different from the non-underscored atomic_cas_uint.

- In __cpu_simple_unlock, use membar_exit instead of mb_memory, and do
it unconditionally.

This ensures that in __cpu_simple_lock/.../__cpu_simple_unlock, all
memory operations in the ellipsis happen before the store that
releases the lock.

- On Octeon, the barrier was omitted altogether, which is a bug --
it needs to be there or else there is no happens-before relation
and whoever takes the lock next might see stale values stored or
even stomp over the unlocking CPU's delayed loads.

- On non-Octeon, the mb_memory was sync. Using membar_exit
preserves this.

XXX On Octeon, membar_exit only issues syncw -- this seems wrong,
only store-before-store and not load/store-before-store, unless the
CNMIPS architecture guarantees it is sufficient here like
SPARCv8/v9 PSO (`Partial Store Order').

- Leave an essay with citations about why we have an apparently
pointless syncw _after_ releasing a lock, to work around a design
bug^W^Wquirk in cnmips which sometimes buffers stores for hundreds
of thousands of cycles for fun unless you issue syncw.
 1.10 10-Aug-2020  skrll More SYNC centralisation
 1.9 01-Aug-2020  skrll Trailing whitespace
 1.8 23-Jun-2015  matt branches: 1.8.16;
Always use sync if mips3 or later or not using O32 ABI. (A little redundant
since not using O32 means you are using mips3 or later.)
 1.7 22-Jun-2015  matt #include "assym.h"
Don't include "assym.h" with _RUMPKERNEL defined.
 1.6 01-Jun-2015  matt Include OCTEON support for syncw and saa/saad (Store Atomic Add).
 1.5 03-Aug-2012  matt Add a missing .set noreorder
 1.4 14-Dec-2009  matt branches: 1.4.6;
Merge from matt-nb5-mips64
 1.3 25-May-2008  chs branches: 1.3.10;
enable profiling of assembly functions.
 1.2 28-Apr-2008  martin branches: 1.2.2;
Remove clause 3 and 4 from TNF licenses
 1.1 30-Nov-2007  ad branches: 1.1.4; 1.1.8;
Memory barriers for MIPS.
 1.1.8.2 04-Jun-2008  yamt sync with head
 1.1.8.1 18-May-2008  yamt sync with head.
 1.1.4.2 09-Jan-2008  matt sync with HEAD
 1.1.4.1 30-Nov-2007  matt file membar_ops.S was added on branch matt-armv6 on 2008-01-09 01:20:59 +0000
 1.2.2.1 23-Jun-2008  wrstuden Sync w/ -current. 34 merge conflicts to follow.
 1.3.10.3 03-Aug-2012  matt Add missing .set noreorder
 1.3.10.2 05-Sep-2009  matt Resolve some conflicts.
 1.3.10.1 05-Sep-2009  matt Only allow to null on o32
 1.4.6.1 30-Oct-2012  yamt sync with head
 1.8.16.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.8.16.1 21-Apr-2020  martin Sync with HEAD
 1.3 14-Dec-2009  matt branches: 1.3.48;
Merge from matt-nb5-mips64
 1.2 08-Feb-2006  simonb branches: 1.2.40;
Don't rename bswap{16,32} if either _KERNEL or _STANDALONE are defined,
instead of just if _KERNEL was defined.
Fixes sbmips bootblocks build problems.
Thanks to Valeriy Ushakov for showing me where the problem was.
 1.1 20-Dec-2005  christos Merge libkern + libc common files. As requested by core.
 1.2.40.2 30-Aug-2009  matt If compiling for mips32r2 or mips64r2, use new wshb/rotr instructions.
 1.2.40.1 16-Aug-2009  matt First pass at making ABI agnostic. Generates same O32 code.
 1.3.48.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.3.48.1 21-Apr-2020  martin Sync with HEAD
 1.3 14-Dec-2009  matt branches: 1.3.48;
Merge from matt-nb5-mips64
 1.2 08-Feb-2006  simonb branches: 1.2.40;
Don't rename bswap{16,32} if either _KERNEL or _STANDALONE are defined,
instead of just if _KERNEL was defined.
Fixes sbmips bootblocks build problems.
Thanks to Valeriy Ushakov for showing me where the problem was.
 1.1 20-Dec-2005  christos Merge libkern + libc common files. As requested by core.
 1.2.40.2 30-Aug-2009  matt If compiling for mips32r2 or mips64r2, use new wshb/rotr instructions.
 1.2.40.1 16-Aug-2009  matt First pass at making ABI agnostic. Generates same O32 code.
 1.3.48.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.3.48.1 21-Apr-2020  martin Sync with HEAD
 1.2 14-Dec-2009  matt branches: 1.2.48;
Merge from matt-nb5-mips64
 1.1 30-Aug-2009  matt branches: 1.1.2;
file byte_swap_8.S was initially added on branch matt-nb5-mips64.
 1.1.2.3 05-Sep-2009  matt Add an assmebly version of bswap64
 1.1.2.2 30-Aug-2009  matt Fix typo (dshb -> dshd)
 1.1.2.1 30-Aug-2009  matt Add bswap64 implementation for mips. for mips32r2 and mips64r2, use the
new dshd/dshb and wshb/rotr instructions.
 1.2.48.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.2.48.1 21-Apr-2020  martin Sync with HEAD
 1.4 27-Aug-2011  bouyer branches: 1.4.44;
loongson2f support:
- Add some loongson2 definitions to cpuregs.h, from OpenBSD
- Make sure that the at register is useable before every jump register
instruction (exept when register is k0 or k1) because -mfix-loongson2f-btb
needs the at register for its workaround
- add code to mips_fixup.c to handle the instructions added by
-mfix-loongson2f-btb
- Add a ls2-specific tlb miss handler: it doesn't have separate handler
for the xtlbmiss exeption.
- Fixes for some #ifdef MIPS3_LOONGSON2 assembly code (using the wrong
register)
 1.3 14-Dec-2009  matt Merge from matt-nb5-mips64
 1.2 27-Dec-2005  tsutsui branches: 1.2.40;
Don't redefine _LOCORE if it's already defined.
Some Makefiles for standalone programs already have it.

XXX Old src/sys/lib/libkern/arch/mips/memcpy.S had some #ifdef MIPS3_5900
XXX which added some extra nops, but this new common bcopy.S doesn't.
 1.1 21-Dec-2005  christos move from libc.
 1.2.40.4 23-Aug-2009  matt Use #if 0 method for RCSID selection.
 1.2.40.3 20-Aug-2009  matt Fix screwup when converting. Use 64bit versions of lwr/lwl ldr/ldl as
appropriate.
 1.2.40.2 19-Aug-2009  matt Make this emit exactly the same code as before under O32.
 1.2.40.1 16-Aug-2009  matt First pass at making ABI agnostic. Generates same O32 code.
 1.4.44.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.4.44.1 21-Apr-2020  martin Sync with HEAD
 1.4 04-Jul-2011  mrg branches: 1.4.44;
add a weak alias from ffs to __ffssi2. newsmips kernels link now.
 1.3 23-Jan-2011  matt Add a new O(log(2) implementation. On mips32/mips64, use clz/dclz.
 1.2 14-Dec-2009  matt branches: 1.2.4;
Merge from matt-nb5-mips64
 1.1 20-Dec-2005  christos branches: 1.1.40;
Merge libkern + libc common files. As requested by core.
 1.1.40.3 22-Apr-2010  matt Fix mips32/mips64 versions of ffs.
 1.1.40.2 21-Apr-2010  matt sync to netbsd-5
 1.1.40.1 16-Aug-2009  matt First pass at making ABI agnostic. Generates same O32 code.
 1.2.4.1 08-Feb-2011  bouyer Sync with HEAD
 1.4.44.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.4.44.1 21-Apr-2020  martin Sync with HEAD
 1.1 20-Dec-2005  christos branches: 1.1.94;
Merge libkern + libc common files. As requested by core.
 1.1.94.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.1.94.1 21-Apr-2020  martin Sync with HEAD
 1.1 20-Dec-2005  christos branches: 1.1.94;
Merge libkern + libc common files. As requested by core.
 1.1.94.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.1.94.1 21-Apr-2020  martin Sync with HEAD
 1.2 14-Dec-2009  matt Merge from matt-nb5-mips64
 1.1 20-Dec-2005  christos branches: 1.1.40;
Merge libkern + libc common files. As requested by core.
 1.1.40.2 18-Aug-2009  matt No longer used.
 1.1.40.1 16-Aug-2009  matt First pass at making ABI agnostic. Generates same O32 code.
 1.5 25-Feb-2017  joerg branches: 1.5.12;
Switch from __ABICALLS__ to __mips_abicalls like upstream GCC does in
the generic MIPS target logic.
 1.4 26-May-2015  matt branches: 1.4.2; 1.4.4;
force 2nd to argument to unsigned byte value.
(found by t_strchr and t_strrchr tests)
 1.3 02-Jan-2011  matt Make these 64-bit clean.
 1.2 17-Jul-2009  dsl branches: 1.2.2;
Change all archs so that strchr.[cS] and strrchr.[cS] exist and generate
duplicate symbols for index() and rindex().
 1.1 16-Jul-2009  dsl Add asm files for str(r)chr by copying the files for (r)index from
src/libc/arch/mips/string.
Add XLEAF() for index entry points.
 1.2.2.2 23-Jul-2009  jym Sync with HEAD.
 1.2.2.1 17-Jul-2009  jym file strchr.S was added on branch jym-xensuspend on 2009-07-23 23:31:34 +0000
 1.4.4.1 21-Apr-2017  bouyer Sync with HEAD
 1.4.2.1 20-Mar-2017  pgoyette Sync with HEAD
 1.5.12.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.5.12.1 21-Apr-2020  martin Sync with HEAD
 1.2 14-Dec-2009  matt branches: 1.2.48;
Merge from matt-nb5-mips64
 1.1 20-Dec-2005  christos branches: 1.1.40;
Merge libkern + libc common files. As requested by core.
 1.1.40.1 16-Aug-2009  matt First pass at making ABI agnostic. Generates same O32 code.
 1.2.48.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.2.48.1 21-Apr-2020  martin Sync with HEAD
 1.2 14-Dec-2009  matt branches: 1.2.48;
Merge from matt-nb5-mips64
 1.1 20-Dec-2005  christos branches: 1.1.40;
Merge libkern + libc common files. As requested by core.
 1.1.40.1 16-Aug-2009  matt First pass at making ABI agnostic. Generates same O32 code.
 1.2.48.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.2.48.1 21-Apr-2020  martin Sync with HEAD
 1.4 25-Feb-2017  joerg branches: 1.4.12;
Switch from __ABICALLS__ to __mips_abicalls like upstream GCC does in
the generic MIPS target logic.
 1.3 26-May-2015  matt branches: 1.3.2; 1.3.4;
force 2nd to argument to unsigned byte value.
(found by t_strchr and t_strrchr tests)
 1.2 02-Jan-2011  matt Make these 64-bit clean.
 1.1 16-Jul-2009  dsl branches: 1.1.2;
Add asm files for str(r)chr by copying the files for (r)index from
src/libc/arch/mips/string.
Add XLEAF() for index entry points.
 1.1.2.2 23-Jul-2009  jym Sync with HEAD.
 1.1.2.1 16-Jul-2009  jym file strrchr.S was added on branch jym-xensuspend on 2009-07-23 23:31:34 +0000
 1.3.4.1 21-Apr-2017  bouyer Sync with HEAD
 1.3.2.1 20-Mar-2017  pgoyette Sync with HEAD
 1.4.12.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.4.12.1 21-Apr-2020  martin Sync with HEAD
 1.1 03-Sep-2014  matt branches: 1.1.16;
New files for OR1K support
 1.1.16.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.1.16.1 21-Apr-2020  martin Sync with HEAD
 1.1 03-Sep-2014  matt branches: 1.1.16;
New files for OR1K support
 1.1.16.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.1.16.1 21-Apr-2020  martin Sync with HEAD
 1.1 03-Sep-2014  matt branches: 1.1.16;
New files for OR1K support
 1.1.16.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.1.16.1 21-Apr-2020  martin Sync with HEAD
 1.1 03-Sep-2014  matt branches: 1.1.16;
New files for OR1K support
 1.1.16.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.1.16.1 21-Apr-2020  martin Sync with HEAD
 1.1 03-Sep-2014  matt branches: 1.1.16;
New files for OR1K support
 1.1.16.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.1.16.1 21-Apr-2020  martin Sync with HEAD
 1.1 03-Sep-2014  matt branches: 1.1.16;
New files for OR1K support
 1.1.16.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.1.16.1 21-Apr-2020  martin Sync with HEAD
 1.1 03-Sep-2014  matt branches: 1.1.16;
New files for OR1K support
 1.1.16.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.1.16.1 21-Apr-2020  martin Sync with HEAD
 1.1 03-Sep-2014  matt branches: 1.1.16;
New files for OR1K support
 1.1.16.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.1.16.1 21-Apr-2020  martin Sync with HEAD
 1.1 03-Sep-2014  matt branches: 1.1.16;
New files for OR1K support
 1.1.16.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.1.16.1 21-Apr-2020  martin Sync with HEAD
 1.2 06-Apr-2022  riastradh Nix trailing whitespace in files of membars, atomics, and lock stubs.

Will be touching many of these files soon for functional changes.

No functional change intended.
 1.1 03-Sep-2014  matt branches: 1.1.16;
New files for OR1K support
 1.1.16.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.1.16.1 21-Apr-2020  martin Sync with HEAD
 1.1 03-Sep-2014  matt branches: 1.1.16;
New files for OR1K support
 1.1.16.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.1.16.1 21-Apr-2020  martin Sync with HEAD
 1.2 09-Apr-2022  riastradh Introduce membar_acquire/release. Deprecate membar_enter/exit.

The names membar_enter/exit were unclear, and the documentation of
membar_enter has disagreed with the implementations on sparc,
powerpc, and even x86(!) for the entire time it has been in NetBSD.

The terms `acquire' and `release' are ubiquitous in the literature
today, and have been adopted in the C and C++ standards to mean
load-before-load/store and load/store-before-store, respectively,
which are exactly the orderings required by acquiring and releasing a
mutex, as well as other useful applications like decrementing a
reference count and then freeing the underlying object if it went to
zero.

Originally I proposed changing one word in the documentation for
membar_enter to make it load-before-load/store instead of
store-before-load/store, i.e., to make it an acquire barrier. I
proposed this on the grounds that

(a) all implementations guarantee load-before-load/store,
(b) some implementations fail to guarantee store-before-load/store,
and
(c) all uses in-tree assume load-before-load/store.

I verified parts (a) and (b) (except, for (a), powerpc didn't even
guarantee load-before-load/store -- isync isn't necessarily enough;
need lwsync in general -- but it _almost_ did, and it certainly didn't
guarantee store-before-load/store).

Part (c) might not be correct, however: under the mistaken assumption
that atomic-r/m/w then membar-w/rw is equivalent to atomic-r/m/w then
membar-r/rw, I only audited the cases of membar_enter that _aren't_
immediately after an atomic-r/m/w. All of those cases assume
load-before-load/store. But my assumption was wrong -- there are
cases of atomic-r/m/w then membar-w/rw that would be broken by
changing to atomic-r/m/w then membar-r/rw:

https://mail-index.netbsd.org/tech-kern/2022/03/29/msg028044.html

Furthermore, the name membar_enter has been adopted in other places
like OpenBSD where it actually does follow the documentation and
guarantee store-before-load/store, even if that order is not useful.
So the name membar_enter currently lives in a bad place where it
means either of two things -- r/rw or w/rw.

With this change, we deprecate membar_enter/exit, introduce
membar_acquire/release as better names for the useful pair (r/rw and
rw/w), and make sure the implementation of membar_enter guarantees
both what was documented _and_ what was implemented, making it an
alias for membar_sync.

While here, rework all of the membar_* definitions and aliases. The
new logic follows a rule to make it easier to audit:

membar_X is defined as an alias for membar_Y iff membar_X is
guaranteed by membar_Y.

The `no stronger than' relation is (the transitive closure of):

- membar_consumer (r/r) is guaranteed by membar_acquire (r/rw)
- membar_producer (w/w) is guaranteed by membar_release (rw/w)
- membar_acquire (r/rw) is guaranteed by membar_sync (rw/rw)
- membar_release (rw/w) is guaranteed by membar_sync (rw/rw)

And, for the deprecated membars:

- membar_enter (whether r/rw, w/rw, or rw/rw) is guaranteed by
membar_sync (rw/rw)
- membar_exit (rw/w) is guaranteed by membar_release (rw/w)

(membar_exit is identical to membar_release, but the name is
deprecated.)

Finally, while here, annotate some of the instructions with their
semantics. For powerpc, leave an essay with citations on the
unfortunate but -- as far as I can tell -- necessary decision to use
lwsync, not isync, for membar_acquire and membar_consumer.

Also add membar(3) and atomic(3) man page links.
 1.1 03-Sep-2014  matt branches: 1.1.16;
New files for OR1K support
 1.1.16.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.1.16.1 21-Apr-2020  martin Sync with HEAD
 1.1 03-Sep-2014  matt branches: 1.1.16;
New files for OR1K support
 1.1.16.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.1.16.1 21-Apr-2020  martin Sync with HEAD
 1.1 03-Sep-2014  matt branches: 1.1.16;
New files for OR1K support
 1.1.16.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.1.16.1 21-Apr-2020  martin Sync with HEAD
 1.1 03-Sep-2014  matt branches: 1.1.16;
New files for OR1K support
 1.1.16.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.1.16.1 21-Apr-2020  martin Sync with HEAD
 1.1 03-Sep-2014  matt branches: 1.1.16;
New files for OR1K support
 1.1.16.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.1.16.1 21-Apr-2020  martin Sync with HEAD
 1.1 03-Sep-2014  matt branches: 1.1.16;
New files for OR1K support
 1.1.16.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.1.16.1 21-Apr-2020  martin Sync with HEAD
 1.1 03-Sep-2014  matt branches: 1.1.16;
New files for OR1K support
 1.1.16.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.1.16.1 21-Apr-2020  martin Sync with HEAD
 1.1 03-Sep-2014  matt branches: 1.1.16;
New files for OR1K support
 1.1.16.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.1.16.1 21-Apr-2020  martin Sync with HEAD
 1.1 03-Sep-2014  matt branches: 1.1.16;
New files for OR1K support
 1.1.16.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.1.16.1 21-Apr-2020  martin Sync with HEAD
 1.1 03-Sep-2014  matt branches: 1.1.16;
New files for OR1K support
 1.1.16.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.1.16.1 21-Apr-2020  martin Sync with HEAD
 1.1 03-Sep-2014  matt branches: 1.1.16;
New files for OR1K support
 1.1.16.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.1.16.1 21-Apr-2020  martin Sync with HEAD
 1.1 03-Sep-2014  matt branches: 1.1.16;
New files for OR1K support
 1.1.16.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.1.16.1 21-Apr-2020  martin Sync with HEAD
 1.1 03-Sep-2014  matt branches: 1.1.16;
New files for OR1K support
 1.1.16.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.1.16.1 21-Apr-2020  martin Sync with HEAD
 1.6 04-Jan-2009  pooka branches: 1.6.50;
allow inclusion of atomic ops in librump
 1.5 29-Sep-2008  ad Allow atomic ops to be built as part of libpthread.
 1.4 11-Feb-2008  ad Only build atomic ops for libkern/libc.
 1.3 10-Feb-2008  ad Enable the atomic ops in userspace.
 1.2 28-Nov-2007  ad branches: 1.2.4;
Atomic ops for powerpc. Please review.
 1.1 26-Apr-2007  thorpej branches: 1.1.2;
file Makefile.inc was initially added on branch thorpej-atomic.
 1.1.2.1 26-Apr-2007  thorpej W-I-P atomic ops for powerpc. No memory barrier ops yet.
 1.2.4.3 23-Mar-2008  matt sync with HEAD
 1.2.4.2 09-Jan-2008  matt sync with HEAD
 1.2.4.1 28-Nov-2007  matt file Makefile.inc was added on branch matt-armv6 on 2008-01-09 01:20:59 +0000
 1.6.50.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.6.50.1 21-Apr-2020  martin Sync with HEAD
 1.9 06-Apr-2022  riastradh Nix trailing whitespace in files of membars, atomics, and lock stubs.

Will be touching many of these files soon for functional changes.

No functional change intended.
 1.8 07-Mar-2014  matt branches: 1.8.26;
Reduce duplication.
Use beqlr to quicken returns
 1.7 15-Jan-2011  matt branches: 1.7.6; 1.7.12;
Use END(foo) everywhere.
Make __cerror hidden.
Use non-PLT calls to __cerror.
Use assym.h when appropriate.
Use addi to adjust stack instead of loading it.
Add __RCSIDs
Force -D_NOREGNAMES for all .S files.
[this is all in preperation for secure plt support]
 1.6 25-May-2008  chs enable profiling of assembly functions.
 1.5 28-Apr-2008  martin branches: 1.5.2;
Remove clause 3 and 4 from TNF licenses
 1.4 13-Feb-2008  dogcow branches: 1.4.4;
add missing %. (hi matt!)
 1.3 12-Feb-2008  matt Use %rN as all new code is supposed to.
 1.2 28-Nov-2007  ad branches: 1.2.4;
Atomic ops for powerpc. Please review.
 1.1 26-Apr-2007  thorpej branches: 1.1.2;
file atomic_add.S was initially added on branch thorpej-atomic.
 1.1.2.2 26-Apr-2007  thorpej First-cut at powerpc64 atomic ops.
 1.1.2.1 26-Apr-2007  thorpej W-I-P atomic ops for powerpc. No memory barrier ops yet.
 1.2.4.3 23-Mar-2008  matt sync with HEAD
 1.2.4.2 09-Jan-2008  matt sync with HEAD
 1.2.4.1 28-Nov-2007  matt file atomic_add.S was added on branch matt-armv6 on 2008-01-09 01:21:00 +0000
 1.4.4.2 04-Jun-2008  yamt sync with head
 1.4.4.1 18-May-2008  yamt sync with head.
 1.5.2.1 23-Jun-2008  wrstuden Sync w/ -current. 34 merge conflicts to follow.
 1.7.12.1 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.7.6.1 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.8.26.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.8.26.1 21-Apr-2020  martin Sync with HEAD
 1.8 06-Apr-2022  riastradh Nix trailing whitespace in files of membars, atomics, and lock stubs.

Will be touching many of these files soon for functional changes.

No functional change intended.
 1.7 07-Mar-2014  matt branches: 1.7.26;
Reduce duplication.
Use beqlr to quicken returns
 1.6 15-Jan-2011  matt branches: 1.6.6; 1.6.12;
Use END(foo) everywhere.
Make __cerror hidden.
Use non-PLT calls to __cerror.
Use assym.h when appropriate.
Use addi to adjust stack instead of loading it.
Add __RCSIDs
Force -D_NOREGNAMES for all .S files.
[this is all in preperation for secure plt support]
 1.5 25-May-2008  chs enable profiling of assembly functions.
 1.4 28-Apr-2008  martin branches: 1.4.2;
Remove clause 3 and 4 from TNF licenses
 1.3 12-Feb-2008  matt branches: 1.3.4;
Use %rN as all new code is supposed to.
 1.2 28-Nov-2007  ad branches: 1.2.4;
Atomic ops for powerpc. Please review.
 1.1 26-Apr-2007  thorpej branches: 1.1.2;
file atomic_and.S was initially added on branch thorpej-atomic.
 1.1.2.2 26-Apr-2007  thorpej First-cut at powerpc64 atomic ops.
 1.1.2.1 26-Apr-2007  thorpej W-I-P atomic ops for powerpc. No memory barrier ops yet.
 1.2.4.3 23-Mar-2008  matt sync with HEAD
 1.2.4.2 09-Jan-2008  matt sync with HEAD
 1.2.4.1 28-Nov-2007  matt file atomic_and.S was added on branch matt-armv6 on 2008-01-09 01:21:00 +0000
 1.3.4.2 04-Jun-2008  yamt sync with head
 1.3.4.1 18-May-2008  yamt sync with head.
 1.4.2.1 23-Jun-2008  wrstuden Sync w/ -current. 34 merge conflicts to follow.
 1.6.12.1 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.6.6.1 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.7.26.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.7.26.1 21-Apr-2020  martin Sync with HEAD
 1.10 06-Apr-2022  riastradh Nix trailing whitespace in files of membars, atomics, and lock stubs.

Will be touching many of these files soon for functional changes.

No functional change intended.
 1.9 01-Mar-2020  rin Implement workaround for IBM405 Errata 77 (aka CPU_210), where
interrupted stwcx. may errantly write data to memory:

https://elinux.org/images/1/1d/Ppc405gp-errata.pdf

This is because stwcx. is split into two pieces in the pipeline.

We need to
(1) insert dcbt before every stwcx. instruction, as well as
(2) insert sync before every rfi/rfci instruction.

It is unclear which processors are affected, but according to Linux,
all 405-based cores up until 405GPR and 405EP are affected:

https://github.com/torvalds/linux/blob/master/arch/powerpc/platforms/40x/Kconfig#L140

For kernel, this workaround can be restricted to affected processors.
However, for kernel modules and userland, we have to enable it for all
32bit powerpc archs in order to share common binaries as before.

Proposed on port-powerpc:

http://mail-index.netbsd.org/port-powerpc/2020/02/21/msg003583.html
 1.8 07-Mar-2014  matt branches: 1.8.26; 1.8.28;
Reduce duplication.
Use beqlr to quicken returns
 1.7 15-Jan-2011  matt branches: 1.7.6; 1.7.12;
Use END(foo) everywhere.
Make __cerror hidden.
Use non-PLT calls to __cerror.
Use assym.h when appropriate.
Use addi to adjust stack instead of loading it.
Add __RCSIDs
Force -D_NOREGNAMES for all .S files.
[this is all in preperation for secure plt support]
 1.6 25-May-2008  chs enable profiling of assembly functions.
 1.5 28-Apr-2008  martin branches: 1.5.2;
Remove clause 3 and 4 from TNF licenses
 1.4 12-Feb-2008  matt branches: 1.4.4;
Use %rN as all new code is supposed to.
 1.3 10-Feb-2008  ad Add atomic_cas_foo_ni().
 1.2 28-Nov-2007  ad branches: 1.2.4;
Atomic ops for powerpc. Please review.
 1.1 26-Apr-2007  thorpej branches: 1.1.2;
file atomic_cas.S was initially added on branch thorpej-atomic.
 1.1.2.2 26-Apr-2007  thorpej First-cut at powerpc64 atomic ops.
 1.1.2.1 26-Apr-2007  thorpej W-I-P atomic ops for powerpc. No memory barrier ops yet.
 1.2.4.3 23-Mar-2008  matt sync with HEAD
 1.2.4.2 09-Jan-2008  matt sync with HEAD
 1.2.4.1 28-Nov-2007  matt file atomic_cas.S was added on branch matt-armv6 on 2008-01-09 01:21:00 +0000
 1.4.4.2 04-Jun-2008  yamt sync with head
 1.4.4.1 18-May-2008  yamt sync with head.
 1.5.2.1 23-Jun-2008  wrstuden Sync w/ -current. 34 merge conflicts to follow.
 1.7.12.1 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.7.6.1 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.8.28.1 03-Mar-2020  martin Pull up following revision(s) (requested by rin in ticket #755):

sys/arch/evbppc/conf/std.virtex: revision 1.4
sys/arch/powerpc/powerpc/trap_subr.S: revision 1.81
sys/arch/powerpc/ibm4xx/4xx_trap_subr.S: revision 1.8
sys/arch/evbppc/conf/std.walnut: revision 1.9
common/lib/libc/arch/powerpc/atomic/atomic_op_asm.h: revision 1.7
sys/arch/powerpc/include/asm.h: revision 1.49
common/lib/libc/arch/powerpc/atomic/atomic_cas.S: revision 1.9
sys/arch/powerpc/ibm4xx/trap_subr.S: revision 1.28
sys/arch/powerpc/include/lock.h: revision 1.15
sys/arch/evbppc/conf/std.obs266: revision 1.3
common/lib/libc/arch/powerpc/atomic/atomic_swap.S: revision 1.8
sys/arch/powerpc/powerpc/locore_subr.S: revision 1.61
sys/arch/powerpc/powerpc/lock_stubs.S: revision 1.12
sys/arch/evbppc/conf/std.obs200: revision 1.5

Implement workaround for IBM405 Errata 77 (aka CPU_210), where
interrupted stwcx. may errantly write data to memory:

https://elinux.org/images/1/1d/Ppc405gp-errata.pdf

This is because stwcx. is split into two pieces in the pipeline.

We need to
(1) insert dcbt before every stwcx. instruction, as well as
(2) insert sync before every rfi/rfci instruction.

It is unclear which processors are affected, but according to Linux,
all 405-based cores up until 405GPR and 405EP are affected:

https://github.com/torvalds/linux/blob/master/arch/powerpc/platforms/40x/Kconfig#L140

For kernel, this workaround can be restricted to affected processors.

However, for kernel modules and userland, we have to enable it for all
32bit powerpc archs in order to share common binaries as before.
Proposed on port-powerpc:

http://mail-index.netbsd.org/port-powerpc/2020/02/21/msg003583.html
 1.8.26.2 21-Apr-2020  martin Sync with HEAD
 1.8.26.1 08-Apr-2020  martin Merge changes from current as of 20200406
 1.8 06-Apr-2022  riastradh Nix trailing whitespace in files of membars, atomics, and lock stubs.

Will be touching many of these files soon for functional changes.

No functional change intended.
 1.7 07-Mar-2014  matt branches: 1.7.26;
Reduce duplication.
Use beqlr to quicken returns
 1.6 15-Jan-2011  matt branches: 1.6.6; 1.6.12;
Use END(foo) everywhere.
Make __cerror hidden.
Use non-PLT calls to __cerror.
Use assym.h when appropriate.
Use addi to adjust stack instead of loading it.
Add __RCSIDs
Force -D_NOREGNAMES for all .S files.
[this is all in preperation for secure plt support]
 1.5 25-May-2008  chs enable profiling of assembly functions.
 1.4 28-Apr-2008  martin branches: 1.4.2;
Remove clause 3 and 4 from TNF licenses
 1.3 12-Feb-2008  matt branches: 1.3.4;
Use %rN as all new code is supposed to.
 1.2 28-Nov-2007  ad branches: 1.2.4;
Atomic ops for powerpc. Please review.
 1.1 26-Apr-2007  thorpej branches: 1.1.2;
file atomic_dec.S was initially added on branch thorpej-atomic.
 1.1.2.2 26-Apr-2007  thorpej First-cut at powerpc64 atomic ops.
 1.1.2.1 26-Apr-2007  thorpej W-I-P atomic ops for powerpc. No memory barrier ops yet.
 1.2.4.3 23-Mar-2008  matt sync with HEAD
 1.2.4.2 09-Jan-2008  matt sync with HEAD
 1.2.4.1 28-Nov-2007  matt file atomic_dec.S was added on branch matt-armv6 on 2008-01-09 01:21:01 +0000
 1.3.4.2 04-Jun-2008  yamt sync with head
 1.3.4.1 18-May-2008  yamt sync with head.
 1.4.2.1 23-Jun-2008  wrstuden Sync w/ -current. 34 merge conflicts to follow.
 1.6.12.1 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.6.6.1 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.7.26.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.7.26.1 21-Apr-2020  martin Sync with HEAD
 1.8 06-Apr-2022  riastradh Nix trailing whitespace in files of membars, atomics, and lock stubs.

Will be touching many of these files soon for functional changes.

No functional change intended.
 1.7 07-Mar-2014  matt branches: 1.7.26;
Reduce duplication.
Use beqlr to quicken returns
 1.6 15-Jan-2011  matt branches: 1.6.6; 1.6.12;
Use END(foo) everywhere.
Make __cerror hidden.
Use non-PLT calls to __cerror.
Use assym.h when appropriate.
Use addi to adjust stack instead of loading it.
Add __RCSIDs
Force -D_NOREGNAMES for all .S files.
[this is all in preperation for secure plt support]
 1.5 25-May-2008  chs enable profiling of assembly functions.
 1.4 28-Apr-2008  martin branches: 1.4.2;
Remove clause 3 and 4 from TNF licenses
 1.3 12-Feb-2008  matt branches: 1.3.4;
Use %rN as all new code is supposed to.
 1.2 28-Nov-2007  ad branches: 1.2.4;
Atomic ops for powerpc. Please review.
 1.1 26-Apr-2007  thorpej branches: 1.1.2;
file atomic_inc.S was initially added on branch thorpej-atomic.
 1.1.2.2 26-Apr-2007  thorpej First-cut at powerpc64 atomic ops.
 1.1.2.1 26-Apr-2007  thorpej W-I-P atomic ops for powerpc. No memory barrier ops yet.
 1.2.4.3 23-Mar-2008  matt sync with HEAD
 1.2.4.2 09-Jan-2008  matt sync with HEAD
 1.2.4.1 28-Nov-2007  matt file atomic_inc.S was added on branch matt-armv6 on 2008-01-09 01:21:01 +0000
 1.3.4.2 04-Jun-2008  yamt sync with head
 1.3.4.1 18-May-2008  yamt sync with head.
 1.4.2.1 23-Jun-2008  wrstuden Sync w/ -current. 34 merge conflicts to follow.
 1.6.12.1 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.6.6.1 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.7.26.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.7.26.1 21-Apr-2020  martin Sync with HEAD
 1.8 06-Apr-2022  riastradh Nix trailing whitespace in files of membars, atomics, and lock stubs.

Will be touching many of these files soon for functional changes.

No functional change intended.
 1.7 01-Mar-2020  rin Implement workaround for IBM405 Errata 77 (aka CPU_210), where
interrupted stwcx. may errantly write data to memory:

https://elinux.org/images/1/1d/Ppc405gp-errata.pdf

This is because stwcx. is split into two pieces in the pipeline.

We need to
(1) insert dcbt before every stwcx. instruction, as well as
(2) insert sync before every rfi/rfci instruction.

It is unclear which processors are affected, but according to Linux,
all 405-based cores up until 405GPR and 405EP are affected:

https://github.com/torvalds/linux/blob/master/arch/powerpc/platforms/40x/Kconfig#L140

For kernel, this workaround can be restricted to affected processors.
However, for kernel modules and userland, we have to enable it for all
32bit powerpc archs in order to share common binaries as before.

Proposed on port-powerpc:

http://mail-index.netbsd.org/port-powerpc/2020/02/21/msg003583.html
 1.6 07-Mar-2014  matt branches: 1.6.26; 1.6.28;
Reduce duplication.
Use beqlr to quicken returns
 1.5 15-Jan-2011  matt branches: 1.5.6; 1.5.12;
Use END(foo) everywhere.
Make __cerror hidden.
Use non-PLT calls to __cerror.
Use assym.h when appropriate.
Use addi to adjust stack instead of loading it.
Add __RCSIDs
Force -D_NOREGNAMES for all .S files.
[this is all in preperation for secure plt support]
 1.4 28-Apr-2008  martin Remove clause 3 and 4 from TNF licenses
 1.3 12-Feb-2008  matt branches: 1.3.4;
Use %rN as all new code is supposed to.
 1.2 28-Nov-2007  ad branches: 1.2.4;
Atomic ops for powerpc. Please review.
 1.1 26-Apr-2007  thorpej branches: 1.1.2;
file atomic_op_asm.h was initially added on branch thorpej-atomic.
 1.1.2.1 26-Apr-2007  thorpej W-I-P atomic ops for powerpc. No memory barrier ops yet.
 1.2.4.3 23-Mar-2008  matt sync with HEAD
 1.2.4.2 09-Jan-2008  matt sync with HEAD
 1.2.4.1 28-Nov-2007  matt file atomic_op_asm.h was added on branch matt-armv6 on 2008-01-09 01:21:01 +0000
 1.3.4.1 18-May-2008  yamt sync with head.
 1.5.12.1 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.5.6.1 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.6.28.1 03-Mar-2020  martin Pull up following revision(s) (requested by rin in ticket #755):

sys/arch/evbppc/conf/std.virtex: revision 1.4
sys/arch/powerpc/powerpc/trap_subr.S: revision 1.81
sys/arch/powerpc/ibm4xx/4xx_trap_subr.S: revision 1.8
sys/arch/evbppc/conf/std.walnut: revision 1.9
common/lib/libc/arch/powerpc/atomic/atomic_op_asm.h: revision 1.7
sys/arch/powerpc/include/asm.h: revision 1.49
common/lib/libc/arch/powerpc/atomic/atomic_cas.S: revision 1.9
sys/arch/powerpc/ibm4xx/trap_subr.S: revision 1.28
sys/arch/powerpc/include/lock.h: revision 1.15
sys/arch/evbppc/conf/std.obs266: revision 1.3
common/lib/libc/arch/powerpc/atomic/atomic_swap.S: revision 1.8
sys/arch/powerpc/powerpc/locore_subr.S: revision 1.61
sys/arch/powerpc/powerpc/lock_stubs.S: revision 1.12
sys/arch/evbppc/conf/std.obs200: revision 1.5

Implement workaround for IBM405 Errata 77 (aka CPU_210), where
interrupted stwcx. may errantly write data to memory:

https://elinux.org/images/1/1d/Ppc405gp-errata.pdf

This is because stwcx. is split into two pieces in the pipeline.

We need to
(1) insert dcbt before every stwcx. instruction, as well as
(2) insert sync before every rfi/rfci instruction.

It is unclear which processors are affected, but according to Linux,
all 405-based cores up until 405GPR and 405EP are affected:

https://github.com/torvalds/linux/blob/master/arch/powerpc/platforms/40x/Kconfig#L140

For kernel, this workaround can be restricted to affected processors.

However, for kernel modules and userland, we have to enable it for all
32bit powerpc archs in order to share common binaries as before.
Proposed on port-powerpc:

http://mail-index.netbsd.org/port-powerpc/2020/02/21/msg003583.html
 1.6.26.2 21-Apr-2020  martin Sync with HEAD
 1.6.26.1 08-Apr-2020  martin Merge changes from current as of 20200406
 1.8 06-Apr-2022  riastradh Nix trailing whitespace in files of membars, atomics, and lock stubs.

Will be touching many of these files soon for functional changes.

No functional change intended.
 1.7 07-Mar-2014  matt branches: 1.7.26;
Reduce duplication.
Use beqlr to quicken returns
 1.6 15-Jan-2011  matt branches: 1.6.6; 1.6.12;
Use END(foo) everywhere.
Make __cerror hidden.
Use non-PLT calls to __cerror.
Use assym.h when appropriate.
Use addi to adjust stack instead of loading it.
Add __RCSIDs
Force -D_NOREGNAMES for all .S files.
[this is all in preperation for secure plt support]
 1.5 25-May-2008  chs enable profiling of assembly functions.
 1.4 28-Apr-2008  martin branches: 1.4.2;
Remove clause 3 and 4 from TNF licenses
 1.3 12-Feb-2008  matt branches: 1.3.4;
Use %rN as all new code is supposed to.
 1.2 28-Nov-2007  ad branches: 1.2.4;
Atomic ops for powerpc. Please review.
 1.1 26-Apr-2007  thorpej branches: 1.1.2;
file atomic_or.S was initially added on branch thorpej-atomic.
 1.1.2.2 26-Apr-2007  thorpej First-cut at powerpc64 atomic ops.
 1.1.2.1 26-Apr-2007  thorpej W-I-P atomic ops for powerpc. No memory barrier ops yet.
 1.2.4.3 23-Mar-2008  matt sync with HEAD
 1.2.4.2 09-Jan-2008  matt sync with HEAD
 1.2.4.1 28-Nov-2007  matt file atomic_or.S was added on branch matt-armv6 on 2008-01-09 01:21:02 +0000
 1.3.4.2 04-Jun-2008  yamt sync with head
 1.3.4.1 18-May-2008  yamt sync with head.
 1.4.2.1 23-Jun-2008  wrstuden Sync w/ -current. 34 merge conflicts to follow.
 1.6.12.1 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.6.6.1 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.7.26.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.7.26.1 21-Apr-2020  martin Sync with HEAD
 1.9 06-Apr-2022  riastradh Nix trailing whitespace in files of membars, atomics, and lock stubs.

Will be touching many of these files soon for functional changes.

No functional change intended.
 1.8 01-Mar-2020  rin Implement workaround for IBM405 Errata 77 (aka CPU_210), where
interrupted stwcx. may errantly write data to memory:

https://elinux.org/images/1/1d/Ppc405gp-errata.pdf

This is because stwcx. is split into two pieces in the pipeline.

We need to
(1) insert dcbt before every stwcx. instruction, as well as
(2) insert sync before every rfi/rfci instruction.

It is unclear which processors are affected, but according to Linux,
all 405-based cores up until 405GPR and 405EP are affected:

https://github.com/torvalds/linux/blob/master/arch/powerpc/platforms/40x/Kconfig#L140

For kernel, this workaround can be restricted to affected processors.
However, for kernel modules and userland, we have to enable it for all
32bit powerpc archs in order to share common binaries as before.

Proposed on port-powerpc:

http://mail-index.netbsd.org/port-powerpc/2020/02/21/msg003583.html
 1.7 07-Mar-2014  matt branches: 1.7.26; 1.7.28;
Reduce duplication.
Use beqlr to quicken returns
 1.6 15-Jan-2011  matt branches: 1.6.6; 1.6.12;
Use END(foo) everywhere.
Make __cerror hidden.
Use non-PLT calls to __cerror.
Use assym.h when appropriate.
Use addi to adjust stack instead of loading it.
Add __RCSIDs
Force -D_NOREGNAMES for all .S files.
[this is all in preperation for secure plt support]
 1.5 25-May-2008  chs enable profiling of assembly functions.
 1.4 28-Apr-2008  martin branches: 1.4.2;
Remove clause 3 and 4 from TNF licenses
 1.3 12-Feb-2008  matt branches: 1.3.4;
Use %rN as all new code is supposed to.
 1.2 28-Nov-2007  ad branches: 1.2.4;
Atomic ops for powerpc. Please review.
 1.1 26-Apr-2007  thorpej branches: 1.1.2;
file atomic_swap.S was initially added on branch thorpej-atomic.
 1.1.2.2 26-Apr-2007  thorpej First-cut at powerpc64 atomic ops.
 1.1.2.1 26-Apr-2007  thorpej W-I-P atomic ops for powerpc. No memory barrier ops yet.
 1.2.4.3 23-Mar-2008  matt sync with HEAD
 1.2.4.2 09-Jan-2008  matt sync with HEAD
 1.2.4.1 28-Nov-2007  matt file atomic_swap.S was added on branch matt-armv6 on 2008-01-09 01:21:02 +0000
 1.3.4.2 04-Jun-2008  yamt sync with head
 1.3.4.1 18-May-2008  yamt sync with head.
 1.4.2.1 23-Jun-2008  wrstuden Sync w/ -current. 34 merge conflicts to follow.
 1.6.12.1 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.6.6.1 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.7.28.1 03-Mar-2020  martin Pull up following revision(s) (requested by rin in ticket #755):

sys/arch/evbppc/conf/std.virtex: revision 1.4
sys/arch/powerpc/powerpc/trap_subr.S: revision 1.81
sys/arch/powerpc/ibm4xx/4xx_trap_subr.S: revision 1.8
sys/arch/evbppc/conf/std.walnut: revision 1.9
common/lib/libc/arch/powerpc/atomic/atomic_op_asm.h: revision 1.7
sys/arch/powerpc/include/asm.h: revision 1.49
common/lib/libc/arch/powerpc/atomic/atomic_cas.S: revision 1.9
sys/arch/powerpc/ibm4xx/trap_subr.S: revision 1.28
sys/arch/powerpc/include/lock.h: revision 1.15
sys/arch/evbppc/conf/std.obs266: revision 1.3
common/lib/libc/arch/powerpc/atomic/atomic_swap.S: revision 1.8
sys/arch/powerpc/powerpc/locore_subr.S: revision 1.61
sys/arch/powerpc/powerpc/lock_stubs.S: revision 1.12
sys/arch/evbppc/conf/std.obs200: revision 1.5

Implement workaround for IBM405 Errata 77 (aka CPU_210), where
interrupted stwcx. may errantly write data to memory:

https://elinux.org/images/1/1d/Ppc405gp-errata.pdf

This is because stwcx. is split into two pieces in the pipeline.

We need to
(1) insert dcbt before every stwcx. instruction, as well as
(2) insert sync before every rfi/rfci instruction.

It is unclear which processors are affected, but according to Linux,
all 405-based cores up until 405GPR and 405EP are affected:

https://github.com/torvalds/linux/blob/master/arch/powerpc/platforms/40x/Kconfig#L140

For kernel, this workaround can be restricted to affected processors.

However, for kernel modules and userland, we have to enable it for all
32bit powerpc archs in order to share common binaries as before.
Proposed on port-powerpc:

http://mail-index.netbsd.org/port-powerpc/2020/02/21/msg003583.html
 1.7.26.2 21-Apr-2020  martin Sync with HEAD
 1.7.26.1 08-Apr-2020  martin Merge changes from current as of 20200406
 1.6 09-Apr-2022  riastradh Introduce membar_acquire/release. Deprecate membar_enter/exit.

The names membar_enter/exit were unclear, and the documentation of
membar_enter has disagreed with the implementations on sparc,
powerpc, and even x86(!) for the entire time it has been in NetBSD.

The terms `acquire' and `release' are ubiquitous in the literature
today, and have been adopted in the C and C++ standards to mean
load-before-load/store and load/store-before-store, respectively,
which are exactly the orderings required by acquiring and releasing a
mutex, as well as other useful applications like decrementing a
reference count and then freeing the underlying object if it went to
zero.

Originally I proposed changing one word in the documentation for
membar_enter to make it load-before-load/store instead of
store-before-load/store, i.e., to make it an acquire barrier. I
proposed this on the grounds that

(a) all implementations guarantee load-before-load/store,
(b) some implementations fail to guarantee store-before-load/store,
and
(c) all uses in-tree assume load-before-load/store.

I verified parts (a) and (b) (except, for (a), powerpc didn't even
guarantee load-before-load/store -- isync isn't necessarily enough;
need lwsync in general -- but it _almost_ did, and it certainly didn't
guarantee store-before-load/store).

Part (c) might not be correct, however: under the mistaken assumption
that atomic-r/m/w then membar-w/rw is equivalent to atomic-r/m/w then
membar-r/rw, I only audited the cases of membar_enter that _aren't_
immediately after an atomic-r/m/w. All of those cases assume
load-before-load/store. But my assumption was wrong -- there are
cases of atomic-r/m/w then membar-w/rw that would be broken by
changing to atomic-r/m/w then membar-r/rw:

https://mail-index.netbsd.org/tech-kern/2022/03/29/msg028044.html

Furthermore, the name membar_enter has been adopted in other places
like OpenBSD where it actually does follow the documentation and
guarantee store-before-load/store, even if that order is not useful.
So the name membar_enter currently lives in a bad place where it
means either of two things -- r/rw or w/rw.

With this change, we deprecate membar_enter/exit, introduce
membar_acquire/release as better names for the useful pair (r/rw and
rw/w), and make sure the implementation of membar_enter guarantees
both what was documented _and_ what was implemented, making it an
alias for membar_sync.

While here, rework all of the membar_* definitions and aliases. The
new logic follows a rule to make it easier to audit:

membar_X is defined as an alias for membar_Y iff membar_X is
guaranteed by membar_Y.

The `no stronger than' relation is (the transitive closure of):

- membar_consumer (r/r) is guaranteed by membar_acquire (r/rw)
- membar_producer (w/w) is guaranteed by membar_release (rw/w)
- membar_acquire (r/rw) is guaranteed by membar_sync (rw/rw)
- membar_release (rw/w) is guaranteed by membar_sync (rw/rw)

And, for the deprecated membars:

- membar_enter (whether r/rw, w/rw, or rw/rw) is guaranteed by
membar_sync (rw/rw)
- membar_exit (rw/w) is guaranteed by membar_release (rw/w)

(membar_exit is identical to membar_release, but the name is
deprecated.)

Finally, while here, annotate some of the instructions with their
semantics. For powerpc, leave an essay with citations on the
unfortunate but -- as far as I can tell -- necessary decision to use
lwsync, not isync, for membar_acquire and membar_consumer.

Also add membar(3) and atomic(3) man page links.
 1.5 06-Apr-2022  riastradh Nix trailing whitespace in files of membars, atomics, and lock stubs.

Will be touching many of these files soon for functional changes.

No functional change intended.
 1.4 15-Jan-2011  matt branches: 1.4.48;
Use END(foo) everywhere.
Make __cerror hidden.
Use non-PLT calls to __cerror.
Use assym.h when appropriate.
Use addi to adjust stack instead of loading it.
Add __RCSIDs
Force -D_NOREGNAMES for all .S files.
[this is all in preperation for secure plt support]
 1.3 25-May-2008  chs enable profiling of assembly functions.
 1.2 28-Apr-2008  martin branches: 1.2.2;
Remove clause 3 and 4 from TNF licenses
 1.1 28-Nov-2007  ad branches: 1.1.4; 1.1.8;
Atomic ops for powerpc. Please review.
 1.1.8.2 04-Jun-2008  yamt sync with head
 1.1.8.1 18-May-2008  yamt sync with head.
 1.1.4.2 09-Jan-2008  matt sync with HEAD
 1.1.4.1 28-Nov-2007  matt file membar_ops.S was added on branch matt-armv6 on 2008-01-09 01:21:02 +0000
 1.2.2.1 23-Jun-2008  wrstuden Sync w/ -current. 34 merge conflicts to follow.
 1.4.48.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.4.48.1 21-Apr-2020  martin Sync with HEAD
 1.1 21-Feb-2008  garbled branches: 1.1.2; 1.1.4; 1.1.70;
Add tuned powerpc assembler written by IBM and released under a 3-clause
BSD Lisc as part of the perflib project.
http://sourceforge.net/projects/ppcperflib/

Tested the new functions with microbenchmarks on a number of different
CPU types, and found that most cpus either benefited greatly, or were
unaffected. Primarily G4 CPU's were unaffected, and all others showed
speedups. My 7044 (POWER3) went from a 70.6 to a 73.2 (thats good) in
bytebench with a complete release built with these. Also passed
regression tests.
 1.1.70.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.1.70.1 21-Apr-2020  martin Sync with HEAD
 1.1.4.2 24-Mar-2008  keiichi sync with head.
 1.1.4.1 21-Feb-2008  keiichi file bcopy.S was added on branch keiichi-mipv6 on 2008-03-24 07:14:29 +0000
 1.1.2.2 23-Mar-2008  matt sync with HEAD
 1.1.2.1 21-Feb-2008  matt file bcopy.S was added on branch matt-armv6 on 2008-03-23 00:12:42 +0000
 1.5 15-Jan-2011  matt branches: 1.5.48;
Use END(foo) everywhere.
Make __cerror hidden.
Use non-PLT calls to __cerror.
Use assym.h when appropriate.
Use addi to adjust stack instead of loading it.
Add __RCSIDs
Force -D_NOREGNAMES for all .S files.
[this is all in preperation for secure plt support]
 1.4 07-Jul-2006  ross revert
 1.3 07-Jul-2006  ross And yet more .S files...
 1.2 23-Dec-2005  skrll Fix PR 32363 by defining _NOREGNAMES.

Someone should complete the transition.
 1.1 20-Dec-2005  christos Merge libkern + libc common files. As requested by core.
 1.5.48.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.5.48.1 21-Apr-2020  martin Sync with HEAD
 1.4 04-Jul-2013  matt branches: 1.4.28;
remove branch prediction hint from bdnzt
 1.3 15-Jan-2011  matt branches: 1.3.6; 1.3.12;
Use END(foo) everywhere.
Make __cerror hidden.
Use non-PLT calls to __cerror.
Use assym.h when appropriate.
Use addi to adjust stack instead of loading it.
Add __RCSIDs
Force -D_NOREGNAMES for all .S files.
[this is all in preperation for secure plt support]
 1.2 06-Mar-2008  phx branches: 1.2.2; 1.2.4;
Including assym.h is not needed and will break the build, because it does not
exist that early. Deleted it.
Approved by garbled.
 1.1 21-Feb-2008  garbled Add tuned powerpc assembler written by IBM and released under a 3-clause
BSD Lisc as part of the perflib project.
http://sourceforge.net/projects/ppcperflib/

Tested the new functions with microbenchmarks on a number of different
CPU types, and found that most cpus either benefited greatly, or were
unaffected. Primarily G4 CPU's were unaffected, and all others showed
speedups. My 7044 (POWER3) went from a 70.6 to a 73.2 (thats good) in
bytebench with a complete release built with these. Also passed
regression tests.
 1.2.4.2 24-Mar-2008  keiichi sync with head.
 1.2.4.1 06-Mar-2008  keiichi file memcmp.S was added on branch keiichi-mipv6 on 2008-03-24 07:14:29 +0000
 1.2.2.2 23-Mar-2008  matt sync with HEAD
 1.2.2.1 06-Mar-2008  matt file memcmp.S was added on branch matt-armv6 on 2008-03-23 00:12:42 +0000
 1.3.12.1 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.3.6.1 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.4.28.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.4.28.1 21-Apr-2020  martin Sync with HEAD
 1.6 04-Mar-2014  macallan branches: 1.6.26;
on 601 CPUs skip to normal memcpy if both source and destination are 32bit
aligned
 1.5 03-Mar-2014  matt Use _KERNEL_OPT around #include "opt_ppcarch.h"
 1.4 03-Mar-2014  macallan on ppc601 do byte-wise copies when in _KERNEL
from scole_mail, ok matt@
 1.3 15-Jan-2011  matt branches: 1.3.6; 1.3.12;
Use END(foo) everywhere.
Make __cerror hidden.
Use non-PLT calls to __cerror.
Use assym.h when appropriate.
Use addi to adjust stack instead of loading it.
Add __RCSIDs
Force -D_NOREGNAMES for all .S files.
[this is all in preperation for secure plt support]
 1.2 06-Mar-2008  phx branches: 1.2.2; 1.2.4;
Including assym.h is not needed and will break the build, because it does not
exist that early. Deleted it.
Approved by garbled.
 1.1 21-Feb-2008  garbled Add tuned powerpc assembler written by IBM and released under a 3-clause
BSD Lisc as part of the perflib project.
http://sourceforge.net/projects/ppcperflib/

Tested the new functions with microbenchmarks on a number of different
CPU types, and found that most cpus either benefited greatly, or were
unaffected. Primarily G4 CPU's were unaffected, and all others showed
speedups. My 7044 (POWER3) went from a 70.6 to a 73.2 (thats good) in
bytebench with a complete release built with these. Also passed
regression tests.
 1.2.4.2 24-Mar-2008  keiichi sync with head.
 1.2.4.1 06-Mar-2008  keiichi file memcpy.S was added on branch keiichi-mipv6 on 2008-03-24 07:14:29 +0000
 1.2.2.2 23-Mar-2008  matt sync with HEAD
 1.2.2.1 06-Mar-2008  matt file memcpy.S was added on branch matt-armv6 on 2008-03-23 00:12:43 +0000
 1.3.12.1 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.3.6.1 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.6.26.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.6.26.1 21-Apr-2020  martin Sync with HEAD
 1.3 15-Jan-2011  matt branches: 1.3.48;
Use END(foo) everywhere.
Make __cerror hidden.
Use non-PLT calls to __cerror.
Use assym.h when appropriate.
Use addi to adjust stack instead of loading it.
Add __RCSIDs
Force -D_NOREGNAMES for all .S files.
[this is all in preperation for secure plt support]
 1.2 06-Mar-2008  phx branches: 1.2.2; 1.2.4;
Including assym.h is not needed and will break the build, because it does not
exist that early. Deleted it.
Approved by garbled.
 1.1 21-Feb-2008  garbled Add tuned powerpc assembler written by IBM and released under a 3-clause
BSD Lisc as part of the perflib project.
http://sourceforge.net/projects/ppcperflib/

Tested the new functions with microbenchmarks on a number of different
CPU types, and found that most cpus either benefited greatly, or were
unaffected. Primarily G4 CPU's were unaffected, and all others showed
speedups. My 7044 (POWER3) went from a 70.6 to a 73.2 (thats good) in
bytebench with a complete release built with these. Also passed
regression tests.
 1.2.4.2 24-Mar-2008  keiichi sync with head.
 1.2.4.1 06-Mar-2008  keiichi file memmove.S was added on branch keiichi-mipv6 on 2008-03-24 07:14:29 +0000
 1.2.2.2 23-Mar-2008  matt sync with HEAD
 1.2.2.1 06-Mar-2008  matt file memmove.S was added on branch matt-armv6 on 2008-03-23 00:12:43 +0000
 1.3.48.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.3.48.1 21-Apr-2020  martin Sync with HEAD
 1.6 15-Jan-2011  matt branches: 1.6.48;
Use END(foo) everywhere.
Make __cerror hidden.
Use non-PLT calls to __cerror.
Use assym.h when appropriate.
Use addi to adjust stack instead of loading it.
Add __RCSIDs
Force -D_NOREGNAMES for all .S files.
[this is all in preperation for secure plt support]
 1.5 15-Aug-2006  ross Fix bug involving addresses >= 4G on ppc64.
 1.4 07-Jul-2006  ross revert
 1.3 07-Jul-2006  ross And yet more .S files...
 1.2 23-Dec-2005  skrll Fix PR 32363 by defining _NOREGNAMES.

Someone should complete the transition.
 1.1 20-Dec-2005  christos Merge libkern + libc common files. As requested by core.
 1.6.48.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.6.48.1 21-Apr-2020  martin Sync with HEAD
 1.7 07-Mar-2014  matt branches: 1.7.26;
Now the sources in ../../arch/powerpc
 1.6 04-Jan-2009  pooka branches: 1.6.8; 1.6.14;
allow inclusion of atomic ops in librump
 1.5 29-Sep-2008  ad Allow atomic ops to be built as part of libpthread.
 1.4 11-Feb-2008  ad Only build atomic ops for libkern/libc.
 1.3 10-Feb-2008  ad Enable the atomic ops in userspace.
 1.2 28-Nov-2007  ad branches: 1.2.4;
ppc64 atomic ops.
 1.1 26-Apr-2007  thorpej branches: 1.1.2;
file Makefile.inc was initially added on branch thorpej-atomic.
 1.1.2.1 26-Apr-2007  thorpej First-cut at powerpc64 atomic ops.
 1.2.4.3 23-Mar-2008  matt sync with HEAD
 1.2.4.2 09-Jan-2008  matt sync with HEAD
 1.2.4.1 28-Nov-2007  matt file Makefile.inc was added on branch matt-armv6 on 2008-01-09 01:21:03 +0000
 1.6.14.1 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.6.8.1 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.7.26.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.7.26.1 21-Apr-2020  martin Sync with HEAD
 1.6 07-Mar-2014  matt Use the sources in ../../arch/powerpc instead
 1.5 08-Mar-2009  he branches: 1.5.6; 1.5.12;
Follow up the powerpc side by using %rN instead of rN for registers.
 1.4 25-May-2008  chs branches: 1.4.6;
enable profiling of assembly functions.
 1.3 28-Apr-2008  martin branches: 1.3.2;
Remove clause 3 and 4 from TNF licenses
 1.2 28-Nov-2007  ad branches: 1.2.4; 1.2.8;
ppc64 atomic ops.
 1.1 26-Apr-2007  thorpej branches: 1.1.2;
file atomic_add.S was initially added on branch thorpej-atomic.
 1.1.2.1 26-Apr-2007  thorpej First-cut at powerpc64 atomic ops.
 1.2.8.2 04-Jun-2008  yamt sync with head
 1.2.8.1 18-May-2008  yamt sync with head.
 1.2.4.2 09-Jan-2008  matt sync with HEAD
 1.2.4.1 28-Nov-2007  matt file atomic_add.S was added on branch matt-armv6 on 2008-01-09 01:21:03 +0000
 1.3.2.1 23-Jun-2008  wrstuden Sync w/ -current. 34 merge conflicts to follow.
 1.4.6.1 13-May-2009  jym Sync with HEAD.

Commit is split, to avoid a "too many arguments" protocol error.
 1.5.12.1 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.5.6.1 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.6 07-Mar-2014  matt Use the sources in ../../arch/powerpc instead
 1.5 08-Mar-2009  he branches: 1.5.6; 1.5.12;
Follow up the powerpc side by using %rN instead of rN for registers.
 1.4 25-May-2008  chs branches: 1.4.6;
enable profiling of assembly functions.
 1.3 28-Apr-2008  martin branches: 1.3.2;
Remove clause 3 and 4 from TNF licenses
 1.2 28-Nov-2007  ad branches: 1.2.4; 1.2.8;
ppc64 atomic ops.
 1.1 26-Apr-2007  thorpej branches: 1.1.2;
file atomic_and.S was initially added on branch thorpej-atomic.
 1.1.2.1 26-Apr-2007  thorpej First-cut at powerpc64 atomic ops.
 1.2.8.2 04-Jun-2008  yamt sync with head
 1.2.8.1 18-May-2008  yamt sync with head.
 1.2.4.2 09-Jan-2008  matt sync with HEAD
 1.2.4.1 28-Nov-2007  matt file atomic_and.S was added on branch matt-armv6 on 2008-01-09 01:21:04 +0000
 1.3.2.1 23-Jun-2008  wrstuden Sync w/ -current. 34 merge conflicts to follow.
 1.4.6.1 13-May-2009  jym Sync with HEAD.

Commit is split, to avoid a "too many arguments" protocol error.
 1.5.12.1 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.5.6.1 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.7 07-Mar-2014  matt Use the sources in ../../arch/powerpc instead
 1.6 08-Mar-2009  he branches: 1.6.6; 1.6.12;
Follow up the powerpc side by using %rN instead of rN for registers.
 1.5 25-May-2008  chs branches: 1.5.6;
enable profiling of assembly functions.
 1.4 28-Apr-2008  martin branches: 1.4.2;
Remove clause 3 and 4 from TNF licenses
 1.3 10-Feb-2008  ad branches: 1.3.4;
Add atomic_cas_foo_ni().
 1.2 28-Nov-2007  ad branches: 1.2.4;
ppc64 atomic ops.
 1.1 26-Apr-2007  thorpej branches: 1.1.2;
file atomic_cas.S was initially added on branch thorpej-atomic.
 1.1.2.1 26-Apr-2007  thorpej First-cut at powerpc64 atomic ops.
 1.2.4.3 23-Mar-2008  matt sync with HEAD
 1.2.4.2 09-Jan-2008  matt sync with HEAD
 1.2.4.1 28-Nov-2007  matt file atomic_cas.S was added on branch matt-armv6 on 2008-01-09 01:21:04 +0000
 1.3.4.2 04-Jun-2008  yamt sync with head
 1.3.4.1 18-May-2008  yamt sync with head.
 1.4.2.1 23-Jun-2008  wrstuden Sync w/ -current. 34 merge conflicts to follow.
 1.5.6.1 13-May-2009  jym Sync with HEAD.

Commit is split, to avoid a "too many arguments" protocol error.
 1.6.12.1 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.6.6.1 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.6 07-Mar-2014  matt Use the sources in ../../arch/powerpc instead
 1.5 08-Mar-2009  he branches: 1.5.6; 1.5.12;
Follow up the powerpc side by using %rN instead of rN for registers.
 1.4 25-May-2008  chs branches: 1.4.6;
enable profiling of assembly functions.
 1.3 28-Apr-2008  martin branches: 1.3.2;
Remove clause 3 and 4 from TNF licenses
 1.2 28-Nov-2007  ad branches: 1.2.4; 1.2.8;
ppc64 atomic ops.
 1.1 26-Apr-2007  thorpej branches: 1.1.2;
file atomic_dec.S was initially added on branch thorpej-atomic.
 1.1.2.1 26-Apr-2007  thorpej First-cut at powerpc64 atomic ops.
 1.2.8.2 04-Jun-2008  yamt sync with head
 1.2.8.1 18-May-2008  yamt sync with head.
 1.2.4.2 09-Jan-2008  matt sync with HEAD
 1.2.4.1 28-Nov-2007  matt file atomic_dec.S was added on branch matt-armv6 on 2008-01-09 01:21:04 +0000
 1.3.2.1 23-Jun-2008  wrstuden Sync w/ -current. 34 merge conflicts to follow.
 1.4.6.1 13-May-2009  jym Sync with HEAD.

Commit is split, to avoid a "too many arguments" protocol error.
 1.5.12.1 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.5.6.1 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.6 07-Mar-2014  matt Use the sources in ../../arch/powerpc instead
 1.5 08-Mar-2009  he branches: 1.5.6; 1.5.12;
Follow up the powerpc side by using %rN instead of rN for registers.
 1.4 25-May-2008  chs branches: 1.4.6;
enable profiling of assembly functions.
 1.3 28-Apr-2008  martin branches: 1.3.2;
Remove clause 3 and 4 from TNF licenses
 1.2 28-Nov-2007  ad branches: 1.2.4; 1.2.8;
ppc64 atomic ops.
 1.1 26-Apr-2007  thorpej branches: 1.1.2;
file atomic_inc.S was initially added on branch thorpej-atomic.
 1.1.2.1 26-Apr-2007  thorpej First-cut at powerpc64 atomic ops.
 1.2.8.2 04-Jun-2008  yamt sync with head
 1.2.8.1 18-May-2008  yamt sync with head.
 1.2.4.2 09-Jan-2008  matt sync with HEAD
 1.2.4.1 28-Nov-2007  matt file atomic_inc.S was added on branch matt-armv6 on 2008-01-09 01:21:05 +0000
 1.3.2.1 23-Jun-2008  wrstuden Sync w/ -current. 34 merge conflicts to follow.
 1.4.6.1 13-May-2009  jym Sync with HEAD.

Commit is split, to avoid a "too many arguments" protocol error.
 1.5.12.1 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.5.6.1 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.5 07-Mar-2014  matt Use the sources in ../../arch/powerpc instead
 1.4 22-Mar-2010  mrg branches: 1.4.6; 1.4.12;
define _NOREGNAMES before including <machine/asm.h>, like powerpc does.

from dennis.c.ferguson@gmail.com in PR#43022.
 1.3 28-Apr-2008  martin Remove clause 3 and 4 from TNF licenses
 1.2 28-Nov-2007  ad branches: 1.2.4; 1.2.8;
ppc64 atomic ops.
 1.1 26-Apr-2007  thorpej branches: 1.1.2;
file atomic_op_asm.h was initially added on branch thorpej-atomic.
 1.1.2.1 26-Apr-2007  thorpej First-cut at powerpc64 atomic ops.
 1.2.8.1 18-May-2008  yamt sync with head.
 1.2.4.2 09-Jan-2008  matt sync with HEAD
 1.2.4.1 28-Nov-2007  matt file atomic_op_asm.h was added on branch matt-armv6 on 2008-01-09 01:21:05 +0000
 1.4.12.1 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.4.6.1 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.6 07-Mar-2014  matt Use the sources in ../../arch/powerpc instead
 1.5 08-Mar-2009  he branches: 1.5.6; 1.5.12;
Follow up the powerpc side by using %rN instead of rN for registers.
 1.4 25-May-2008  chs branches: 1.4.6;
enable profiling of assembly functions.
 1.3 28-Apr-2008  martin branches: 1.3.2;
Remove clause 3 and 4 from TNF licenses
 1.2 28-Nov-2007  ad branches: 1.2.4; 1.2.8;
ppc64 atomic ops.
 1.1 26-Apr-2007  thorpej branches: 1.1.2;
file atomic_or.S was initially added on branch thorpej-atomic.
 1.1.2.1 26-Apr-2007  thorpej First-cut at powerpc64 atomic ops.
 1.2.8.2 04-Jun-2008  yamt sync with head
 1.2.8.1 18-May-2008  yamt sync with head.
 1.2.4.2 09-Jan-2008  matt sync with HEAD
 1.2.4.1 28-Nov-2007  matt file atomic_or.S was added on branch matt-armv6 on 2008-01-09 01:21:05 +0000
 1.3.2.1 23-Jun-2008  wrstuden Sync w/ -current. 34 merge conflicts to follow.
 1.4.6.1 13-May-2009  jym Sync with HEAD.

Commit is split, to avoid a "too many arguments" protocol error.
 1.5.12.1 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.5.6.1 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.6 07-Mar-2014  matt Use the sources in ../../arch/powerpc instead
 1.5 08-Mar-2009  he branches: 1.5.6; 1.5.12;
Follow up the powerpc side by using %rN instead of rN for registers.
 1.4 25-May-2008  chs branches: 1.4.6;
enable profiling of assembly functions.
 1.3 28-Apr-2008  martin branches: 1.3.2;
Remove clause 3 and 4 from TNF licenses
 1.2 28-Nov-2007  ad branches: 1.2.4; 1.2.8;
ppc64 atomic ops.
 1.1 26-Apr-2007  thorpej branches: 1.1.2;
file atomic_swap.S was initially added on branch thorpej-atomic.
 1.1.2.1 26-Apr-2007  thorpej First-cut at powerpc64 atomic ops.
 1.2.8.2 04-Jun-2008  yamt sync with head
 1.2.8.1 18-May-2008  yamt sync with head.
 1.2.4.2 09-Jan-2008  matt sync with HEAD
 1.2.4.1 28-Nov-2007  matt file atomic_swap.S was added on branch matt-armv6 on 2008-01-09 01:21:06 +0000
 1.3.2.1 23-Jun-2008  wrstuden Sync w/ -current. 34 merge conflicts to follow.
 1.4.6.1 13-May-2009  jym Sync with HEAD.

Commit is split, to avoid a "too many arguments" protocol error.
 1.5.12.1 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.5.6.1 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.2 07-Mar-2014  matt Use the sources in ../../arch/powerpc instead
 1.1 28-Nov-2007  ad branches: 1.1.4; 1.1.30; 1.1.36;
ppc64 atomic ops.
 1.1.36.1 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.1.30.1 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.1.4.2 09-Jan-2008  matt sync with HEAD
 1.1.4.1 28-Nov-2007  matt file membar_ops.S was added on branch matt-armv6 on 2008-01-09 01:21:06 +0000
 1.1 07-Jul-2006  ross branches: 1.1.94;
add reachover ffs and strlen for ppc64
 1.1.94.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.1.94.1 21-Apr-2020  martin Sync with HEAD
 1.1 07-Jul-2006  ross branches: 1.1.94;
add reachover ffs and strlen for ppc64
 1.1.94.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.1.94.1 21-Apr-2020  martin Sync with HEAD
 1.4 01-Jun-2019  maxv Misc changes in RISC-V. Start changing the memory layout, too.
 1.3 17-Apr-2019  mrg add missing atomic_and_16_nv_cas.c atomic_and_8_nv_cas.c.
 1.2 16-Oct-2014  dennis branches: 1.2.16;
Add missing C11 atomic support functions to repair prior
build breakage. matt@ made me do this.
 1.1 19-Sep-2014  matt New files for Userland support of UCB RISC-V (both 32-bit and 64-bit)
 1.2.16.2 21-Apr-2020  martin Sync with HEAD
 1.2.16.1 10-Jun-2019  christos Sync with HEAD
 1.2 27-Mar-2015  matt branches: 1.2.16;
Switch to new ABI (a0/a1 replace v0/v1)
 1.1 19-Sep-2014  matt New files for Userland support of UCB RISC-V (both 32-bit and 64-bit)
 1.2.16.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.2.16.1 21-Apr-2020  martin Sync with HEAD
 1.2 27-Mar-2015  matt branches: 1.2.16;
Switch to new ABI (a0/a1 replace v0/v1)
 1.1 19-Sep-2014  matt New files for Userland support of UCB RISC-V (both 32-bit and 64-bit)
 1.2.16.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.2.16.1 21-Apr-2020  martin Sync with HEAD
 1.2 27-Mar-2015  matt branches: 1.2.16;
Switch to new ABI (a0/a1 replace v0/v1)
 1.1 19-Sep-2014  matt New files for Userland support of UCB RISC-V (both 32-bit and 64-bit)
 1.2.16.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.2.16.1 21-Apr-2020  martin Sync with HEAD
 1.2 27-Mar-2015  matt branches: 1.2.16;
Switch to new ABI (a0/a1 replace v0/v1)
 1.1 19-Sep-2014  matt New files for Userland support of UCB RISC-V (both 32-bit and 64-bit)
 1.2.16.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.2.16.1 21-Apr-2020  martin Sync with HEAD
 1.3 17-Apr-2019  mrg add missing aliases for _atomic_cas_32_ni and atomic_cas_uint_ni.
 1.2 27-Mar-2015  matt branches: 1.2.16;
Switch to new ABI (a0/a1 replace v0/v1)
 1.1 19-Sep-2014  matt New files for Userland support of UCB RISC-V (both 32-bit and 64-bit)
 1.2.16.2 21-Apr-2020  martin Sync with HEAD
 1.2.16.1 10-Jun-2019  christos Sync with HEAD
 1.2 27-Mar-2015  matt branches: 1.2.16;
Switch to new ABI (a0/a1 replace v0/v1)
 1.1 19-Sep-2014  matt New files for Userland support of UCB RISC-V (both 32-bit and 64-bit)
 1.2.16.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.2.16.1 21-Apr-2020  martin Sync with HEAD
 1.2 27-Mar-2015  matt branches: 1.2.16;
Switch to new ABI (a0/a1 replace v0/v1)
 1.1 19-Sep-2014  matt New files for Userland support of UCB RISC-V (both 32-bit and 64-bit)
 1.2.16.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.2.16.1 21-Apr-2020  martin Sync with HEAD
 1.2 27-Mar-2015  matt branches: 1.2.16;
Switch to new ABI (a0/a1 replace v0/v1)
 1.1 19-Sep-2014  matt New files for Userland support of UCB RISC-V (both 32-bit and 64-bit)
 1.2.16.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.2.16.1 21-Apr-2020  martin Sync with HEAD
 1.2 18-May-2015  justin branches: 1.2.16;
Do not set CRT_ALIAS for rump kernel; symbols provided by toolchain
 1.1 19-Sep-2014  matt New files for Userland support of UCB RISC-V (both 32-bit and 64-bit)
 1.2.16.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.2.16.1 21-Apr-2020  martin Sync with HEAD
 1.2 27-Mar-2015  matt branches: 1.2.16;
Switch to new ABI (a0/a1 replace v0/v1)
 1.1 19-Sep-2014  matt New files for Userland support of UCB RISC-V (both 32-bit and 64-bit)
 1.2.16.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.2.16.1 21-Apr-2020  martin Sync with HEAD
 1.2 27-Mar-2015  matt branches: 1.2.16;
Switch to new ABI (a0/a1 replace v0/v1)
 1.1 19-Sep-2014  matt New files for Userland support of UCB RISC-V (both 32-bit and 64-bit)
 1.2.16.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.2.16.1 21-Apr-2020  martin Sync with HEAD
 1.2 27-Mar-2015  matt branches: 1.2.16;
Switch to new ABI (a0/a1 replace v0/v1)
 1.1 19-Sep-2014  matt New files for Userland support of UCB RISC-V (both 32-bit and 64-bit)
 1.2.16.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.2.16.1 21-Apr-2020  martin Sync with HEAD
 1.3 07-May-2023  skrll RISC-V support that works on QEMU with a single hart.

Thanks for Simon Burge for plic(4).
 1.2 27-Mar-2015  matt branches: 1.2.16;
Switch to new ABI (a0/a1 replace v0/v1)
 1.1 19-Sep-2014  matt New files for Userland support of UCB RISC-V (both 32-bit and 64-bit)
 1.2.16.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.2.16.1 21-Apr-2020  martin Sync with HEAD
 1.2 27-Mar-2015  matt branches: 1.2.16;
Switch to new ABI (a0/a1 replace v0/v1)
 1.1 19-Sep-2014  matt New files for Userland support of UCB RISC-V (both 32-bit and 64-bit)
 1.2.16.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.2.16.1 21-Apr-2020  martin Sync with HEAD
 1.2 27-Mar-2015  matt branches: 1.2.16;
Switch to new ABI (a0/a1 replace v0/v1)
 1.1 19-Sep-2014  matt New files for Userland support of UCB RISC-V (both 32-bit and 64-bit)
 1.2.16.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.2.16.1 21-Apr-2020  martin Sync with HEAD
 1.2 27-Mar-2015  matt branches: 1.2.16;
Switch to new ABI (a0/a1 replace v0/v1)
 1.1 19-Sep-2014  matt New files for Userland support of UCB RISC-V (both 32-bit and 64-bit)
 1.2.16.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.2.16.1 21-Apr-2020  martin Sync with HEAD
 1.2 27-Mar-2015  matt branches: 1.2.16;
Switch to new ABI (a0/a1 replace v0/v1)
 1.1 19-Sep-2014  matt New files for Userland support of UCB RISC-V (both 32-bit and 64-bit)
 1.2.16.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.2.16.1 21-Apr-2020  martin Sync with HEAD
 1.4 09-Apr-2022  riastradh Introduce membar_acquire/release. Deprecate membar_enter/exit.

The names membar_enter/exit were unclear, and the documentation of
membar_enter has disagreed with the implementations on sparc,
powerpc, and even x86(!) for the entire time it has been in NetBSD.

The terms `acquire' and `release' are ubiquitous in the literature
today, and have been adopted in the C and C++ standards to mean
load-before-load/store and load/store-before-store, respectively,
which are exactly the orderings required by acquiring and releasing a
mutex, as well as other useful applications like decrementing a
reference count and then freeing the underlying object if it went to
zero.

Originally I proposed changing one word in the documentation for
membar_enter to make it load-before-load/store instead of
store-before-load/store, i.e., to make it an acquire barrier. I
proposed this on the grounds that

(a) all implementations guarantee load-before-load/store,
(b) some implementations fail to guarantee store-before-load/store,
and
(c) all uses in-tree assume load-before-load/store.

I verified parts (a) and (b) (except, for (a), powerpc didn't even
guarantee load-before-load/store -- isync isn't necessarily enough;
need lwsync in general -- but it _almost_ did, and it certainly didn't
guarantee store-before-load/store).

Part (c) might not be correct, however: under the mistaken assumption
that atomic-r/m/w then membar-w/rw is equivalent to atomic-r/m/w then
membar-r/rw, I only audited the cases of membar_enter that _aren't_
immediately after an atomic-r/m/w. All of those cases assume
load-before-load/store. But my assumption was wrong -- there are
cases of atomic-r/m/w then membar-w/rw that would be broken by
changing to atomic-r/m/w then membar-r/rw:

https://mail-index.netbsd.org/tech-kern/2022/03/29/msg028044.html

Furthermore, the name membar_enter has been adopted in other places
like OpenBSD where it actually does follow the documentation and
guarantee store-before-load/store, even if that order is not useful.
So the name membar_enter currently lives in a bad place where it
means either of two things -- r/rw or w/rw.

With this change, we deprecate membar_enter/exit, introduce
membar_acquire/release as better names for the useful pair (r/rw and
rw/w), and make sure the implementation of membar_enter guarantees
both what was documented _and_ what was implemented, making it an
alias for membar_sync.

While here, rework all of the membar_* definitions and aliases. The
new logic follows a rule to make it easier to audit:

membar_X is defined as an alias for membar_Y iff membar_X is
guaranteed by membar_Y.

The `no stronger than' relation is (the transitive closure of):

- membar_consumer (r/r) is guaranteed by membar_acquire (r/rw)
- membar_producer (w/w) is guaranteed by membar_release (rw/w)
- membar_acquire (r/rw) is guaranteed by membar_sync (rw/rw)
- membar_release (rw/w) is guaranteed by membar_sync (rw/rw)

And, for the deprecated membars:

- membar_enter (whether r/rw, w/rw, or rw/rw) is guaranteed by
membar_sync (rw/rw)
- membar_exit (rw/w) is guaranteed by membar_release (rw/w)

(membar_exit is identical to membar_release, but the name is
deprecated.)

Finally, while here, annotate some of the instructions with their
semantics. For powerpc, leave an essay with citations on the
unfortunate but -- as far as I can tell -- necessary decision to use
lwsync, not isync, for membar_acquire and membar_consumer.

Also add membar(3) and atomic(3) man page links.
 1.3 09-Apr-2022  riastradh riscv/membar_ops: Upgrade membar_enter from W/RW to RW/RW.

This will be deprecated soon but let's avoid leaving rakes to trip on
with it arising from disagreement over the documentation (W/RW) and
implementation and usage (R/RW).
 1.2 06-Apr-2022  riastradh Nix trailing whitespace in files of membars, atomics, and lock stubs.

Will be touching many of these files soon for functional changes.

No functional change intended.
 1.1 19-Sep-2014  matt branches: 1.1.16;
New files for Userland support of UCB RISC-V (both 32-bit and 64-bit)
 1.1.16.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.1.16.1 21-Apr-2020  martin Sync with HEAD
 1.1 19-Sep-2014  matt branches: 1.1.16;
New files for Userland support of UCB RISC-V (both 32-bit and 64-bit)
 1.1.16.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.1.16.1 21-Apr-2020  martin Sync with HEAD
 1.9 30-Mar-2023  riastradh libc: Define __atomic_is_lock_free.

Limited to architectures where it is actually needed by gcc for any
calls to stdatomic.h atomic_is_lock_free for now.

We should also add it to other architectures too, along with lockful
atomic r/m/w operations for sizes that can't be handled natively, but
that's a lot more work. It is also necessary for -fno-inline-atomics
but we're missing a lot of other symbols for that too, to be fixed.
For now, this should enable the OpenSSL build to complete on these
architectures again after I reverted a local change.

XXX pullup-10
 1.8 28-Feb-2019  isaki branches: 1.8.14;
Add missing atomic_and_{8,16}_nv_cas.c for __sync_and_and_fetch_{1,2}.
XXX why is not only atomic_and_* symmetric unlike the others?
(in common/lib/libc/atomic/)
 1.7 13-Oct-2014  martin branches: 1.7.16;
Provide <atomic> C++ 2011 support functions for mips and sh3.
 1.6 22-Feb-2014  martin branches: 1.6.4;
Add missing __sync_* ops to libc
 1.5 04-Jan-2009  pooka branches: 1.5.8; 1.5.14;
allow inclusion of atomic ops in librump
 1.4 29-Sep-2008  ad Allow atomic ops to be built as part of libpthread.
 1.3 11-Feb-2008  ad Only build atomic ops for libkern/libc.
 1.2 10-Feb-2008  ad Enable the atomic ops in userspace.
 1.1 29-Nov-2007  ad branches: 1.1.4;
Atomic ops for sh3.
 1.1.4.3 23-Mar-2008  matt sync with HEAD
 1.1.4.2 09-Jan-2008  matt sync with HEAD
 1.1.4.1 29-Nov-2007  matt file Makefile.inc was added on branch matt-armv6 on 2008-01-09 01:21:07 +0000
 1.5.14.1 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.5.8.1 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.6.4.1 12-Nov-2014  snj Pull up following revision(s) (requested by martin in ticket #218):
common/lib/libc/arch/arm/atomic/Makefile.inc: revision 1.24-1.26
common/lib/libc/arch/hppa/atomic/Makefile.inc: revision 1.13
common/lib/libc/arch/mips/atomic/Makefile.inc: revision 1.13
common/lib/libc/arch/sh3/atomic/Makefile.inc: revision 1.7
common/lib/libc/arch/sparc/atomic/Makefile.inc: revision 1.18
common/lib/libc/arch/vax/atomic/Makefile.inc: revision 1.7
common/lib/libc/atomic/atomic_and_16_nv_cas.c: revision 1.2
common/lib/libc/atomic/atomic_and_8_nv_cas.c: revision 1.2
common/lib/libc/atomic/atomic_c11_compare_exchange_cas_16.c: revision 1.1-1.2
common/lib/libc/atomic/atomic_c11_compare_exchange_cas_32.c: revision 1.1-1.2
common/lib/libc/atomic/atomic_c11_compare_exchange_cas_8.c: revision 1.1-1.2
common/lib/libc/atomic/atomic_cas_by_cas32.c: revision 1.4
common/lib/libc/atomic/atomic_op_namespace.h: revision 1.7
Add __sync_val_compare_and_swap_{1,2} aliases for _atomic_cas_{8,16}
--
Provide __atomic_compare_exchange_N (as needed for the C11 2011 <atomic>
ops) via the corresponding CAS.
--
Hook __atomic_compare_exchange_N into vax libc.
--
Provide __sync_and_and_fetch_2 and __sync_and_and_fetch_1 for pre-ARMv6,
they are needed for the C++ 2011 <atomic> stuff.
--
Add C++ 2011 <atomic> support functions.
--
Move the and_{16,8}_nv sources into the right (libc only) block.
--
Provide <atomic> C++ 2011 support functions for mips and sh3.
--
Provide C++ 2011 <atomic> support functions for hppa and arm.
--
Provide prototypes to fix build with clang.
 1.7.16.3 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.7.16.2 21-Apr-2020  martin Sync with HEAD
 1.7.16.1 10-Jun-2019  christos Sync with HEAD
 1.8.14.1 31-Jul-2023  martin Pull up following revision(s) (requested by riastradh in ticket #275):

common/lib/libc/arch/sparc/atomic/Makefile.inc: revision 1.24
common/lib/libc/arch/m68k/atomic/Makefile.inc: revision 1.16
common/lib/libc/arch/mips/atomic/Makefile.inc: revision 1.16
common/lib/libc/arch/hppa/atomic/Makefile.inc: revision 1.15
common/lib/libc/arch/vax/atomic/Makefile.inc: revision 1.9
common/lib/libc/atomic/atomic_is_lock_free.c: revision 1.1
common/lib/libc/arch/sh3/atomic/Makefile.inc: revision 1.9

libc: Define __atomic_is_lock_free.

Limited to architectures where it is actually needed by gcc for any
calls to stdatomic.h atomic_is_lock_free for now.

We should also add it to other architectures too, along with lockful
atomic r/m/w operations for sizes that can't be handled natively, but
that's a lot more work. It is also necessary for -fno-inline-atomics
but we're missing a lot of other symbols for that too, to be fixed.

For now, this should enable the OpenSSL build to complete on these
architectures again after I reverted a local change.
 1.3 23-Apr-2006  uwe Now that bcopy.S doesn't contain the actual implemenation, move it
back to libc.
 1.2 22-Apr-2006  uwe Move the code into memcpy.S. Adjust the other files accordingly.
Same binaries are generated.
 1.1 22-Dec-2005  christos move from libc.
 1.3 04-Jul-2011  mrg branches: 1.3.44;
sh3 needs the __ffssi2 weak symbol, too.
 1.2 28-Apr-2008  martin Remove clause 3 and 4 from TNF licenses
 1.1 20-Dec-2005  christos branches: 1.1.26;
Merge libkern + libc common files. As requested by core.
 1.1.26.1 18-May-2008  yamt sync with head.
 1.3.44.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.3.44.1 21-Apr-2020  martin Sync with HEAD
 1.3 07-Feb-2024  msaitoh Remove ryo@'s mail addresses.
 1.2 22-Apr-2006  uwe branches: 1.2.94;
Move the code into memcpy.S. Adjust the other files accordingly.
Same binaries are generated.
 1.1 20-Dec-2005  christos Merge libkern + libc common files. As requested by core.
 1.2.94.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.2.94.1 21-Apr-2020  martin Sync with HEAD
 1.2 22-Apr-2006  uwe branches: 1.2.94;
Move the code into memcpy.S. Adjust the other files accordingly.
Same binaries are generated.
 1.1 20-Dec-2005  christos Merge libkern + libc common files. As requested by core.
 1.2.94.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.2.94.1 21-Apr-2020  martin Sync with HEAD
 1.2 16-Feb-2008  apb branches: 1.2.70;
For all files under src/common, as well as related man pages under
src/lib/libc, or related headers under src/sys/sys: change u_intNN_t to
uintNN_t.
 1.1 20-Dec-2005  christos branches: 1.1.6; 1.1.18;
Merge libkern + libc common files. As requested by core.
 1.1.18.1 23-Mar-2008  matt sync with HEAD
 1.1.6.1 18-Nov-2008  snj Pull up following revision(s) (requested by tsutsui in ticket #1224):
common/lib/libc/gen/bswap64.c: revision 1.2
common/lib/libprop/prop_data.c: revision 1.10
common/lib/libc/inet/inet_addr.c: revision 1.2
common/lib/libc/hash/rmd160/rmd160.c: revision 1.4
common/lib/libc/gen/bswap32.c: revision 1.2
common/lib/libc/gen/bswap16.c: revision 1.2
common/lib/libc/arch/alpha/gen/byte_swap_4.S: revision 1.3
common/lib/libc/arch/sh3/string/memset.S: revision 1.2
common/lib/libc/arch/alpha/gen/byte_swap_2.S: revision 1.3
common/lib/libc/md/md5c.c: revision 1.3
common/lib/libc/hash/sha1/sha1.c: revision 1.3
common/lib/libc/md/md4c.c: revision 1.3
sys/sys/rmd160.h: revision 1.2
sys/sys/sha2.h: revision 1.2
For all files under src/common, as well as related man pages under
src/lib/libc, or related headers under src/sys/sys: change u_intNN_t to
uintNN_t.
 1.2.70.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.2.70.1 21-Apr-2020  martin Sync with HEAD
 1.24 30-Mar-2023  riastradh libc: Define __atomic_is_lock_free.

Limited to architectures where it is actually needed by gcc for any
calls to stdatomic.h atomic_is_lock_free for now.

We should also add it to other architectures too, along with lockful
atomic r/m/w operations for sizes that can't be handled natively, but
that's a lot more work. It is also necessary for -fno-inline-atomics
but we're missing a lot of other symbols for that too, to be fixed.
For now, this should enable the OpenSSL build to complete on these
architectures again after I reverted a local change.

XXX pullup-10
 1.23 28-Feb-2019  isaki branches: 1.23.14;
Add missing atomic_and_{8,16}_nv_cas.c for __sync_and_and_fetch_{1,2}.
XXX why is not only atomic_and_* symmetric unlike the others?
(in common/lib/libc/atomic/)
 1.22 29-May-2016  nakayama branches: 1.22.16;
Use sparc64 code always on 32-bit sparc64 kernels since %psr read/write
instructions used in sparc's atomic_cas.S don't exist in SPARC-V9.
 1.21 27-Feb-2016  joerg When using clang, do not use the sparc64 code as it doesn't provide the
full set of symbols and clang in -m32 defaults to sparcv8, not
sparcv8plus.
 1.20 27-Feb-2016  joerg Include C11 load/store helpers.
 1.19 18-Oct-2015  nakayama Fix 64-bit atomic operations on 32-bit kernel and userland, and add
support for missing __sync_*_8 builtins.
 1.18 13-Oct-2014  martin Add C++ 2011 <atomic> support functions.
 1.17 21-Feb-2014  martin branches: 1.17.4;
Provide the missing __sync_* ops for sparc.
 1.16 29-Jan-2014  martin Provide _atomic_cas_{16,8}_up
 1.15 28-Nov-2012  martin Provide an assembler version of _atomic_cas_up for sparc - the C code
does not compile to something usable in a RAS. See PR 38482.
 1.14 13-Mar-2009  abs branches: 1.14.6; 1.14.12;
Revert test from MACHINE_ARCH to MACHINE, as this can be called with
with MACHINE_ARCH=sparc MACHINE=sparc64 when building 32bit kernels for
ultrasparc hardware. Picked up by nakayama@ - thanks.
Add a comment explaining this odd case
 1.13 12-Mar-2009  abs Prefer MACHINE_ARCH to MACHINE in some tests
 1.12 10-Jan-2009  pooka branches: 1.12.2;
Include userland version of atomic cas routines in librump.
(the kernel version disables interrupts and therefore can't be used)
 1.11 04-Jan-2009  pooka allow inclusion of atomic ops in librump
 1.10 29-Sep-2008  ad Allow atomic ops to be built as part of libpthread.
 1.9 07-Mar-2008  ad oops
 1.8 07-Mar-2008  ad In libc, use the MI atomic_cas(). The assembly version is faster, but I
think there are PIC issues with it and I don't have a way to fix and test.
 1.7 11-Feb-2008  ad branches: 1.7.2;
Only build atomic ops for libkern/libc.
 1.6 10-Feb-2008  ad Enable the atomic ops in userspace.
 1.5 22-Jan-2008  nakayama Search files in arch/sparc64/atomic before arch/sparc/atomic in case of
sparc64 32-bit kernel.

Make sparc64 32-bit kernel works again.
 1.4 22-Dec-2007  mrg branches: 1.4.2;
- provide 32 bit versions of the sparc v9 atomic ops.
- use them on sparc64 32 bit kernels.

fixes issues on 32 bit sparc64 kernels that were using sparc v8 ops.
tested by rjs@.
 1.3 29-Nov-2007  ad Use the CAS-based inc/dec variants, since these CPUs don't have atomic
add in hardware (does arm?).
 1.2 29-Nov-2007  ad sparc atomic ops
 1.1 27-Apr-2007  thorpej branches: 1.1.2;
file Makefile.inc was initially added on branch thorpej-atomic.
 1.1.2.1 27-Apr-2007  thorpej First cut at atomic ops for sparc.
 1.4.2.3 23-Mar-2008  matt sync with HEAD
 1.4.2.2 09-Jan-2008  matt sync with HEAD
 1.4.2.1 22-Dec-2007  matt file Makefile.inc was added on branch matt-armv6 on 2008-01-09 01:21:07 +0000
 1.7.2.1 24-Mar-2008  keiichi sync with head.
 1.12.2.1 13-May-2009  jym Sync with HEAD.

Commit is split, to avoid a "too many arguments" protocol error.
 1.14.12.2 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.14.12.1 25-Feb-2013  tls resync with head
 1.14.6.2 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.14.6.1 16-Jan-2013  yamt sync with (a bit old) head
 1.17.4.1 12-Nov-2014  snj Pull up following revision(s) (requested by martin in ticket #218):
common/lib/libc/arch/arm/atomic/Makefile.inc: revision 1.24-1.26
common/lib/libc/arch/hppa/atomic/Makefile.inc: revision 1.13
common/lib/libc/arch/mips/atomic/Makefile.inc: revision 1.13
common/lib/libc/arch/sh3/atomic/Makefile.inc: revision 1.7
common/lib/libc/arch/sparc/atomic/Makefile.inc: revision 1.18
common/lib/libc/arch/vax/atomic/Makefile.inc: revision 1.7
common/lib/libc/atomic/atomic_and_16_nv_cas.c: revision 1.2
common/lib/libc/atomic/atomic_and_8_nv_cas.c: revision 1.2
common/lib/libc/atomic/atomic_c11_compare_exchange_cas_16.c: revision 1.1-1.2
common/lib/libc/atomic/atomic_c11_compare_exchange_cas_32.c: revision 1.1-1.2
common/lib/libc/atomic/atomic_c11_compare_exchange_cas_8.c: revision 1.1-1.2
common/lib/libc/atomic/atomic_cas_by_cas32.c: revision 1.4
common/lib/libc/atomic/atomic_op_namespace.h: revision 1.7
Add __sync_val_compare_and_swap_{1,2} aliases for _atomic_cas_{8,16}
--
Provide __atomic_compare_exchange_N (as needed for the C11 2011 <atomic>
ops) via the corresponding CAS.
--
Hook __atomic_compare_exchange_N into vax libc.
--
Provide __sync_and_and_fetch_2 and __sync_and_and_fetch_1 for pre-ARMv6,
they are needed for the C++ 2011 <atomic> stuff.
--
Add C++ 2011 <atomic> support functions.
--
Move the and_{16,8}_nv sources into the right (libc only) block.
--
Provide <atomic> C++ 2011 support functions for mips and sh3.
--
Provide C++ 2011 <atomic> support functions for hppa and arm.
--
Provide prototypes to fix build with clang.
 1.22.16.2 21-Apr-2020  martin Sync with HEAD
 1.22.16.1 10-Jun-2019  christos Sync with HEAD
 1.23.14.1 31-Jul-2023  martin Pull up following revision(s) (requested by riastradh in ticket #275):

common/lib/libc/arch/sparc/atomic/Makefile.inc: revision 1.24
common/lib/libc/arch/m68k/atomic/Makefile.inc: revision 1.16
common/lib/libc/arch/mips/atomic/Makefile.inc: revision 1.16
common/lib/libc/arch/hppa/atomic/Makefile.inc: revision 1.15
common/lib/libc/arch/vax/atomic/Makefile.inc: revision 1.9
common/lib/libc/atomic/atomic_is_lock_free.c: revision 1.1
common/lib/libc/arch/sh3/atomic/Makefile.inc: revision 1.9

libc: Define __atomic_is_lock_free.

Limited to architectures where it is actually needed by gcc for any
calls to stdatomic.h atomic_is_lock_free for now.

We should also add it to other architectures too, along with lockful
atomic r/m/w operations for sizes that can't be handled natively, but
that's a lot more work. It is also necessary for -fno-inline-atomics
but we're missing a lot of other symbols for that too, to be fixed.

For now, this should enable the OpenSSL build to complete on these
architectures again after I reverted a local change.
 1.1 27-Apr-2007  thorpej branches: 1.1.2;
file atomic_add.c was initially added on branch thorpej-atomic.
 1.1.2.1 27-Apr-2007  thorpej First cut at atomic ops for sparc.
 1.1 27-Apr-2007  thorpej branches: 1.1.2;
file atomic_and.c was initially added on branch thorpej-atomic.
 1.1.2.1 27-Apr-2007  thorpej First cut at atomic ops for sparc.
 1.13 21-Feb-2014  martin branches: 1.13.26;
Provide the missing __sync_* ops for sparc.
 1.12 12-Sep-2013  joerg Pass PICFLAGS down to cc-as-as and use __PIC__ to decide if it is small
vs big PIC mode. Retire -DPIC and -DBIGPIC.
 1.11 22-Feb-2011  pooka branches: 1.11.4; 1.11.10;
Add PIC variant for fetching the lock address.

librump.so now works on sparc
 1.10 13-Mar-2009  nakayama branches: 1.10.4;
Fix typo. (s/_HARKDERNEL/_HARDKERNEL/)
 1.9 12-Jan-2009  pooka branches: 1.9.2;
include sys/param.h for _HARDKERNEL instead of homegrown def.
 1.8 10-Jan-2009  pooka Include userland version of atomic cas routines in librump.
(the kernel version disables interrupts and therefore can't be used)
 1.7 25-May-2008  chs enable profiling of assembly functions.
 1.6 28-Apr-2008  martin branches: 1.6.2;
Remove clause 3 and 4 from TNF licenses
 1.5 10-Feb-2008  ad branches: 1.5.4;
Add atomic_cas_foo_ni().
 1.4 08-Dec-2007  ad branches: 1.4.4;
Add some goo to make this work in userspace.
 1.3 08-Dec-2007  ad objdump -d to the rescue. I think this should work now.
 1.2 29-Nov-2007  ad sparc atomic ops
 1.1 27-Apr-2007  thorpej branches: 1.1.2;
file atomic_cas.S was initially added on branch thorpej-atomic.
 1.1.2.1 27-Apr-2007  thorpej First cut at atomic ops for sparc.
 1.4.4.3 23-Mar-2008  matt sync with HEAD
 1.4.4.2 09-Jan-2008  matt sync with HEAD
 1.4.4.1 08-Dec-2007  matt file atomic_cas.S was added on branch matt-armv6 on 2008-01-09 01:21:08 +0000
 1.5.4.2 04-Jun-2008  yamt sync with head
 1.5.4.1 18-May-2008  yamt sync with head.
 1.6.2.1 23-Jun-2008  wrstuden Sync w/ -current. 34 merge conflicts to follow.
 1.9.2.1 13-May-2009  jym Sync with HEAD.

Commit is split, to avoid a "too many arguments" protocol error.
 1.10.4.1 05-Mar-2011  bouyer Sync with HEAD
 1.11.10.1 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.11.4.1 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.13.26.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.13.26.1 21-Apr-2020  martin Sync with HEAD
 1.3 06-Apr-2022  riastradh Nix trailing whitespace in files of membars, atomics, and lock stubs.

Will be touching many of these files soon for functional changes.

No functional change intended.
 1.2 29-Jan-2014  martin branches: 1.2.26;
Provide _atomic_cas_{16,8}_up
 1.1 28-Nov-2012  martin branches: 1.1.2; 1.1.6;
Provide an assembler version of _atomic_cas_up for sparc - the C code
does not compile to something usable in a RAS. See PR 38482.
 1.1.6.3 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.1.6.2 25-Feb-2013  tls resync with head
 1.1.6.1 28-Nov-2012  tls file atomic_cas_up.S was added on branch tls-maxphys on 2013-02-25 00:24:00 +0000
 1.1.2.3 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.1.2.2 16-Jan-2013  yamt sync with (a bit old) head
 1.1.2.1 28-Nov-2012  yamt file atomic_cas_up.S was added on branch yamt-pagecache on 2013-01-16 05:25:53 +0000
 1.2.26.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.2.26.1 21-Apr-2020  martin Sync with HEAD
 1.1 27-Apr-2007  thorpej branches: 1.1.2;
file atomic_dec.c was initially added on branch thorpej-atomic.
 1.1.2.1 27-Apr-2007  thorpej First cut at atomic ops for sparc.
 1.1 27-Apr-2007  thorpej branches: 1.1.2;
file atomic_inc.c was initially added on branch thorpej-atomic.
 1.1.2.1 27-Apr-2007  thorpej First cut at atomic ops for sparc.
 1.7 06-Apr-2022  riastradh Nix trailing whitespace in files of membars, atomics, and lock stubs.

Will be touching many of these files soon for functional changes.

No functional change intended.
 1.6 17-Jan-2011  joerg branches: 1.6.48;
Move SPARC and SPARC64 to modern CPP. Update UPDATING note for this and
Alpha.
 1.5 28-Apr-2008  martin Remove clause 3 and 4 from TNF licenses
 1.4 22-Jan-2008  nakayama branches: 1.4.4;
Use __arch64__ instead of __sparc64__, since -D__sparc64__ is appended
to build option of sparc64 32-bit kernel.
 1.3 22-Dec-2007  mrg branches: 1.3.2;
- provide 32 bit versions of the sparc v9 atomic ops.
- use them on sparc64 32 bit kernels.

fixes issues on 32 bit sparc64 kernels that were using sparc v8 ops.
tested by rjs@.
 1.2 29-Nov-2007  ad sparc atomic ops
 1.1 27-Apr-2007  thorpej branches: 1.1.2;
file atomic_op_asm.h was initially added on branch thorpej-atomic.
 1.1.2.1 27-Apr-2007  thorpej First cut at atomic ops for sparc.
 1.3.2.3 23-Mar-2008  matt sync with HEAD
 1.3.2.2 09-Jan-2008  matt sync with HEAD
 1.3.2.1 22-Dec-2007  matt file atomic_op_asm.h was added on branch matt-armv6 on 2008-01-09 01:21:08 +0000
 1.4.4.1 18-May-2008  yamt sync with head.
 1.6.48.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.6.48.1 21-Apr-2020  martin Sync with HEAD
 1.1 27-Apr-2007  thorpej branches: 1.1.2;
file atomic_or.c was initially added on branch thorpej-atomic.
 1.1.2.1 27-Apr-2007  thorpej First cut at atomic ops for sparc.
 1.1 27-Apr-2007  thorpej branches: 1.1.2;
file atomic_swap.c was initially added on branch thorpej-atomic.
 1.1.2.1 27-Apr-2007  thorpej First cut at atomic ops for sparc.
 1.9 06-Sep-2025  riastradh paravirt_membar_sync(9): New memory barrier.

For use in paravirtualized drivers which require store-before-load
ordering -- irrespective of whether the kernel is built for a single
processor, or whether the (virtual) machine is booted with a single
processor.

This is even required on architectures that don't even have a
store-before-load ordering barrier, like m68k; adding, e.g., a virtio
bus is _as if_ the architecture has been extended with relaxed memory
ordering when talking with that new bus. Such architectures need
some way to request the hypervisor enforce that ordering -- on m68k,
that's done by issuing a CASL instruction, which qemu maps to an
atomic r/m/w with sequential consistency ordering in the host.

PR kern/59618: occasional virtio block device lock ups/hangs
 1.8 09-Apr-2022  riastradh Introduce membar_acquire/release. Deprecate membar_enter/exit.

The names membar_enter/exit were unclear, and the documentation of
membar_enter has disagreed with the implementations on sparc,
powerpc, and even x86(!) for the entire time it has been in NetBSD.

The terms `acquire' and `release' are ubiquitous in the literature
today, and have been adopted in the C and C++ standards to mean
load-before-load/store and load/store-before-store, respectively,
which are exactly the orderings required by acquiring and releasing a
mutex, as well as other useful applications like decrementing a
reference count and then freeing the underlying object if it went to
zero.

Originally I proposed changing one word in the documentation for
membar_enter to make it load-before-load/store instead of
store-before-load/store, i.e., to make it an acquire barrier. I
proposed this on the grounds that

(a) all implementations guarantee load-before-load/store,
(b) some implementations fail to guarantee store-before-load/store,
and
(c) all uses in-tree assume load-before-load/store.

I verified parts (a) and (b) (except, for (a), powerpc didn't even
guarantee load-before-load/store -- isync isn't necessarily enough;
need lwsync in general -- but it _almost_ did, and it certainly didn't
guarantee store-before-load/store).

Part (c) might not be correct, however: under the mistaken assumption
that atomic-r/m/w then membar-w/rw is equivalent to atomic-r/m/w then
membar-r/rw, I only audited the cases of membar_enter that _aren't_
immediately after an atomic-r/m/w. All of those cases assume
load-before-load/store. But my assumption was wrong -- there are
cases of atomic-r/m/w then membar-w/rw that would be broken by
changing to atomic-r/m/w then membar-r/rw:

https://mail-index.netbsd.org/tech-kern/2022/03/29/msg028044.html

Furthermore, the name membar_enter has been adopted in other places
like OpenBSD where it actually does follow the documentation and
guarantee store-before-load/store, even if that order is not useful.
So the name membar_enter currently lives in a bad place where it
means either of two things -- r/rw or w/rw.

With this change, we deprecate membar_enter/exit, introduce
membar_acquire/release as better names for the useful pair (r/rw and
rw/w), and make sure the implementation of membar_enter guarantees
both what was documented _and_ what was implemented, making it an
alias for membar_sync.

While here, rework all of the membar_* definitions and aliases. The
new logic follows a rule to make it easier to audit:

membar_X is defined as an alias for membar_Y iff membar_X is
guaranteed by membar_Y.

The `no stronger than' relation is (the transitive closure of):

- membar_consumer (r/r) is guaranteed by membar_acquire (r/rw)
- membar_producer (w/w) is guaranteed by membar_release (rw/w)
- membar_acquire (r/rw) is guaranteed by membar_sync (rw/rw)
- membar_release (rw/w) is guaranteed by membar_sync (rw/rw)

And, for the deprecated membars:

- membar_enter (whether r/rw, w/rw, or rw/rw) is guaranteed by
membar_sync (rw/rw)
- membar_exit (rw/w) is guaranteed by membar_release (rw/w)

(membar_exit is identical to membar_release, but the name is
deprecated.)

Finally, while here, annotate some of the instructions with their
semantics. For powerpc, leave an essay with citations on the
unfortunate but -- as far as I can tell -- necessary decision to use
lwsync, not isync, for membar_acquire and membar_consumer.

Also add membar(3) and atomic(3) man page links.
 1.7 09-Apr-2022  riastradh sparc/membar_ops: Upgrade membar_enter from R/RW to RW/RW.

This will be deprecated soon but let's avoid leaving rakes to trip on
with it arising from disagreement over the documentation (W/RW) and
implementation and usage (R/RW).
 1.6 09-Apr-2022  riastradh sparc: Fix membar_sync with LDSTUB.

membar_sync is required to be a full sequential consistency barrier,
equivalent to MEMBAR #StoreStore|LoadStore|StoreLoad|LoadLoad on
sparcv9. LDSTUB and SWAP are the only pre-v9 instructions that do
this and SWAP doesn't exist on all v7 hardware, so use LDSTUB.

Note: I'm having a hard time nailing down a reference for the
ordering implied by LDSTUB and SWAP. I'm _pretty sure_ SWAP has to
imply store-load ordering since the SPARCv8 manual recommends it for
Dekker's algorithm (which notoriously requires store-load ordering),
and the formal memory model treats LDSTUB and SWAP the same for
ordering. But the v8 and v9 manuals aren't clear.

GCC issues STBAR and LDSTUB, but (a) I don't see why STBAR is
necessary here, (b) STBAR doesn't exist on v7 so it'd be a pain to
use, and (c) from what I've heard (although again it's hard to nail
down authoritative references here) all actual SPARC hardware is TSO
or SC anyway so STBAR is a noop in all the silicon anyway.

Either way, certainly this is better than what we had before, which
was nothing implying ordering at all, just a store!
 1.5 06-Apr-2022  riastradh Nix trailing whitespace in files of membars, atomics, and lock stubs.

Will be touching many of these files soon for functional changes.

No functional change intended.
 1.4 25-May-2008  chs branches: 1.4.62;
enable profiling of assembly functions.
 1.3 28-Apr-2008  martin branches: 1.3.2;
Remove clause 3 and 4 from TNF licenses
 1.2 29-Nov-2007  ad branches: 1.2.4; 1.2.8;
Fix the aliases.
 1.1 29-Nov-2007  ad sparc atomic ops
 1.2.8.2 04-Jun-2008  yamt sync with head
 1.2.8.1 18-May-2008  yamt sync with head.
 1.2.4.2 09-Jan-2008  matt sync with HEAD
 1.2.4.1 29-Nov-2007  matt file membar_ops.S was added on branch matt-armv6 on 2008-01-09 01:21:08 +0000
 1.3.2.1 23-Jun-2008  wrstuden Sync w/ -current. 34 merge conflicts to follow.
 1.4.62.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.4.62.1 21-Apr-2020  martin Sync with HEAD
 1.1 20-Dec-2005  christos branches: 1.1.94;
Merge libkern + libc common files. As requested by core.
 1.1.94.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.1.94.1 21-Apr-2020  martin Sync with HEAD
 1.1 20-Dec-2005  christos branches: 1.1.94;
Merge libkern + libc common files. As requested by core.
 1.1.94.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.1.94.1 21-Apr-2020  martin Sync with HEAD
 1.1 20-Dec-2005  christos branches: 1.1.94;
Merge libkern + libc common files. As requested by core.
 1.1.94.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.1.94.1 21-Apr-2020  martin Sync with HEAD
 1.1 20-Dec-2005  christos branches: 1.1.94;
Merge libkern + libc common files. As requested by core.
 1.1.94.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.1.94.1 21-Apr-2020  martin Sync with HEAD
 1.1 20-Dec-2005  christos branches: 1.1.94;
Merge libkern + libc common files. As requested by core.
 1.1.94.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.1.94.1 21-Apr-2020  martin Sync with HEAD
 1.1 20-Dec-2005  christos branches: 1.1.94;
Merge libkern + libc common files. As requested by core.
 1.1.94.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.1.94.1 21-Apr-2020  martin Sync with HEAD
 1.1 20-Dec-2005  christos branches: 1.1.94;
Merge libkern + libc common files. As requested by core.
 1.1.94.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.1.94.1 21-Apr-2020  martin Sync with HEAD
 1.1 20-Dec-2005  christos branches: 1.1.94;
Merge libkern + libc common files. As requested by core.
 1.1.94.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.1.94.1 21-Apr-2020  martin Sync with HEAD
 1.4 12-Sep-2013  joerg branches: 1.4.26;
Pass PICFLAGS down to cc-as-as and use __PIC__ to decide if it is small
vs big PIC mode. Retire -DPIC and -DBIGPIC.
 1.3 04-Jul-2011  mrg branches: 1.3.2; 1.3.8;
add a weak alias from ffs to __ffssi2. sparc dynamic works with gcc 4.5 now.
 1.2 19-Dec-2009  pooka Don't export __ffstab, it's used only in this routine. This gets
rid of one of those evil common symbols in the __ namespace.

reviewed by martin
 1.1 20-Dec-2005  christos Merge libkern + libc common files. As requested by core.
 1.3.8.1 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.3.2.1 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.4.26.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.4.26.1 21-Apr-2020  martin Sync with HEAD
 1.1 20-Dec-2005  christos branches: 1.1.94;
Merge libkern + libc common files. As requested by core.
 1.1.94.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.1.94.1 21-Apr-2020  martin Sync with HEAD
 1.1 20-Dec-2005  christos branches: 1.1.94;
Merge libkern + libc common files. As requested by core.
 1.1.94.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.1.94.1 21-Apr-2020  martin Sync with HEAD
 1.6 04-Jan-2009  pooka branches: 1.6.50;
allow inclusion of atomic ops in librump
 1.5 29-Sep-2008  ad Allow atomic ops to be built as part of libpthread.
 1.4 11-Feb-2008  ad Only build atomic ops for libkern/libc.
 1.3 10-Feb-2008  ad Enable the atomic ops in userspace.
 1.2 28-Nov-2007  ad branches: 1.2.4;
Pull in sparc64 atomic ops from the thorpej-atomic branch.
 1.1 27-Apr-2007  thorpej branches: 1.1.2;
file Makefile.inc was initially added on branch thorpej-atomic.
 1.1.2.1 27-Apr-2007  thorpej First cut at atomic ops for sparc64.
 1.2.4.3 23-Mar-2008  matt sync with HEAD
 1.2.4.2 09-Jan-2008  matt sync with HEAD
 1.2.4.1 28-Nov-2007  matt file Makefile.inc was added on branch matt-armv6 on 2008-01-09 01:21:09 +0000
 1.6.50.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.6.50.1 21-Apr-2020  martin Sync with HEAD
 1.8 06-Apr-2022  riastradh Nix trailing whitespace in files of membars, atomics, and lock stubs.

Will be touching many of these files soon for functional changes.

No functional change intended.
 1.7 18-Oct-2015  nakayama branches: 1.7.16;
Fix 64-bit atomic operations on 32-bit kernel and userland, and add
support for missing __sync_*_8 builtins.
 1.6 25-May-2008  chs enable profiling of assembly functions.
 1.5 28-Apr-2008  martin branches: 1.5.2;
Remove clause 3 and 4 from TNF licenses
 1.4 09-Apr-2008  nakayama branches: 1.4.2;
Use %xcc for 64-bit comparisons.
 1.3 22-Dec-2007  mrg branches: 1.3.2;
- provide 32 bit versions of the sparc v9 atomic ops.
- use them on sparc64 32 bit kernels.

fixes issues on 32 bit sparc64 kernels that were using sparc v8 ops.
tested by rjs@.
 1.2 28-Nov-2007  ad Pull in sparc64 atomic ops from the thorpej-atomic branch.
 1.1 27-Apr-2007  thorpej branches: 1.1.2;
file atomic_add.S was initially added on branch thorpej-atomic.
 1.1.2.1 27-Apr-2007  thorpej First cut at atomic ops for sparc64.
 1.3.2.2 09-Jan-2008  matt sync with HEAD
 1.3.2.1 22-Dec-2007  matt file atomic_add.S was added on branch matt-armv6 on 2008-01-09 01:21:09 +0000
 1.4.2.2 04-Jun-2008  yamt sync with head
 1.4.2.1 18-May-2008  yamt sync with head.
 1.5.2.1 23-Jun-2008  wrstuden Sync w/ -current. 34 merge conflicts to follow.
 1.7.16.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.7.16.1 21-Apr-2020  martin Sync with HEAD
 1.8 06-Apr-2022  riastradh Nix trailing whitespace in files of membars, atomics, and lock stubs.

Will be touching many of these files soon for functional changes.

No functional change intended.
 1.7 18-Oct-2015  nakayama branches: 1.7.16;
Fix 64-bit atomic operations on 32-bit kernel and userland, and add
support for missing __sync_*_8 builtins.
 1.6 25-May-2008  chs enable profiling of assembly functions.
 1.5 28-Apr-2008  martin branches: 1.5.2;
Remove clause 3 and 4 from TNF licenses
 1.4 09-Apr-2008  nakayama branches: 1.4.2;
Use %xcc for 64-bit comparisons.
 1.3 22-Dec-2007  mrg branches: 1.3.2;
- provide 32 bit versions of the sparc v9 atomic ops.
- use them on sparc64 32 bit kernels.

fixes issues on 32 bit sparc64 kernels that were using sparc v8 ops.
tested by rjs@.
 1.2 28-Nov-2007  ad Pull in sparc64 atomic ops from the thorpej-atomic branch.
 1.1 27-Apr-2007  thorpej branches: 1.1.2;
file atomic_and.S was initially added on branch thorpej-atomic.
 1.1.2.1 27-Apr-2007  thorpej First cut at atomic ops for sparc64.
 1.3.2.2 09-Jan-2008  matt sync with HEAD
 1.3.2.1 22-Dec-2007  matt file atomic_and.S was added on branch matt-armv6 on 2008-01-09 01:21:10 +0000
 1.4.2.2 04-Jun-2008  yamt sync with head
 1.4.2.1 18-May-2008  yamt sync with head.
 1.5.2.1 23-Jun-2008  wrstuden Sync w/ -current. 34 merge conflicts to follow.
 1.7.16.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.7.16.1 21-Apr-2020  martin Sync with HEAD
 1.8 06-Apr-2022  riastradh Nix trailing whitespace in files of membars, atomics, and lock stubs.

Will be touching many of these files soon for functional changes.

No functional change intended.
 1.7 18-Oct-2015  nakayama branches: 1.7.16;
Fix 64-bit atomic operations on 32-bit kernel and userland, and add
support for missing __sync_*_8 builtins.
 1.6 25-May-2008  chs enable profiling of assembly functions.
 1.5 28-Apr-2008  martin branches: 1.5.2;
Remove clause 3 and 4 from TNF licenses
 1.4 10-Feb-2008  ad branches: 1.4.4;
Add atomic_cas_foo_ni().
 1.3 22-Dec-2007  mrg branches: 1.3.2;
- provide 32 bit versions of the sparc v9 atomic ops.
- use them on sparc64 32 bit kernels.

fixes issues on 32 bit sparc64 kernels that were using sparc v8 ops.
tested by rjs@.
 1.2 28-Nov-2007  ad Pull in sparc64 atomic ops from the thorpej-atomic branch.
 1.1 27-Apr-2007  thorpej branches: 1.1.2;
file atomic_cas.S was initially added on branch thorpej-atomic.
 1.1.2.1 27-Apr-2007  thorpej First cut at atomic ops for sparc64.
 1.3.2.3 23-Mar-2008  matt sync with HEAD
 1.3.2.2 09-Jan-2008  matt sync with HEAD
 1.3.2.1 22-Dec-2007  matt file atomic_cas.S was added on branch matt-armv6 on 2008-01-09 01:21:10 +0000
 1.4.4.2 04-Jun-2008  yamt sync with head
 1.4.4.1 18-May-2008  yamt sync with head.
 1.5.2.1 23-Jun-2008  wrstuden Sync w/ -current. 34 merge conflicts to follow.
 1.7.16.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.7.16.1 21-Apr-2020  martin Sync with HEAD
 1.9 06-Apr-2022  riastradh Nix trailing whitespace in files of membars, atomics, and lock stubs.

Will be touching many of these files soon for functional changes.

No functional change intended.
 1.8 18-Oct-2015  nakayama branches: 1.8.16;
Fix 64-bit atomic operations on 32-bit kernel and userland, and add
support for missing __sync_*_8 builtins.
 1.7 25-May-2008  chs enable profiling of assembly functions.
 1.6 28-Apr-2008  martin branches: 1.6.2;
Remove clause 3 and 4 from TNF licenses
 1.5 09-Apr-2008  nakayama branches: 1.5.2;
Use %xcc for 64-bit comparisons.
 1.4 22-Dec-2007  mrg branches: 1.4.2;
- provide 32 bit versions of the sparc v9 atomic ops.
- use them on sparc64 32 bit kernels.

fixes issues on 32 bit sparc64 kernels that were using sparc v8 ops.
tested by rjs@.
 1.3 30-Nov-2007  ad This is atomic_dec, not atomic_inc!
 1.2 28-Nov-2007  ad Pull in sparc64 atomic ops from the thorpej-atomic branch.
 1.1 27-Apr-2007  thorpej branches: 1.1.2;
file atomic_dec.S was initially added on branch thorpej-atomic.
 1.1.2.1 27-Apr-2007  thorpej First cut at atomic ops for sparc64.
 1.4.2.2 09-Jan-2008  matt sync with HEAD
 1.4.2.1 22-Dec-2007  matt file atomic_dec.S was added on branch matt-armv6 on 2008-01-09 01:21:11 +0000
 1.5.2.2 04-Jun-2008  yamt sync with head
 1.5.2.1 18-May-2008  yamt sync with head.
 1.6.2.1 23-Jun-2008  wrstuden Sync w/ -current. 34 merge conflicts to follow.
 1.8.16.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.8.16.1 21-Apr-2020  martin Sync with HEAD
 1.8 06-Apr-2022  riastradh Nix trailing whitespace in files of membars, atomics, and lock stubs.

Will be touching many of these files soon for functional changes.

No functional change intended.
 1.7 18-Oct-2015  nakayama branches: 1.7.16;
Fix 64-bit atomic operations on 32-bit kernel and userland, and add
support for missing __sync_*_8 builtins.
 1.6 25-May-2008  chs enable profiling of assembly functions.
 1.5 28-Apr-2008  martin branches: 1.5.2;
Remove clause 3 and 4 from TNF licenses
 1.4 09-Apr-2008  nakayama branches: 1.4.2;
Use %xcc for 64-bit comparisons.
 1.3 22-Dec-2007  mrg branches: 1.3.2;
- provide 32 bit versions of the sparc v9 atomic ops.
- use them on sparc64 32 bit kernels.

fixes issues on 32 bit sparc64 kernels that were using sparc v8 ops.
tested by rjs@.
 1.2 28-Nov-2007  ad Pull in sparc64 atomic ops from the thorpej-atomic branch.
 1.1 27-Apr-2007  thorpej branches: 1.1.2;
file atomic_inc.S was initially added on branch thorpej-atomic.
 1.1.2.1 27-Apr-2007  thorpej First cut at atomic ops for sparc64.
 1.3.2.2 09-Jan-2008  matt sync with HEAD
 1.3.2.1 22-Dec-2007  matt file atomic_inc.S was added on branch matt-armv6 on 2008-01-09 01:21:12 +0000
 1.4.2.2 04-Jun-2008  yamt sync with head
 1.4.2.1 18-May-2008  yamt sync with head.
 1.5.2.1 23-Jun-2008  wrstuden Sync w/ -current. 34 merge conflicts to follow.
 1.7.16.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.7.16.1 21-Apr-2020  martin Sync with HEAD
 1.8 06-Apr-2022  riastradh Nix trailing whitespace in files of membars, atomics, and lock stubs.

Will be touching many of these files soon for functional changes.

No functional change intended.
 1.7 18-Oct-2015  nakayama branches: 1.7.16;
Fix 64-bit atomic operations on 32-bit kernel and userland, and add
support for missing __sync_*_8 builtins.
 1.6 17-Jan-2011  joerg Move SPARC and SPARC64 to modern CPP. Update UPDATING note for this and
Alpha.
 1.5 28-Apr-2008  martin Remove clause 3 and 4 from TNF licenses
 1.4 22-Jan-2008  nakayama branches: 1.4.4;
Use __arch64__ instead of __sparc64__, since -D__sparc64__ is appended
to build option of sparc64 32-bit kernel.
 1.3 22-Dec-2007  mrg branches: 1.3.2;
- provide 32 bit versions of the sparc v9 atomic ops.
- use them on sparc64 32 bit kernels.

fixes issues on 32 bit sparc64 kernels that were using sparc v8 ops.
tested by rjs@.
 1.2 28-Nov-2007  ad Pull in sparc64 atomic ops from the thorpej-atomic branch.
 1.1 27-Apr-2007  thorpej branches: 1.1.2;
file atomic_op_asm.h was initially added on branch thorpej-atomic.
 1.1.2.1 27-Apr-2007  thorpej First cut at atomic ops for sparc64.
 1.3.2.3 23-Mar-2008  matt sync with HEAD
 1.3.2.2 09-Jan-2008  matt sync with HEAD
 1.3.2.1 22-Dec-2007  matt file atomic_op_asm.h was added on branch matt-armv6 on 2008-01-09 01:21:12 +0000
 1.4.4.1 18-May-2008  yamt sync with head.
 1.7.16.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.7.16.1 21-Apr-2020  martin Sync with HEAD
 1.9 06-Apr-2022  riastradh Nix trailing whitespace in files of membars, atomics, and lock stubs.

Will be touching many of these files soon for functional changes.

No functional change intended.
 1.8 18-Oct-2015  nakayama branches: 1.8.16;
Fix 64-bit atomic operations on 32-bit kernel and userland, and add
support for missing __sync_*_8 builtins.
 1.7 25-May-2008  chs enable profiling of assembly functions.
 1.6 28-Apr-2008  martin branches: 1.6.2;
Remove clause 3 and 4 from TNF licenses
 1.5 09-Apr-2008  nakayama branches: 1.5.2;
Use %xcc for 64-bit comparisons.
 1.4 12-Feb-2008  ad Correct a dodgy alias.
 1.3 22-Dec-2007  mrg branches: 1.3.2;
- provide 32 bit versions of the sparc v9 atomic ops.
- use them on sparc64 32 bit kernels.

fixes issues on 32 bit sparc64 kernels that were using sparc v8 ops.
tested by rjs@.
 1.2 28-Nov-2007  ad Pull in sparc64 atomic ops from the thorpej-atomic branch.
 1.1 27-Apr-2007  thorpej branches: 1.1.2;
file atomic_or.S was initially added on branch thorpej-atomic.
 1.1.2.1 27-Apr-2007  thorpej First cut at atomic ops for sparc64.
 1.3.2.3 23-Mar-2008  matt sync with HEAD
 1.3.2.2 09-Jan-2008  matt sync with HEAD
 1.3.2.1 22-Dec-2007  matt file atomic_or.S was added on branch matt-armv6 on 2008-01-09 01:21:12 +0000
 1.5.2.2 04-Jun-2008  yamt sync with head
 1.5.2.1 18-May-2008  yamt sync with head.
 1.6.2.1 23-Jun-2008  wrstuden Sync w/ -current. 34 merge conflicts to follow.
 1.8.16.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.8.16.1 21-Apr-2020  martin Sync with HEAD
 1.8 06-Apr-2022  riastradh Nix trailing whitespace in files of membars, atomics, and lock stubs.

Will be touching many of these files soon for functional changes.

No functional change intended.
 1.7 18-Oct-2015  nakayama branches: 1.7.16;
Fix 64-bit atomic operations on 32-bit kernel and userland, and add
support for missing __sync_*_8 builtins.
 1.6 25-May-2008  chs enable profiling of assembly functions.
 1.5 28-Apr-2008  martin branches: 1.5.2;
Remove clause 3 and 4 from TNF licenses
 1.4 09-Apr-2008  nakayama branches: 1.4.2;
Use %xcc for 64-bit comparisons.
 1.3 22-Dec-2007  mrg branches: 1.3.2;
- provide 32 bit versions of the sparc v9 atomic ops.
- use them on sparc64 32 bit kernels.

fixes issues on 32 bit sparc64 kernels that were using sparc v8 ops.
tested by rjs@.
 1.2 28-Nov-2007  ad Pull in sparc64 atomic ops from the thorpej-atomic branch.
 1.1 27-Apr-2007  thorpej branches: 1.1.2;
file atomic_swap.S was initially added on branch thorpej-atomic.
 1.1.2.1 27-Apr-2007  thorpej First cut at atomic ops for sparc64.
 1.3.2.2 09-Jan-2008  matt sync with HEAD
 1.3.2.1 22-Dec-2007  matt file atomic_swap.S was added on branch matt-armv6 on 2008-01-09 01:21:13 +0000
 1.4.2.2 04-Jun-2008  yamt sync with head
 1.4.2.1 18-May-2008  yamt sync with head.
 1.5.2.1 23-Jun-2008  wrstuden Sync w/ -current. 34 merge conflicts to follow.
 1.7.16.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.7.16.1 21-Apr-2020  martin Sync with HEAD
 1.10 06-Sep-2025  riastradh paravirt_membar_sync(9): New memory barrier.

For use in paravirtualized drivers which require store-before-load
ordering -- irrespective of whether the kernel is built for a single
processor, or whether the (virtual) machine is booted with a single
processor.

This is even required on architectures that don't even have a
store-before-load ordering barrier, like m68k; adding, e.g., a virtio
bus is _as if_ the architecture has been extended with relaxed memory
ordering when talking with that new bus. Such architectures need
some way to request the hypervisor enforce that ordering -- on m68k,
that's done by issuing a CASL instruction, which qemu maps to an
atomic r/m/w with sequential consistency ordering in the host.

PR kern/59618: occasional virtio block device lock ups/hangs
 1.9 09-Apr-2022  riastradh Introduce membar_acquire/release. Deprecate membar_enter/exit.

The names membar_enter/exit were unclear, and the documentation of
membar_enter has disagreed with the implementations on sparc,
powerpc, and even x86(!) for the entire time it has been in NetBSD.

The terms `acquire' and `release' are ubiquitous in the literature
today, and have been adopted in the C and C++ standards to mean
load-before-load/store and load/store-before-store, respectively,
which are exactly the orderings required by acquiring and releasing a
mutex, as well as other useful applications like decrementing a
reference count and then freeing the underlying object if it went to
zero.

Originally I proposed changing one word in the documentation for
membar_enter to make it load-before-load/store instead of
store-before-load/store, i.e., to make it an acquire barrier. I
proposed this on the grounds that

(a) all implementations guarantee load-before-load/store,
(b) some implementations fail to guarantee store-before-load/store,
and
(c) all uses in-tree assume load-before-load/store.

I verified parts (a) and (b) (except, for (a), powerpc didn't even
guarantee load-before-load/store -- isync isn't necessarily enough;
need lwsync in general -- but it _almost_ did, and it certainly didn't
guarantee store-before-load/store).

Part (c) might not be correct, however: under the mistaken assumption
that atomic-r/m/w then membar-w/rw is equivalent to atomic-r/m/w then
membar-r/rw, I only audited the cases of membar_enter that _aren't_
immediately after an atomic-r/m/w. All of those cases assume
load-before-load/store. But my assumption was wrong -- there are
cases of atomic-r/m/w then membar-w/rw that would be broken by
changing to atomic-r/m/w then membar-r/rw:

https://mail-index.netbsd.org/tech-kern/2022/03/29/msg028044.html

Furthermore, the name membar_enter has been adopted in other places
like OpenBSD where it actually does follow the documentation and
guarantee store-before-load/store, even if that order is not useful.
So the name membar_enter currently lives in a bad place where it
means either of two things -- r/rw or w/rw.

With this change, we deprecate membar_enter/exit, introduce
membar_acquire/release as better names for the useful pair (r/rw and
rw/w), and make sure the implementation of membar_enter guarantees
both what was documented _and_ what was implemented, making it an
alias for membar_sync.

While here, rework all of the membar_* definitions and aliases. The
new logic follows a rule to make it easier to audit:

membar_X is defined as an alias for membar_Y iff membar_X is
guaranteed by membar_Y.

The `no stronger than' relation is (the transitive closure of):

- membar_consumer (r/r) is guaranteed by membar_acquire (r/rw)
- membar_producer (w/w) is guaranteed by membar_release (rw/w)
- membar_acquire (r/rw) is guaranteed by membar_sync (rw/rw)
- membar_release (rw/w) is guaranteed by membar_sync (rw/rw)

And, for the deprecated membars:

- membar_enter (whether r/rw, w/rw, or rw/rw) is guaranteed by
membar_sync (rw/rw)
- membar_exit (rw/w) is guaranteed by membar_release (rw/w)

(membar_exit is identical to membar_release, but the name is
deprecated.)

Finally, while here, annotate some of the instructions with their
semantics. For powerpc, leave an essay with citations on the
unfortunate but -- as far as I can tell -- necessary decision to use
lwsync, not isync, for membar_acquire and membar_consumer.

Also add membar(3) and atomic(3) man page links.
 1.8 09-Apr-2022  riastradh sparc64/membar_ops: Upgrade membar_enter from R/RW to RW/RW.

This will be deprecated soon but let's avoid leaving rakes to trip on
with it arising from disagreement over the documentation (W/RW) and
implementation and usage (R/RW).
 1.7 09-Apr-2022  riastradh sparc64: Fix membar_sync by issuing membar #StoreLoad.

In TSO this is the only memory barrier ever needed, and somehow we
got this wrong and instead issued an unnecessary membar #LoadLoad --
not needed even in PSO let alone in TSO.

XXX Apparently we may run userland programs with PSO or RMO, in which
case all of these membars need fixing:

PSO RMO
membar_consumer nop membar #LoadLoad
membar_producer membar #StoreStore membar #StoreStore
membar_enter nop membar #LoadLoad|LoadStore
membar_exit membar #StoreStore membar #LoadStore|StoreStore
membar_sync membar #StoreLoad|StoreStore
membar #...everything...

But at least this fixes the TSO case in which we run the kernel.
Also I'm not sure there's any non-TSO hardware out there in practice.
 1.6 06-Apr-2022  riastradh Nix trailing whitespace in files of membars, atomics, and lock stubs.

Will be touching many of these files soon for functional changes.

No functional change intended.
 1.5 25-May-2008  chs branches: 1.5.62;
enable profiling of assembly functions.
 1.4 28-Apr-2008  martin branches: 1.4.2;
Remove clause 3 and 4 from TNF licenses
 1.3 12-Jan-2008  ad branches: 1.3.4;
Don't put membar in the delay slot. thorpej@ says it may be problematic.
 1.2 29-Nov-2007  ad branches: 1.2.4;
Fix the aliases.
 1.1 28-Nov-2007  ad Memory barriers for sparc64. XXX Can the 'membar' go in retl delay slot?
 1.2.4.3 23-Mar-2008  matt sync with HEAD
 1.2.4.2 09-Jan-2008  matt sync with HEAD
 1.2.4.1 29-Nov-2007  matt file membar_ops.S was added on branch matt-armv6 on 2008-01-09 01:21:13 +0000
 1.3.4.2 04-Jun-2008  yamt sync with head
 1.3.4.1 18-May-2008  yamt sync with head.
 1.4.2.1 23-Jun-2008  wrstuden Sync w/ -current. 34 merge conflicts to follow.
 1.5.62.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.5.62.1 21-Apr-2020  martin Sync with HEAD
 1.1 20-Dec-2005  christos branches: 1.1.94;
Merge libkern + libc common files. As requested by core.
 1.1.94.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.1.94.1 21-Apr-2020  martin Sync with HEAD
 1.1 20-Dec-2005  christos branches: 1.1.94;
Merge libkern + libc common files. As requested by core.
 1.1.94.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.1.94.1 21-Apr-2020  martin Sync with HEAD
 1.1 20-Dec-2005  christos branches: 1.1.94;
Merge libkern + libc common files. As requested by core.
 1.1.94.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.1.94.1 21-Apr-2020  martin Sync with HEAD
 1.1 20-Dec-2005  christos branches: 1.1.94;
Merge libkern + libc common files. As requested by core.
 1.1.94.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.1.94.1 21-Apr-2020  martin Sync with HEAD
 1.1 20-Dec-2005  christos branches: 1.1.94;
Merge libkern + libc common files. As requested by core.
 1.1.94.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.1.94.1 21-Apr-2020  martin Sync with HEAD
 1.1 20-Dec-2005  christos branches: 1.1.94;
Merge libkern + libc common files. As requested by core.
 1.1.94.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.1.94.1 21-Apr-2020  martin Sync with HEAD
 1.1 20-Dec-2005  christos branches: 1.1.94;
Merge libkern + libc common files. As requested by core.
 1.1.94.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.1.94.1 21-Apr-2020  martin Sync with HEAD
 1.3 12-Sep-2013  joerg branches: 1.3.26;
Pass PICFLAGS down to cc-as-as and use __PIC__ to decide if it is small
vs big PIC mode. Retire -DPIC and -DBIGPIC.
 1.2 19-Dec-2009  pooka branches: 1.2.6; 1.2.12;
Don't export __ffstab, it's used only in this routine. This gets
rid of one of those evil common symbols in the __ namespace.

reviewed by martin
 1.1 20-Dec-2005  christos Merge libkern + libc common files. As requested by core.
 1.2.12.1 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.2.6.1 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.3.26.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.3.26.1 21-Apr-2020  martin Sync with HEAD
 1.2 17-Mar-2013  christos branches: 1.2.6; 1.2.12; 1.2.36;
Add RCSID
Fix out of date comment
 1.1 17-Mar-2013  christos Use a single copy of the source.
 1.2.36.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.2.36.1 21-Apr-2020  martin Sync with HEAD
 1.2.12.2 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.2.12.1 17-Mar-2013  yamt file memcpy.S was added on branch yamt-pagecache on 2014-05-22 11:26:30 +0000
 1.2.6.2 23-Jun-2013  tls resync from head
 1.2.6.1 17-Mar-2013  tls file memcpy.S was added on branch tls-maxphys on 2013-06-23 06:26:13 +0000
 1.3 13-Aug-2021  andvar fix typos in words "pointer" and s/fram /frame/
 1.2 17-Mar-2013  christos branches: 1.2.6; 1.2.12; 1.2.36;
add RCSID; use clr for pattern
 1.1 17-Mar-2013  christos Use a single copy of the source.
 1.2.36.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.2.36.1 21-Apr-2020  martin Sync with HEAD
 1.2.12.2 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.2.12.1 17-Mar-2013  yamt file memset.S was added on branch yamt-pagecache on 2014-05-22 11:26:30 +0000
 1.2.6.2 23-Jun-2013  tls resync from head
 1.2.6.1 17-Mar-2013  tls file memset.S was added on branch tls-maxphys on 2013-06-23 06:26:13 +0000
 1.1 20-Dec-2005  christos branches: 1.1.94;
Merge libkern + libc common files. As requested by core.
 1.1.94.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.1.94.1 21-Apr-2020  martin Sync with HEAD
 1.1 17-Mar-2013  christos branches: 1.1.6; 1.1.12; 1.1.36;
Use a single copy of the source.
 1.1.36.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.1.36.1 21-Apr-2020  martin Sync with HEAD
 1.1.12.2 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.1.12.1 17-Mar-2013  yamt file strmacros.h was added on branch yamt-pagecache on 2014-05-22 11:26:30 +0000
 1.1.6.2 23-Jun-2013  tls resync from head
 1.1.6.1 17-Mar-2013  tls file strmacros.h was added on branch tls-maxphys on 2013-06-23 06:26:13 +0000
 1.9 30-Mar-2023  riastradh libc: Define __atomic_is_lock_free.

Limited to architectures where it is actually needed by gcc for any
calls to stdatomic.h atomic_is_lock_free for now.

We should also add it to other architectures too, along with lockful
atomic r/m/w operations for sizes that can't be handled natively, but
that's a lot more work. It is also necessary for -fno-inline-atomics
but we're missing a lot of other symbols for that too, to be fixed.
For now, this should enable the OpenSSL build to complete on these
architectures again after I reverted a local change.

XXX pullup-10
 1.8 28-Feb-2019  isaki branches: 1.8.14;
Add missing atomic_and_{8,16}_nv_cas.c for __sync_and_and_fetch_{1,2}.
XXX why is not only atomic_and_* symmetric unlike the others?
(in common/lib/libc/atomic/)
 1.7 12-Oct-2014  martin branches: 1.7.16;
Hook __atomic_compare_exchange_N into vax libc.
 1.6 22-Feb-2014  martin branches: 1.6.4;
Add the missing __sync_* ops to libc
 1.5 04-Jan-2009  pooka branches: 1.5.8; 1.5.14;
allow inclusion of atomic ops in librump
 1.4 29-Sep-2008  ad Allow atomic ops to be built as part of libpthread.
 1.3 11-Feb-2008  ad branches: 1.3.4;
Only build atomic ops for libkern/libc.
 1.2 10-Feb-2008  ad Enable the atomic ops in userspace.
 1.1 03-Feb-2008  matt Add the .C versions for the atomic ops.
 1.3.4.2 23-Mar-2008  matt sync with HEAD
 1.3.4.1 11-Feb-2008  matt file Makefile.inc was added on branch matt-armv6 on 2008-03-23 00:12:43 +0000
 1.5.14.1 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.5.8.1 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.6.4.1 12-Nov-2014  snj Pull up following revision(s) (requested by martin in ticket #218):
common/lib/libc/arch/arm/atomic/Makefile.inc: revision 1.24-1.26
common/lib/libc/arch/hppa/atomic/Makefile.inc: revision 1.13
common/lib/libc/arch/mips/atomic/Makefile.inc: revision 1.13
common/lib/libc/arch/sh3/atomic/Makefile.inc: revision 1.7
common/lib/libc/arch/sparc/atomic/Makefile.inc: revision 1.18
common/lib/libc/arch/vax/atomic/Makefile.inc: revision 1.7
common/lib/libc/atomic/atomic_and_16_nv_cas.c: revision 1.2
common/lib/libc/atomic/atomic_and_8_nv_cas.c: revision 1.2
common/lib/libc/atomic/atomic_c11_compare_exchange_cas_16.c: revision 1.1-1.2
common/lib/libc/atomic/atomic_c11_compare_exchange_cas_32.c: revision 1.1-1.2
common/lib/libc/atomic/atomic_c11_compare_exchange_cas_8.c: revision 1.1-1.2
common/lib/libc/atomic/atomic_cas_by_cas32.c: revision 1.4
common/lib/libc/atomic/atomic_op_namespace.h: revision 1.7
Add __sync_val_compare_and_swap_{1,2} aliases for _atomic_cas_{8,16}
--
Provide __atomic_compare_exchange_N (as needed for the C11 2011 <atomic>
ops) via the corresponding CAS.
--
Hook __atomic_compare_exchange_N into vax libc.
--
Provide __sync_and_and_fetch_2 and __sync_and_and_fetch_1 for pre-ARMv6,
they are needed for the C++ 2011 <atomic> stuff.
--
Add C++ 2011 <atomic> support functions.
--
Move the and_{16,8}_nv sources into the right (libc only) block.
--
Provide <atomic> C++ 2011 support functions for mips and sh3.
--
Provide C++ 2011 <atomic> support functions for hppa and arm.
--
Provide prototypes to fix build with clang.
 1.7.16.3 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.7.16.2 21-Apr-2020  martin Sync with HEAD
 1.7.16.1 10-Jun-2019  christos Sync with HEAD
 1.8.14.1 31-Jul-2023  martin Pull up following revision(s) (requested by riastradh in ticket #275):

common/lib/libc/arch/sparc/atomic/Makefile.inc: revision 1.24
common/lib/libc/arch/m68k/atomic/Makefile.inc: revision 1.16
common/lib/libc/arch/mips/atomic/Makefile.inc: revision 1.16
common/lib/libc/arch/hppa/atomic/Makefile.inc: revision 1.15
common/lib/libc/arch/vax/atomic/Makefile.inc: revision 1.9
common/lib/libc/atomic/atomic_is_lock_free.c: revision 1.1
common/lib/libc/arch/sh3/atomic/Makefile.inc: revision 1.9

libc: Define __atomic_is_lock_free.

Limited to architectures where it is actually needed by gcc for any
calls to stdatomic.h atomic_is_lock_free for now.

We should also add it to other architectures too, along with lockful
atomic r/m/w operations for sizes that can't be handled natively, but
that's a lot more work. It is also necessary for -fno-inline-atomics
but we're missing a lot of other symbols for that too, to be fixed.

For now, this should enable the OpenSSL build to complete on these
architectures again after I reverted a local change.
 1.3 25-Jan-2011  matt branches: 1.3.46;
Use END(sym)
use RCSID
 1.2 28-Apr-2008  martin branches: 1.2.20;
Remove clause 3 and 4 from TNF licenses
 1.1 31-Mar-2007  matt branches: 1.1.12;
Add these so kernels built -O0 will link.
 1.1.12.1 18-May-2008  yamt sync with head.
 1.2.20.1 08-Feb-2011  bouyer Sync with HEAD
 1.3.46.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.3.46.1 21-Apr-2020  martin Sync with HEAD
 1.3 25-Jan-2011  matt branches: 1.3.46;
Use END(sym)
use RCSID
 1.2 28-Apr-2008  martin branches: 1.2.20;
Remove clause 3 and 4 from TNF licenses
 1.1 31-Mar-2007  matt branches: 1.1.12;
Add these so kernels built -O0 will link.
 1.1.12.1 18-May-2008  yamt sync with head.
 1.2.20.1 08-Feb-2011  bouyer Sync with HEAD
 1.3.46.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.3.46.1 21-Apr-2020  martin Sync with HEAD
 1.3 25-Jan-2011  matt branches: 1.3.46;
Use END(sym)
use RCSID
 1.2 24-Dec-2005  matt branches: 1.2.48;
Don't include "DEFS.h" when <machine/asm.h> is enough.
 1.1 20-Dec-2005  christos Merge libkern + libc common files. As requested by core.
 1.2.48.1 08-Feb-2011  bouyer Sync with HEAD
 1.3.46.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.3.46.1 21-Apr-2020  martin Sync with HEAD
 1.3 25-Jan-2011  matt branches: 1.3.46;
Use END(sym)
use RCSID
 1.2 24-Dec-2005  matt branches: 1.2.48;
Don't include "DEFS.h" when <machine/asm.h> is enough.
 1.1 20-Dec-2005  christos Merge libkern + libc common files. As requested by core.
 1.2.48.1 08-Feb-2011  bouyer Sync with HEAD
 1.3.46.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.3.46.1 21-Apr-2020  martin Sync with HEAD
 1.3 25-Jan-2011  matt branches: 1.3.46;
Use END(sym)
use RCSID
 1.2 24-Dec-2005  matt branches: 1.2.48;
Don't include "DEFS.h" when <machine/asm.h> is enough.
 1.1 20-Dec-2005  christos Merge libkern + libc common files. As requested by core.
 1.2.48.1 08-Feb-2011  bouyer Sync with HEAD
 1.3.46.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.3.46.1 21-Apr-2020  martin Sync with HEAD
 1.3 25-Jan-2011  matt branches: 1.3.46;
Use END(sym)
use RCSID
 1.2 24-Dec-2005  matt branches: 1.2.48;
Don't include "DEFS.h" when <machine/asm.h> is enough.
 1.1 20-Dec-2005  christos Merge libkern + libc common files. As requested by core.
 1.2.48.1 08-Feb-2011  bouyer Sync with HEAD
 1.3.46.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.3.46.1 21-Apr-2020  martin Sync with HEAD
 1.3 25-Jan-2011  matt branches: 1.3.46;
Use END(sym)
use RCSID
 1.2 24-Dec-2005  matt branches: 1.2.48;
Don't include "DEFS.h" when <machine/asm.h> is enough.
 1.1 20-Dec-2005  christos Merge libkern + libc common files. As requested by core.
 1.2.48.1 08-Feb-2011  bouyer Sync with HEAD
 1.3.46.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.3.46.1 21-Apr-2020  martin Sync with HEAD
 1.2 25-Jan-2011  matt branches: 1.2.46;
Use END(sym)
use RCSID
 1.1 12-Mar-2010  uwe branches: 1.1.4;
Add memset to libkern on vax, moving existing memset.S to src/common.

While kernel source code is supposed to use <machine/macros.h> via
<sys/param.h> and pick an inline asm version from there, gcc itself
can generate direct calls to memset() for structure initialization, so
the symbol must be provided.

Fixes INSTALL kernel, for which -Os caused gcc to use memset() call to
initialize a struct in uvm_fault_internal().
 1.1.4.1 08-Feb-2011  bouyer Sync with HEAD
 1.2.46.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.2.46.1 21-Apr-2020  martin Sync with HEAD
 1.6 04-Jan-2009  pooka branches: 1.6.50;
allow inclusion of atomic ops in librump
 1.5 29-Sep-2008  ad Allow atomic ops to be built as part of libpthread.
 1.4 11-Feb-2008  ad Only build atomic ops for libkern/libc.
 1.3 10-Feb-2008  ad Enable the atomic ops in userspace.
 1.2 28-Nov-2007  ad branches: 1.2.4;
x86 atomic ops.
 1.1 18-Apr-2007  thorpej branches: 1.1.2;
file Makefile.inc was initially added on branch thorpej-atomic.
 1.1.2.2 26-Apr-2007  thorpej *_nv() ops implemented in terms of CMPXCHG directly. This is much tighter
code than what GCC can genererate for the generic-in-C versions.
 1.1.2.1 18-Apr-2007  thorpej Build glue for amd64.
 1.2.4.3 23-Mar-2008  matt sync with HEAD
 1.2.4.2 09-Jan-2008  matt sync with HEAD
 1.2.4.1 28-Nov-2007  matt file Makefile.inc was added on branch matt-armv6 on 2008-01-09 01:21:14 +0000
 1.6.50.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.6.50.1 21-Apr-2020  martin Sync with HEAD
 1.32 06-Sep-2025  riastradh paravirt_membar_sync(9): New memory barrier.

For use in paravirtualized drivers which require store-before-load
ordering -- irrespective of whether the kernel is built for a single
processor, or whether the (virtual) machine is booted with a single
processor.

This is even required on architectures that don't even have a
store-before-load ordering barrier, like m68k; adding, e.g., a virtio
bus is _as if_ the architecture has been extended with relaxed memory
ordering when talking with that new bus. Such architectures need
some way to request the hypervisor enforce that ordering -- on m68k,
that's done by issuing a CASL instruction, which qemu maps to an
atomic r/m/w with sequential consistency ordering in the host.

PR kern/59618: occasional virtio block device lock ups/hangs
 1.31 16-Jul-2024  riastradh amd64: Fix performance regression in uniprocessor atomics/membars.

Back in 2022, I eliminated the MFENCE hotpatch in membar_sync because
it's essentially always more expensive than LOCK ADD with no benefit
for CPU/CPU store-before-load ordering. (It is relevant only for
non-temporal stores or write-combining memory.)

https://mail-index.netbsd.org/source-changes/2022/07/30/msg140047.html

But in that change, I made a mistake and _also_ eliminated the LOCK
hotpatch on uniprocessor amd64. And our assembler gas helpfully
interprets uppercase LOCK just like lowercase lock and assembles them
the same way, so I didn't notice.

This change restores the LOCK hotpatch, so that when booting on a
uniprocessor system (or a uniprocessor guest on a multicore host),
the LOCK prefix is replaced by NOP for a cheaper instruction.

Found by puzzling over how my explanation for PR kern/57199 could
possibly be correct when (on an amd64 guest) ddb x/i membar_sync kept
showing the lock prefix even in uniprocessor boots.
 1.30 16-Jul-2024  riastradh xen: Don't hotpatch away LOCK prefix in xen_mb, even on UP boots.

Both xen_mb and membar_sync are designed to provide store-before-load
ordering, but xen_mb has to provide it in synchronizing guest with
hypervisor, while membar_sync only has to provide it in synchronizing
one (guest) CPU with another (guest) CPU.

It is safe to hotpatch away the LOCK prefix in membar_sync on a
uniprocessor boot because membar_sync is only designed to coordinate
between normal memory on multiple CPUs, and is never necessary when
there's only one CPU involved.

But xen_mb is used to coordinate between the guest and the `device'
implemented by a hypervisor, which might be running on another
_physical_ CPU even if the NetBSD guest only sees one `CPU', i.e.,
one _virtual_ CPU. So even on `uniprocessor' boots, xen_mb must
still issue an instruction with store-before-load ordering on
multiprocessor systems, such as a LOCK ADD (or MFENCE, but MFENCE is
costlier for no benefit here).

No need to change xen_wmb (release ordering, load/store-before-store)
or xen_rmb (acquire ordering, load-before-load/store) because every
x86 store is a store-release and every x86 load is a load-acquire,
even on multiprocessor systems, so there's no hotpatching involved
anyway.

PR kern/57199
 1.29 30-Jul-2022  riastradh branches: 1.29.2; 1.29.8;
x86: Eliminate mfence hotpatch for membar_sync.

The more-compatible LOCK ADD $0,-N(%rsp) turns out to be cheaper
than MFENCE anyway. Let's save some space and maintenance and rip
out the hotpatching for it.
 1.28 09-Apr-2022  riastradh Introduce membar_acquire/release. Deprecate membar_enter/exit.

The names membar_enter/exit were unclear, and the documentation of
membar_enter has disagreed with the implementations on sparc,
powerpc, and even x86(!) for the entire time it has been in NetBSD.

The terms `acquire' and `release' are ubiquitous in the literature
today, and have been adopted in the C and C++ standards to mean
load-before-load/store and load/store-before-store, respectively,
which are exactly the orderings required by acquiring and releasing a
mutex, as well as other useful applications like decrementing a
reference count and then freeing the underlying object if it went to
zero.

Originally I proposed changing one word in the documentation for
membar_enter to make it load-before-load/store instead of
store-before-load/store, i.e., to make it an acquire barrier. I
proposed this on the grounds that

(a) all implementations guarantee load-before-load/store,
(b) some implementations fail to guarantee store-before-load/store,
and
(c) all uses in-tree assume load-before-load/store.

I verified parts (a) and (b) (except, for (a), powerpc didn't even
guarantee load-before-load/store -- isync isn't necessarily enough;
need lwsync in general -- but it _almost_ did, and it certainly didn't
guarantee store-before-load/store).

Part (c) might not be correct, however: under the mistaken assumption
that atomic-r/m/w then membar-w/rw is equivalent to atomic-r/m/w then
membar-r/rw, I only audited the cases of membar_enter that _aren't_
immediately after an atomic-r/m/w. All of those cases assume
load-before-load/store. But my assumption was wrong -- there are
cases of atomic-r/m/w then membar-w/rw that would be broken by
changing to atomic-r/m/w then membar-r/rw:

https://mail-index.netbsd.org/tech-kern/2022/03/29/msg028044.html

Furthermore, the name membar_enter has been adopted in other places
like OpenBSD where it actually does follow the documentation and
guarantee store-before-load/store, even if that order is not useful.
So the name membar_enter currently lives in a bad place where it
means either of two things -- r/rw or w/rw.

With this change, we deprecate membar_enter/exit, introduce
membar_acquire/release as better names for the useful pair (r/rw and
rw/w), and make sure the implementation of membar_enter guarantees
both what was documented _and_ what was implemented, making it an
alias for membar_sync.

While here, rework all of the membar_* definitions and aliases. The
new logic follows a rule to make it easier to audit:

membar_X is defined as an alias for membar_Y iff membar_X is
guaranteed by membar_Y.

The `no stronger than' relation is (the transitive closure of):

- membar_consumer (r/r) is guaranteed by membar_acquire (r/rw)
- membar_producer (w/w) is guaranteed by membar_release (rw/w)
- membar_acquire (r/rw) is guaranteed by membar_sync (rw/rw)
- membar_release (rw/w) is guaranteed by membar_sync (rw/rw)

And, for the deprecated membars:

- membar_enter (whether r/rw, w/rw, or rw/rw) is guaranteed by
membar_sync (rw/rw)
- membar_exit (rw/w) is guaranteed by membar_release (rw/w)

(membar_exit is identical to membar_release, but the name is
deprecated.)

Finally, while here, annotate some of the instructions with their
semantics. For powerpc, leave an essay with citations on the
unfortunate but -- as far as I can tell -- necessary decision to use
lwsync, not isync, for membar_acquire and membar_consumer.

Also add membar(3) and atomic(3) man page links.
 1.27 09-Apr-2022  riastradh x86_64/membar_ops: Upgrade membar_enter from R/RW to RW/RW.

This will be deprecated soon but let's avoid leaving rakes to trip on
with it arising from disagreement over the documentation (W/RW) and
implementation and usage (R/RW).
 1.26 09-Apr-2022  riastradh x86: Add a note on membar_sync and mfence.
 1.25 09-Apr-2022  riastradh x86: Omit needless store in membar_producer/exit.

On x86, every store is a store-release, so there is no need for any
barrier. But this wasn't a barrier anyway; it was just a store,
which was redundant with the store of the return address to the stack
implied by CALL even if issuing a store made a difference.
 1.24 09-Apr-2022  riastradh x86: Every load is a load-acquire, so membar_consumer is a noop.

lfence is only needed for MD logic, such as operations on I/O memory
rather than normal cacheable memory, or special instructions like
RDTSC -- never for MI synchronization between threads/CPUs. No need
for hot-patching to do lfence here.

(The x86_lfence function might reasonably be patched on i386 to do
lfence for MD logic, but it isn't now and this doesn't change that.)
 1.23 06-Apr-2022  riastradh Nix trailing whitespace in files of membars, atomics, and lock stubs.

Will be touching many of these files soon for functional changes.

No functional change intended.
 1.22 26-Apr-2020  maxv Use the hotpatch framework for LFENCE/MFENCE.
 1.21 26-Apr-2020  maxv Remove unused argument in macro.
 1.20 26-Apr-2020  maxv Remove unused.
 1.19 26-Apr-2020  maxv Drop the hardcoded array, use the hotpatch section.
 1.18 17-Feb-2019  isaki Add missing export of atomic_or_64 (since rev1.1).
 1.17 22-May-2014  uebayasi branches: 1.17.24;
Put missing END() markers to set ELF symbol size.
 1.16 12-Jan-2011  joerg branches: 1.16.12; 1.16.24;
Allow use of traditional CPP to be set on a per platform base in sys.mk.
Honour this for dependency processing in bsd.dep.mk. Switch i386 and
amd64 assembly to use ISO C90 preprocessor concat and drop the
-traditional-cpp on this platform.
 1.15 26-Nov-2009  pooka Use strong alias within the kernel namespace regardless of if we're
dealing with a hard or soft kernel (kernel linker doesn't support
weak symbols).
 1.14 12-Jan-2009  pooka include sys/param.h for _HARDKERNEL instead of homegrown def.
 1.13 04-Jan-2009  pooka Do not use lockpatches with _RUMPKERNEL (non-PIC ... and pointless).
 1.12 25-May-2008  chs enable profiling of assembly functions.
 1.11 03-May-2008  yamt branches: 1.11.2;
rename END to ENDLABEL. i'll use END for other purpose. ok by Andrew Doran.
 1.10 28-Apr-2008  martin Remove clause 3 and 4 from TNF licenses
 1.9 10-Feb-2008  ad branches: 1.9.4;
Add atomic_cas_foo_ni().
 1.8 09-Feb-2008  ad membar_enter was doing the wrong thing. For x86 we can alias:

membar_enter -> membar_consumer
membar_exit -> membar_producer
 1.7 10-Dec-2007  ad branches: 1.7.4;
Fix _atomic_cas_64. Noted by bouyer@.
 1.6 09-Dec-2007  ad Add missing strong aliases; sure I did this before?
 1.5 29-Nov-2007  ad Fix ia32 -> amd64 thinko.
 1.4 29-Nov-2007  ad atomic_add_* takes signed integers, the others take unsigned.
 1.3 28-Nov-2007  ad A lock prefix on xchg is meaningless.
 1.2 28-Nov-2007  ad Fix up a few minor problems.
 1.1 28-Nov-2007  ad x86 atomic ops.
 1.7.4.3 23-Mar-2008  matt sync with HEAD
 1.7.4.2 09-Jan-2008  matt sync with HEAD
 1.7.4.1 10-Dec-2007  matt file atomic.S was added on branch matt-armv6 on 2008-01-09 01:21:14 +0000
 1.9.4.2 04-Jun-2008  yamt sync with head
 1.9.4.1 18-May-2008  yamt sync with head.
 1.11.2.1 23-Jun-2008  wrstuden Sync w/ -current. 34 merge conflicts to follow.
 1.16.24.1 10-Aug-2014  tls Rebase.
 1.16.12.1 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.17.24.3 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.17.24.2 21-Apr-2020  martin Sync with HEAD
 1.17.24.1 10-Jun-2019  christos Sync with HEAD
 1.29.8.1 02-Aug-2025  perseant Sync with HEAD
 1.29.2.1 20-Jul-2024  martin Pull up following revision(s) (requested by riastradh in ticket #764):

common/lib/libc/arch/i386/atomic/atomic.S: revision 1.37
sys/arch/xen/include/xenring.h: revision 1.8
sys/arch/i386/i386/cpufunc.S: revision 1.52
sys/arch/amd64/amd64/cpufunc.S: revision 1.68
sys/arch/xen/include/hypervisor.h: revision 1.60
common/lib/libc/arch/x86_64/atomic/atomic.S: revision 1.30

xen: Don't hotpatch away LOCK prefix in xen_mb, even on UP boots.

Both xen_mb and membar_sync are designed to provide store-before-load
ordering, but xen_mb has to provide it in synchronizing guest with
hypervisor, while membar_sync only has to provide it in synchronizing
one (guest) CPU with another (guest) CPU.

It is safe to hotpatch away the LOCK prefix in membar_sync on a
uniprocessor boot because membar_sync is only designed to coordinate
between normal memory on multiple CPUs, and is never necessary when
there's only one CPU involved.

But xen_mb is used to coordinate between the guest and the `device'
implemented by a hypervisor, which might be running on another
_physical_ CPU even if the NetBSD guest only sees one `CPU', i.e.,
one _virtual_ CPU. So even on `uniprocessor' boots, xen_mb must
still issue an instruction with store-before-load ordering on
multiprocessor systems, such as a LOCK ADD (or MFENCE, but MFENCE is
costlier for no benefit here).

No need to change xen_wmb (release ordering, load/store-before-store)
or xen_rmb (acquire ordering, load-before-load/store) because every
x86 store is a store-release and every x86 load is a load-acquire,
even on multiprocessor systems, so there's no hotpatching involved
anyway.

PR kern/57199
 1.1 17-Apr-2007  thorpej branches: 1.1.2;
file atomic_add.S was initially added on branch thorpej-atomic.
 1.1.2.3 26-Apr-2007  thorpej *_nv() ops implemented in terms of CMPXCHG directly. This is much tighter
code than what GCC can genererate for the generic-in-C versions.
 1.1.2.2 22-Apr-2007  thorpej Make sure namespace-cleansed aliases are avaialble for all atomic ops.
 1.1.2.1 17-Apr-2007  thorpej First draft implementation of atomic ops for amd64
 1.1 17-Apr-2007  thorpej branches: 1.1.2;
file atomic_and.S was initially added on branch thorpej-atomic.
 1.1.2.3 26-Apr-2007  thorpej *_nv() ops implemented in terms of CMPXCHG directly. This is much tighter
code than what GCC can genererate for the generic-in-C versions.
 1.1.2.2 22-Apr-2007  thorpej Make sure namespace-cleansed aliases are avaialble for all atomic ops.
 1.1.2.1 17-Apr-2007  thorpej First draft implementation of atomic ops for amd64
 1.1 17-Apr-2007  thorpej branches: 1.1.2;
file atomic_cas.S was initially added on branch thorpej-atomic.
 1.1.2.3 22-Apr-2007  thorpej Make sure namespace-cleansed aliases are avaialble for all atomic ops.
 1.1.2.2 17-Apr-2007  thorpej Fix typo.
 1.1.2.1 17-Apr-2007  thorpej First draft implementation of atomic ops for amd64
 1.1 17-Apr-2007  thorpej branches: 1.1.2;
file atomic_dec.S was initially added on branch thorpej-atomic.
 1.1.2.3 26-Apr-2007  thorpej *_nv() ops implemented in terms of CMPXCHG directly. This is much tighter
code than what GCC can genererate for the generic-in-C versions.
 1.1.2.2 22-Apr-2007  thorpej Make sure namespace-cleansed aliases are avaialble for all atomic ops.
 1.1.2.1 17-Apr-2007  thorpej First draft implementation of atomic ops for amd64
 1.1 17-Apr-2007  thorpej branches: 1.1.2;
file atomic_inc.S was initially added on branch thorpej-atomic.
 1.1.2.3 26-Apr-2007  thorpej *_nv() ops implemented in terms of CMPXCHG directly. This is much tighter
code than what GCC can genererate for the generic-in-C versions.
 1.1.2.2 22-Apr-2007  thorpej Make sure namespace-cleansed aliases are avaialble for all atomic ops.
 1.1.2.1 17-Apr-2007  thorpej First draft implementation of atomic ops for amd64
 1.1 17-Apr-2007  thorpej branches: 1.1.2;
file atomic_op_asm.h was initially added on branch thorpej-atomic.
 1.1.2.2 17-Apr-2007  thorpej amd64 doesn't have opt_cputype.h
 1.1.2.1 17-Apr-2007  thorpej First draft implementation of atomic ops for amd64
 1.1 17-Apr-2007  thorpej branches: 1.1.2;
file atomic_or.S was initially added on branch thorpej-atomic.
 1.1.2.3 26-Apr-2007  thorpej *_nv() ops implemented in terms of CMPXCHG directly. This is much tighter
code than what GCC can genererate for the generic-in-C versions.
 1.1.2.2 22-Apr-2007  thorpej Make sure namespace-cleansed aliases are avaialble for all atomic ops.
 1.1.2.1 17-Apr-2007  thorpej First draft implementation of atomic ops for amd64
 1.1 17-Apr-2007  thorpej branches: 1.1.2;
file atomic_swap.S was initially added on branch thorpej-atomic.
 1.1.2.2 22-Apr-2007  thorpej Make sure namespace-cleansed aliases are avaialble for all atomic ops.
 1.1.2.1 17-Apr-2007  thorpej First draft implementation of atomic ops for amd64
 1.1 17-Apr-2007  thorpej branches: 1.1.2;
file membar_ops.S was initially added on branch thorpej-atomic.
 1.1.2.4 22-Apr-2007  thorpej Make sure namespace-cleansed aliases are avaialble for all atomic ops.
 1.1.2.3 17-Apr-2007  thorpej Tidy up a comment.
 1.1.2.2 17-Apr-2007  thorpej amd64 doesn't have opt_cputype.h
 1.1.2.1 17-Apr-2007  thorpej First draft implementation of atomic ops for amd64
 1.3 22-May-2014  uebayasi branches: 1.3.24;
Put missing END() markers to set ELF symbol size.
 1.2 04-Feb-2006  uwe branches: 1.2.56; 1.2.70;
libc wants __bswapNN, kernel wants bswapNN. That was not accounted
for during the merge of kernel and libc versions. Fix to match
e.g. i386 code.
 1.1 20-Dec-2005  christos Merge libkern + libc common files. As requested by core.
 1.2.70.1 10-Aug-2014  tls Rebase.
 1.2.56.1 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.3.24.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.3.24.1 21-Apr-2020  martin Sync with HEAD
 1.3 22-May-2014  uebayasi branches: 1.3.24;
Put missing END() markers to set ELF symbol size.
 1.2 04-Feb-2006  uwe branches: 1.2.56; 1.2.70;
libc wants __bswapNN, kernel wants bswapNN. That was not accounted
for during the merge of kernel and libc versions. Fix to match
e.g. i386 code.
 1.1 20-Dec-2005  christos Merge libkern + libc common files. As requested by core.
 1.2.70.1 10-Aug-2014  tls Rebase.
 1.2.56.1 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.3.24.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.3.24.1 21-Apr-2020  martin Sync with HEAD
 1.2 22-May-2014  uebayasi branches: 1.2.24;
Put missing END() markers to set ELF symbol size.
 1.1 14-Jan-2010  joerg branches: 1.1.12; 1.1.24;
Move AMD64's bswap64 implementation from libc to src/common and share it
with the kernel.
 1.1.24.1 10-Aug-2014  tls Rebase.
 1.1.12.1 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.2.24.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.2.24.1 21-Apr-2020  martin Sync with HEAD
 1.5 27-Jan-2020  ad x86 uses the C versions of bcmp() and memcmp() now.
 1.4 15-Jan-2020  ad Rewrite bcmp() & memcmp() to not use REP CMPS. Seems about 5-10x faster for
small strings on modern hardware.
 1.3 22-Mar-2014  jakllsch branches: 1.3.26; 1.3.30;
For all x86_64 string assembly functions that don't overlap (i.e. every
one except memset and bzero) use END() so that symbol size information
is available.
 1.2 12-Nov-2007  ad branches: 1.2.28; 1.2.34;
Don't unconditionally clear the direction flag. The ABI says it must always
be clear when making a function call, and 'cld' takes about 50 clock cyles
on the P4.
 1.1 20-Dec-2005  christos branches: 1.1.18;
Merge libkern + libc common files. As requested by core.
 1.1.18.1 09-Jan-2008  matt sync with HEAD
 1.2.34.1 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.2.28.1 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.3.30.1 29-Feb-2020  ad Sync with head.
 1.3.26.1 08-Apr-2020  martin Merge changes from current as of 20200406
 1.5 22-Mar-2014  jakllsch branches: 1.5.26;
For all x86_64 string assembly functions that don't overlap (i.e. every
one except memset and bzero) use END() so that symbol size information
is available.
 1.4 22-Nov-2009  dsl branches: 1.4.6; 1.4.12;
Align to the destination buffer.
This probably costs 1 clock (on modern cpus) in the normal case.
But gives a big benefit when the destination is misaligned.
In particular when the source has the same misalignment - although
that may not be a gain on Nehalem!
Fixes PR/35535
 1.3 21-Nov-2009  dsl Avoid doing two 'rep movs' operations.
 1.2 12-Nov-2007  ad Don't unconditionally clear the direction flag. The ABI says it must always
be clear when making a function call, and 'cld' takes about 50 clock cyles
on the P4.
 1.1 20-Dec-2005  christos branches: 1.1.18;
Merge libkern + libc common files. As requested by core.
 1.1.18.1 09-Jan-2008  matt sync with HEAD
 1.4.12.1 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.4.6.1 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.5.26.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.5.26.1 21-Apr-2020  martin Sync with HEAD
 1.4 23-Nov-2013  jakllsch Remove x86_64 bzero.S; which since 2009 has only contained instructions
that it should be removed in 2010.
 1.3 01-Aug-2009  dsl branches: 1.3.6; 1.3.12;
Remove some long dependant instruction sequences (ie allow parallel code).
Since 'rep stos' will have a long setup time, avoid doing it more than once.
For misaligned (start address or length) write an unaligned word at both
ends of the buffer then aligned 'rep stosd' the middle.
Use the same code for bzero().
bzero.S is left being compiled for a while (empty) - to avoid issues with
duplicate symbols in libc.a after update builds.
 1.2 12-Nov-2007  ad Don't unconditionally clear the direction flag. The ABI says it must always
be clear when making a function call, and 'cld' takes about 50 clock cyles
on the P4.
 1.1 20-Dec-2005  christos branches: 1.1.18;
Merge libkern + libc common files. As requested by core.
 1.1.18.1 09-Jan-2008  matt sync with HEAD
 1.3.12.1 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.3.6.1 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.5 22-Mar-2014  jakllsch branches: 1.5.26;
For all x86_64 string assembly functions that don't overlap (i.e. every
one except memset and bzero) use END() so that symbol size information
is available.
 1.4 20-Jul-2009  christos branches: 1.4.6; 1.4.12;
Put back dsl's string changes, but fix memchr.S to use cmp so that the
condition code is set (and fix the comments 0x10->0x01). From Anon Ymous
We need a test for memchr(x, -1)...
 1.3 19-Jul-2009  christos revert changes that made new kernels hang in ACPI detection
 1.2 18-Jul-2009  dsl Remove a pointless _ALIGN_TEXT.
XXX ffs() ought to be a gcc inline asm.
 1.1 20-Dec-2005  christos branches: 1.1.36;
Merge libkern + libc common files. As requested by core.
 1.1.36.1 23-Jul-2009  jym Sync with HEAD.
 1.4.12.1 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.4.6.1 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.5.26.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.5.26.1 21-Apr-2020  martin Sync with HEAD
 1.2 17-Jul-2009  dsl Delete files that are no longer needed.
 1.1 20-Dec-2005  christos branches: 1.1.36;
Merge libkern + libc common files. As requested by core.
 1.1.36.1 23-Jul-2009  jym Sync with HEAD.
 1.6 22-Mar-2014  jakllsch branches: 1.6.26;
For all x86_64 string assembly functions that don't overlap (i.e. every
one except memset and bzero) use END() so that symbol size information
is available.
 1.5 01-Aug-2009  dsl branches: 1.5.6; 1.5.12;
In the misaligned case, xor the read word with the target pattern
before making the unwanted bytes non-zero.
Means that memchr(buf, 0xff) is no longer a special case.
 1.4 20-Jul-2009  christos Put back dsl's string changes, but fix memchr.S to use cmp so that the
condition code is set (and fix the comments 0x10->0x01). From Anon Ymous
We need a test for memchr(x, -1)...
 1.3 19-Jul-2009  christos revert changes that made new kernels hang in ACPI detection
 1.2 18-Jul-2009  dsl A better memchr().
Always read aligned words, invalidating unwanted bytes in first word,
and checking that any match in the last word is before the buffer end.
No loops apart from the one through the data.
 1.1 20-Dec-2005  christos branches: 1.1.36;
Merge libkern + libc common files. As requested by core.
 1.1.36.1 23-Jul-2009  jym Sync with HEAD.
 1.5.12.1 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.5.6.1 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.6.26.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.6.26.1 21-Apr-2020  martin Sync with HEAD
 1.6 27-Jan-2020  ad x86 uses the C versions of bcmp() and memcmp() now.
 1.5 16-Jan-2020  ad Back out previous, it's broken.
 1.4 15-Jan-2020  ad Rewrite bcmp() & memcmp() to not use REP CMPS. Seems about 5-10x faster for
small strings on modern hardware.
 1.3 22-Mar-2014  jakllsch branches: 1.3.26; 1.3.30;
For all x86_64 string assembly functions that don't overlap (i.e. every
one except memset and bzero) use END() so that symbol size information
is available.
 1.2 12-Nov-2007  ad branches: 1.2.28; 1.2.34;
Don't unconditionally clear the direction flag. The ABI says it must always
be clear when making a function call, and 'cld' takes about 50 clock cyles
on the P4.
 1.1 20-Dec-2005  christos branches: 1.1.18;
Merge libkern + libc common files. As requested by core.
 1.1.18.1 09-Jan-2008  matt sync with HEAD
 1.2.34.1 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.2.28.1 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.3.30.1 29-Feb-2020  ad Sync with head.
 1.3.26.1 08-Apr-2020  martin Merge changes from current as of 20200406
 1.1 20-Dec-2005  christos branches: 1.1.94;
Merge libkern + libc common files. As requested by core.
 1.1.94.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.1.94.1 21-Apr-2020  martin Sync with HEAD
 1.1 20-Dec-2005  christos branches: 1.1.94;
Merge libkern + libc common files. As requested by core.
 1.1.94.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.1.94.1 21-Apr-2020  martin Sync with HEAD
 1.5 22-May-2014  pooka branches: 1.5.24;
fix build for _KERNEL
 1.4 22-May-2014  uebayasi Put missing END() markers to set ELF symbol size.
 1.3 01-Aug-2009  dsl branches: 1.3.12; 1.3.24;
Remove some long dependant instruction sequences (ie allow parallel code).
Since 'rep stos' will have a long setup time, avoid doing it more than once.
For misaligned (start address or length) write an unaligned word at both
ends of the buffer then aligned 'rep stosd' the middle.
Use the same code for bzero().
bzero.S is left being compiled for a while (empty) - to avoid issues with
duplicate symbols in libc.a after update builds.
 1.2 12-Nov-2007  ad Don't unconditionally clear the direction flag. The ABI says it must always
be clear when making a function call, and 'cld' takes about 50 clock cyles
on the P4.
 1.1 20-Dec-2005  christos branches: 1.1.18;
Merge libkern + libc common files. As requested by core.
 1.1.18.1 09-Jan-2008  matt sync with HEAD
 1.3.24.1 10-Aug-2014  tls Rebase.
 1.3.12.1 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.5.24.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.5.24.1 21-Apr-2020  martin Sync with HEAD
 1.2 17-Jul-2009  dsl Delete files that are no longer needed.
 1.1 20-Dec-2005  christos branches: 1.1.36;
Merge libkern + libc common files. As requested by core.
 1.1.36.1 23-Jul-2009  jym Sync with HEAD.
 1.2 22-Mar-2014  jakllsch branches: 1.2.26;
For all x86_64 string assembly functions that don't overlap (i.e. every
one except memset and bzero) use END() so that symbol size information
is available.
 1.1 20-Dec-2005  christos branches: 1.1.50; 1.1.56;
Merge libkern + libc common files. As requested by core.
 1.1.56.1 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.1.50.1 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.2.26.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.2.26.1 21-Apr-2020  martin Sync with HEAD
 1.7 22-Mar-2014  jakllsch branches: 1.7.26;
For all x86_64 string assembly functions that don't overlap (i.e. every
one except memset and bzero) use END() so that symbol size information
is available.
 1.6 20-Jul-2009  christos branches: 1.6.6; 1.6.12;
Put back dsl's string changes, but fix memchr.S to use cmp (rather
than test) so that the condition code is set correctly (and fix the
comments: 0x10->0x01 and ^->&). From Anon Ymous

XXX: There are similar comment errors in some of the other string code.

XXX: We really need a regression test that includes misaligned memory
with searches designed to catch corner cases such as searching for 0,
-1, etc, and search length limit violations. Searching for 0 on
misaligned memory would have caught this problem.
 1.5 19-Jul-2009  christos revert changes that made new kernels hang in ACPI detection
 1.4 18-Jul-2009  dsl Shorten a dependency chain by using 'sbb, xor' (at a time when carry is set)
instead of 'mov, neg, dec'.
('mov, not' can't be used because it doesn't set the flags.)
 1.3 18-Jul-2009  dsl Replace with a version that:
1) doesn't do byte compares to find which byte matched
2) doesn't do byte compares if any top bits are set
3) doesn't use a loop when the input is misaligned
4) has less mispredicted branches
Passes regression tests and 'build.sh' doesn't explode (and more than usual).
 1.2 17-Jul-2009  dsl Change all archs so that strchr.[cS] and strrchr.[cS] exist and generate
duplicate symbols for index() and rindex().
 1.1 20-Dec-2005  christos branches: 1.1.36;
Merge libkern + libc common files. As requested by core.
 1.1.36.1 23-Jul-2009  jym Sync with HEAD.
 1.6.12.1 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.6.6.1 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.7.26.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.7.26.1 21-Apr-2020  martin Sync with HEAD
 1.2 22-Mar-2014  jakllsch branches: 1.2.26;
For all x86_64 string assembly functions that don't overlap (i.e. every
one except memset and bzero) use END() so that symbol size information
is available.
 1.1 20-Dec-2005  christos branches: 1.1.50; 1.1.56;
Merge libkern + libc common files. As requested by core.
 1.1.56.1 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.1.50.1 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.2.26.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.2.26.1 21-Apr-2020  martin Sync with HEAD
 1.2 22-Mar-2014  jakllsch branches: 1.2.26;
For all x86_64 string assembly functions that don't overlap (i.e. every
one except memset and bzero) use END() so that symbol size information
is available.
 1.1 20-Dec-2005  christos branches: 1.1.50; 1.1.56;
Merge libkern + libc common files. As requested by core.
 1.1.56.1 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.1.50.1 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.2.26.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.2.26.1 21-Apr-2020  martin Sync with HEAD
 1.8 30-Mar-2024  andvar s/Westley/Wesley/ in a book reference (in comments).
 1.7 08-Dec-2021  andvar s/efficent/efficient/ in comments.
 1.6 22-Mar-2014  jakllsch branches: 1.6.26;
For all x86_64 string assembly functions that don't overlap (i.e. every
one except memset and bzero) use END() so that symbol size information
is available.
 1.5 12-Jul-2009  dsl branches: 1.5.6; 1.5.12;
Add netbsd copyright.
Reorder a few instructions to interleave a dependency chain.
(I'm really not sure of the best order for those instructions!)
 1.4 12-Jul-2009  dsl Correct some comments
 1.3 11-Jul-2009  dsl After alg 2 triggers, mask with ~x (alg 3) to ignore bytes with top bit set.
Then use bit scan to work out which byte is zero.
If the source is misaligned read the aligned word and make the unwanted
(low order) bytes non-zero.
Passes regression test - which probably tests just enough cases.
 1.2 11-Jul-2009  dsl Change comments about algorithms, 99.6% for random data isn't 'rare' in my book!
(The 'rare' case is any byte values 0x80-0xff.)
 1.1 20-Dec-2005  christos branches: 1.1.36;
Merge libkern + libc common files. As requested by core.
 1.1.36.1 23-Jul-2009  jym Sync with HEAD.
 1.5.12.1 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.5.6.1 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.6.26.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.6.26.1 21-Apr-2020  martin Sync with HEAD
 1.3 22-Mar-2014  jakllsch branches: 1.3.26;
For all x86_64 string assembly functions that don't overlap (i.e. every
one except memset and bzero) use END() so that symbol size information
is available.
 1.2 17-Jul-2009  dsl branches: 1.2.6; 1.2.12;
Change all archs so that strchr.[cS] and strrchr.[cS] exist and generate
duplicate symbols for index() and rindex().
 1.1 20-Dec-2005  christos branches: 1.1.36;
Merge libkern + libc common files. As requested by core.
 1.1.36.1 23-Jul-2009  jym Sync with HEAD.
 1.2.12.1 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.2.6.1 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.3.26.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.3.26.1 21-Apr-2020  martin Sync with HEAD
 1.4 23-Jun-2008  ad Install manual pages for the atomic ops.
 1.3 30-Apr-2008  martin branches: 1.3.2;
Convert TNF licenses to new 2 clause variant
 1.2 28-Nov-2007  ad branches: 1.2.4; 1.2.8;
Support files from the thorpej-atomic branch.
 1.1 12-Apr-2007  thorpej branches: 1.1.2;
file atomic_add.3 was initially added on branch thorpej-atomic.
 1.1.2.1 12-Apr-2007  thorpej Man pages for the atomic ops API.
 1.2.8.1 18-May-2008  yamt sync with head.
 1.2.4.2 09-Jan-2008  matt sync with HEAD
 1.2.4.1 28-Nov-2007  matt file atomic_add.3 was added on branch matt-armv6 on 2008-01-09 01:21:16 +0000
 1.3.2.1 07-Sep-2008  wrstuden Delete a bunch of things that are now gone in head.
 1.3 23-Jun-2014  joerg branches: 1.3.4; 1.3.26;
Add aliases for the builtins used to implement C11/C++11 atomics.
 1.2 21-Feb-2014  martin branches: 1.2.2;
Provide 8 and 16 bit sync ops (using 16 bit and 8 bit cas)
 1.1 13-Apr-2007  thorpej branches: 1.1.2; 1.1.24;
file atomic_add_16_cas.c was initially added on branch thorpej-atomic.
 1.1.24.1 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.1.2.4 22-Apr-2007  thorpej Make sure namespace-cleansed aliases are avaialble for all atomic ops.
 1.1.2.3 17-Apr-2007  thorpej Work around pesky macro argument expansion rules.
 1.1.2.2 16-Apr-2007  thorpej Namespace protection for atomic operations.
 1.1.2.1 13-Apr-2007  thorpej Add generic atomic_add routines built on atomic_cas.
 1.2.2.1 10-Aug-2014  tls Rebase.
 1.3.26.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.3.26.1 21-Apr-2020  martin Sync with HEAD
 1.3.4.2 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.3.4.1 23-Jun-2014  tls file atomic_add_16_cas.c was added on branch tls-maxphys on 2014-08-19 23:45:14 +0000
 1.1 13-Apr-2007  thorpej branches: 1.1.2;
file atomic_add_16_nv_cas.c was initially added on branch thorpej-atomic.
 1.1.2.4 22-Apr-2007  thorpej Make sure namespace-cleansed aliases are avaialble for all atomic ops.
 1.1.2.3 17-Apr-2007  thorpej Work around pesky macro argument expansion rules.
 1.1.2.2 16-Apr-2007  thorpej Namespace protection for atomic operations.
 1.1.2.1 13-Apr-2007  thorpej Add generic atomic_add routines built on atomic_cas.
 1.9 23-Jun-2014  joerg branches: 1.9.24;
Add aliases for the builtins used to implement C11/C++11 atomics.
 1.8 22-Feb-2014  martin branches: 1.8.2;
Try to hide the C runtime implementation specific __sync_* ops from librump,
to avoid duplicates.
 1.7 29-Jan-2014  joerg Fix up the remaining __sync_* functions to build with Clang on ARM.
 1.6 27-Jan-2014  matt Rework so that __sync_fetch_and_<OP>_<N> actually returns a value.
 1.5 27-Jan-2014  matt Add __sync_fetch_and_add_4 alias
 1.4 28-Apr-2008  martin branches: 1.4.4; 1.4.26;
Remove clause 3 and 4 from TNF licenses
 1.3 28-Nov-2007  ad branches: 1.3.4; 1.3.8;
Memory barrier in respect of the CAS isn't necessary.
 1.2 28-Nov-2007  ad Support files from the thorpej-atomic branch.
 1.1 13-Apr-2007  thorpej branches: 1.1.2;
file atomic_add_32_cas.c was initially added on branch thorpej-atomic.
 1.1.2.4 22-Apr-2007  thorpej Make sure namespace-cleansed aliases are avaialble for all atomic ops.
 1.1.2.3 17-Apr-2007  thorpej Work around pesky macro argument expansion rules.
 1.1.2.2 16-Apr-2007  thorpej Namespace protection for atomic operations.
 1.1.2.1 13-Apr-2007  thorpej Add generic atomic_add routines built on atomic_cas.
 1.3.8.1 18-May-2008  yamt sync with head.
 1.3.4.2 09-Jan-2008  matt sync with HEAD
 1.3.4.1 28-Nov-2007  matt file atomic_add_32_cas.c was added on branch matt-armv6 on 2008-01-09 01:21:17 +0000
 1.4.26.1 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.4.4.1 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.8.2.1 10-Aug-2014  tls Rebase.
 1.9.24.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.9.24.1 21-Apr-2020  martin Sync with HEAD
 1.7 27-Feb-2016  joerg branches: 1.7.16;
Include the C11 atomic builtin alias.
 1.6 22-Feb-2014  martin Try to hide the C runtime implementation specific __sync_* ops from librump,
to avoid duplicates.
 1.5 27-Jan-2014  matt Add __sync_<OP>_and_fetch_<N> aliases
 1.4 28-Apr-2008  martin branches: 1.4.4; 1.4.26;
Remove clause 3 and 4 from TNF licenses
 1.3 28-Nov-2007  ad branches: 1.3.4; 1.3.8;
Memory barrier in respect of the CAS isn't necessary.
 1.2 28-Nov-2007  ad Support files from the thorpej-atomic branch.
 1.1 13-Apr-2007  thorpej branches: 1.1.2;
file atomic_add_32_nv_cas.c was initially added on branch thorpej-atomic.
 1.1.2.4 22-Apr-2007  thorpej Make sure namespace-cleansed aliases are avaialble for all atomic ops.
 1.1.2.3 17-Apr-2007  thorpej Work around pesky macro argument expansion rules.
 1.1.2.2 16-Apr-2007  thorpej Namespace protection for atomic operations.
 1.1.2.1 13-Apr-2007  thorpej Add generic atomic_add routines built on atomic_cas.
 1.3.8.1 18-May-2008  yamt sync with head.
 1.3.4.2 09-Jan-2008  matt sync with HEAD
 1.3.4.1 28-Nov-2007  matt file atomic_add_32_nv_cas.c was added on branch matt-armv6 on 2008-01-09 01:21:17 +0000
 1.4.26.1 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.4.4.1 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.7.16.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.7.16.1 21-Apr-2020  martin Sync with HEAD
 1.9 23-Jun-2014  joerg branches: 1.9.24;
Add aliases for the builtins used to implement C11/C++11 atomics.
 1.8 28-Jan-2014  joerg branches: 1.8.2;
Allow building with clang.
XXX __RENAME is currently not allowed for the kernel, so use plain asm.
Clang rejects definitions of builtins, so it doesn't work without.
 1.7 27-Jan-2014  matt Rework so that __sync_fetch_and_<OP>_<N> actually returns a value.
 1.6 27-Jan-2014  matt Add __sync_fetch_and_add_8 alias
 1.5 28-Apr-2008  martin branches: 1.5.4; 1.5.26;
Remove clause 3 and 4 from TNF licenses
 1.4 29-Nov-2007  ad branches: 1.4.4; 1.4.8;
Only build the 64-bit versions if __HAVE_ATOMIC64_OPS.
 1.3 28-Nov-2007  ad Memory barrier in respect of the CAS isn't necessary.
 1.2 28-Nov-2007  ad Support files from the thorpej-atomic branch.
 1.1 13-Apr-2007  thorpej branches: 1.1.2;
file atomic_add_64_cas.c was initially added on branch thorpej-atomic.
 1.1.2.4 22-Apr-2007  thorpej Make sure namespace-cleansed aliases are avaialble for all atomic ops.
 1.1.2.3 17-Apr-2007  thorpej Work around pesky macro argument expansion rules.
 1.1.2.2 16-Apr-2007  thorpej Namespace protection for atomic operations.
 1.1.2.1 13-Apr-2007  thorpej Add generic atomic_add routines built on atomic_cas.
 1.4.8.1 18-May-2008  yamt sync with head.
 1.4.4.2 09-Jan-2008  matt sync with HEAD
 1.4.4.1 29-Nov-2007  matt file atomic_add_64_cas.c was added on branch matt-armv6 on 2008-01-09 01:21:18 +0000
 1.5.26.1 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.5.4.1 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.8.2.1 10-Aug-2014  tls Rebase.
 1.9.24.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.9.24.1 21-Apr-2020  martin Sync with HEAD
 1.7 22-Feb-2014  martin branches: 1.7.26;
Try to hide the C runtime implementation specific __sync_* ops from librump,
to avoid duplicates.
 1.6 27-Jan-2014  matt Add __sync_<OP>_and_fetch_<N> aliases
 1.5 28-Apr-2008  martin branches: 1.5.4; 1.5.26;
Remove clause 3 and 4 from TNF licenses
 1.4 29-Nov-2007  ad branches: 1.4.4; 1.4.8;
Only build the 64-bit versions if __HAVE_ATOMIC64_OPS.
 1.3 28-Nov-2007  ad Memory barrier in respect of the CAS isn't necessary.
 1.2 28-Nov-2007  ad Support files from the thorpej-atomic branch.
 1.1 13-Apr-2007  thorpej branches: 1.1.2;
file atomic_add_64_nv_cas.c was initially added on branch thorpej-atomic.
 1.1.2.4 22-Apr-2007  thorpej Make sure namespace-cleansed aliases are avaialble for all atomic ops.
 1.1.2.3 17-Apr-2007  thorpej Work around pesky macro argument expansion rules.
 1.1.2.2 16-Apr-2007  thorpej Namespace protection for atomic operations.
 1.1.2.1 13-Apr-2007  thorpej Add generic atomic_add routines built on atomic_cas.
 1.4.8.1 18-May-2008  yamt sync with head.
 1.4.4.2 09-Jan-2008  matt sync with HEAD
 1.4.4.1 29-Nov-2007  matt file atomic_add_64_nv_cas.c was added on branch matt-armv6 on 2008-01-09 01:21:18 +0000
 1.5.26.1 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.5.4.1 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.7.26.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.7.26.1 21-Apr-2020  martin Sync with HEAD
 1.3 23-Jun-2014  joerg branches: 1.3.4; 1.3.26;
Add aliases for the builtins used to implement C11/C++11 atomics.
 1.2 21-Feb-2014  martin branches: 1.2.2;
Provide 8 and 16 bit sync ops (using 16 bit and 8 bit cas)
 1.1 13-Apr-2007  thorpej branches: 1.1.2; 1.1.24;
file atomic_add_8_cas.c was initially added on branch thorpej-atomic.
 1.1.24.1 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.1.2.4 22-Apr-2007  thorpej Make sure namespace-cleansed aliases are avaialble for all atomic ops.
 1.1.2.3 17-Apr-2007  thorpej Work around pesky macro argument expansion rules.
 1.1.2.2 16-Apr-2007  thorpej Namespace protection for atomic operations.
 1.1.2.1 13-Apr-2007  thorpej Add generic atomic_add routines built on atomic_cas.
 1.2.2.1 10-Aug-2014  tls Rebase.
 1.3.26.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.3.26.1 21-Apr-2020  martin Sync with HEAD
 1.3.4.2 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.3.4.1 23-Jun-2014  tls file atomic_add_8_cas.c was added on branch tls-maxphys on 2014-08-19 23:45:14 +0000
 1.1 13-Apr-2007  thorpej branches: 1.1.2;
file atomic_add_8_nv_cas.c was initially added on branch thorpej-atomic.
 1.1.2.4 22-Apr-2007  thorpej Make sure namespace-cleansed aliases are avaialble for all atomic ops.
 1.1.2.3 17-Apr-2007  thorpej Work around pesky macro argument expansion rules.
 1.1.2.2 16-Apr-2007  thorpej Namespace protection for atomic operations.
 1.1.2.1 13-Apr-2007  thorpej Add generic atomic_add routines built on atomic_cas.
 1.4 23-Jun-2008  ad Install manual pages for the atomic ops.
 1.3 30-Apr-2008  martin branches: 1.3.2;
Convert TNF licenses to new 2 clause variant
 1.2 28-Nov-2007  ad branches: 1.2.4; 1.2.8;
Support files from the thorpej-atomic branch.
 1.1 12-Apr-2007  thorpej branches: 1.1.2;
file atomic_and.3 was initially added on branch thorpej-atomic.
 1.1.2.1 12-Apr-2007  thorpej Man pages for the atomic ops API.
 1.2.8.1 18-May-2008  yamt sync with head.
 1.2.4.2 09-Jan-2008  matt sync with HEAD
 1.2.4.1 28-Nov-2007  matt file atomic_and.3 was added on branch matt-armv6 on 2008-01-09 01:21:19 +0000
 1.3.2.1 07-Sep-2008  wrstuden Delete a bunch of things that are now gone in head.
 1.3 23-Jun-2014  joerg branches: 1.3.4; 1.3.26;
Add aliases for the builtins used to implement C11/C++11 atomics.
 1.2 21-Feb-2014  martin branches: 1.2.2;
Provide 8 and 16 bit sync ops (using 16 bit and 8 bit cas)
 1.1 13-Apr-2007  thorpej branches: 1.1.2; 1.1.24;
file atomic_and_16_cas.c was initially added on branch thorpej-atomic.
 1.1.24.1 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.1.2.4 22-Apr-2007  thorpej Make sure namespace-cleansed aliases are avaialble for all atomic ops.
 1.1.2.3 17-Apr-2007  thorpej Work around pesky macro argument expansion rules.
 1.1.2.2 16-Apr-2007  thorpej Namespace protection for atomic operations.
 1.1.2.1 13-Apr-2007  thorpej Add generic atomic_and routines built on atomic_cas.
 1.2.2.1 10-Aug-2014  tls Rebase.
 1.3.26.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.3.26.1 21-Apr-2020  martin Sync with HEAD
 1.3.4.2 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.3.4.1 23-Jun-2014  tls file atomic_and_16_cas.c was added on branch tls-maxphys on 2014-08-19 23:45:14 +0000
 1.2 13-Oct-2014  martin branches: 1.2.2; 1.2.18;
Provide __sync_and_and_fetch_2 and __sync_and_and_fetch_1 for pre-ARMv6,
they are needed for the C++ 2011 <atomic> stuff.
 1.1 13-Apr-2007  thorpej branches: 1.1.2;
file atomic_and_16_nv_cas.c was initially added on branch thorpej-atomic.
 1.1.2.4 22-Apr-2007  thorpej Make sure namespace-cleansed aliases are avaialble for all atomic ops.
 1.1.2.3 17-Apr-2007  thorpej Work around pesky macro argument expansion rules.
 1.1.2.2 16-Apr-2007  thorpej Namespace protection for atomic operations.
 1.1.2.1 13-Apr-2007  thorpej Add generic atomic_and routines built on atomic_cas.
 1.2.18.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.2.18.1 21-Apr-2020  martin Sync with HEAD
 1.2.2.2 12-Nov-2014  snj Pull up following revision(s) (requested by martin in ticket #218):
common/lib/libc/arch/arm/atomic/Makefile.inc: revision 1.24-1.26
common/lib/libc/arch/hppa/atomic/Makefile.inc: revision 1.13
common/lib/libc/arch/mips/atomic/Makefile.inc: revision 1.13
common/lib/libc/arch/sh3/atomic/Makefile.inc: revision 1.7
common/lib/libc/arch/sparc/atomic/Makefile.inc: revision 1.18
common/lib/libc/arch/vax/atomic/Makefile.inc: revision 1.7
common/lib/libc/atomic/atomic_and_16_nv_cas.c: revision 1.2
common/lib/libc/atomic/atomic_and_8_nv_cas.c: revision 1.2
common/lib/libc/atomic/atomic_c11_compare_exchange_cas_16.c: revision 1.1-1.2
common/lib/libc/atomic/atomic_c11_compare_exchange_cas_32.c: revision 1.1-1.2
common/lib/libc/atomic/atomic_c11_compare_exchange_cas_8.c: revision 1.1-1.2
common/lib/libc/atomic/atomic_cas_by_cas32.c: revision 1.4
common/lib/libc/atomic/atomic_op_namespace.h: revision 1.7
Add __sync_val_compare_and_swap_{1,2} aliases for _atomic_cas_{8,16}
--
Provide __atomic_compare_exchange_N (as needed for the C11 2011 <atomic>
ops) via the corresponding CAS.
--
Hook __atomic_compare_exchange_N into vax libc.
--
Provide __sync_and_and_fetch_2 and __sync_and_and_fetch_1 for pre-ARMv6,
they are needed for the C++ 2011 <atomic> stuff.
--
Add C++ 2011 <atomic> support functions.
--
Move the and_{16,8}_nv sources into the right (libc only) block.
--
Provide <atomic> C++ 2011 support functions for mips and sh3.
--
Provide C++ 2011 <atomic> support functions for hppa and arm.
--
Provide prototypes to fix build with clang.
 1.2.2.1 13-Oct-2014  snj file atomic_and_16_nv_cas.c was added on branch netbsd-7 on 2014-11-12 19:20:06 +0000
 1.10 23-Jun-2014  joerg branches: 1.10.24;
Add aliases for the builtins used to implement C11/C++11 atomics.
 1.9 22-Feb-2014  martin branches: 1.9.2;
Try to hide the C runtime implementation specific __sync_* ops from librump,
to avoid duplicates.
 1.8 29-Jan-2014  joerg Fix up the remaining __sync_* functions to build with Clang on ARM.
 1.7 27-Jan-2014  matt fix tpyos.
 1.6 27-Jan-2014  matt Rework so that __sync_fetch_and_<OP>_<N> actually returns a value.
 1.5 27-Jan-2014  matt Add __sync_fetch_and_<OP>_<N> aliases
 1.4 28-Apr-2008  martin branches: 1.4.4; 1.4.26;
Remove clause 3 and 4 from TNF licenses
 1.3 28-Nov-2007  ad branches: 1.3.4; 1.3.8;
Memory barrier in respect of the CAS isn't necessary.
 1.2 28-Nov-2007  ad Support files from the thorpej-atomic branch.
 1.1 13-Apr-2007  thorpej branches: 1.1.2;
file atomic_and_32_cas.c was initially added on branch thorpej-atomic.
 1.1.2.4 22-Apr-2007  thorpej Make sure namespace-cleansed aliases are avaialble for all atomic ops.
 1.1.2.3 17-Apr-2007  thorpej Work around pesky macro argument expansion rules.
 1.1.2.2 16-Apr-2007  thorpej Namespace protection for atomic operations.
 1.1.2.1 13-Apr-2007  thorpej Add generic atomic_and routines built on atomic_cas.
 1.3.8.1 18-May-2008  yamt sync with head.
 1.3.4.2 09-Jan-2008  matt sync with HEAD
 1.3.4.1 28-Nov-2007  matt file atomic_and_32_cas.c was added on branch matt-armv6 on 2008-01-09 01:21:19 +0000
 1.4.26.1 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.4.4.1 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.9.2.1 10-Aug-2014  tls Rebase.
 1.10.24.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.10.24.1 21-Apr-2020  martin Sync with HEAD
 1.7 22-Feb-2014  martin branches: 1.7.26;
Try to hide the C runtime implementation specific __sync_* ops from librump,
to avoid duplicates.
 1.6 27-Jan-2014  matt Fix tpyo (add->and)
 1.5 27-Jan-2014  matt Add __sync_<OP>_and_fetch_<N> aliases
 1.4 28-Apr-2008  martin branches: 1.4.4; 1.4.26;
Remove clause 3 and 4 from TNF licenses
 1.3 28-Nov-2007  ad branches: 1.3.4; 1.3.8;
Memory barrier in respect of the CAS isn't necessary.
 1.2 28-Nov-2007  ad Support files from the thorpej-atomic branch.
 1.1 13-Apr-2007  thorpej branches: 1.1.2;
file atomic_and_32_nv_cas.c was initially added on branch thorpej-atomic.
 1.1.2.4 22-Apr-2007  thorpej Make sure namespace-cleansed aliases are avaialble for all atomic ops.
 1.1.2.3 17-Apr-2007  thorpej Work around pesky macro argument expansion rules.
 1.1.2.2 16-Apr-2007  thorpej Namespace protection for atomic operations.
 1.1.2.1 13-Apr-2007  thorpej Add generic atomic_and routines built on atomic_cas.
 1.3.8.1 18-May-2008  yamt sync with head.
 1.3.4.2 09-Jan-2008  matt sync with HEAD
 1.3.4.1 28-Nov-2007  matt file atomic_and_32_nv_cas.c was added on branch matt-armv6 on 2008-01-09 01:21:20 +0000
 1.4.26.1 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.4.4.1 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.7.26.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.7.26.1 21-Apr-2020  martin Sync with HEAD
 1.10 23-Jun-2014  joerg branches: 1.10.24;
Add aliases for the builtins used to implement C11/C++11 atomics.
 1.9 28-Jan-2014  joerg branches: 1.9.2;
Allow building with clang.
XXX __RENAME is currently not allowed for the kernel, so use plain asm.
Clang rejects definitions of builtins, so it doesn't work without.
 1.8 27-Jan-2014  matt Return old
 1.7 27-Jan-2014  matt Rework so that __sync_fetch_and_<OP>_<N> actually returns a value.
 1.6 27-Jan-2014  matt Add __sync_fetch_and_<OP>_<N> aliases
 1.5 28-Apr-2008  martin branches: 1.5.4; 1.5.26;
Remove clause 3 and 4 from TNF licenses
 1.4 29-Nov-2007  ad branches: 1.4.4; 1.4.8;
Only build the 64-bit versions if __HAVE_ATOMIC64_OPS.
 1.3 28-Nov-2007  ad Memory barrier in respect of the CAS isn't necessary.
 1.2 28-Nov-2007  ad Support files from the thorpej-atomic branch.
 1.1 13-Apr-2007  thorpej branches: 1.1.2;
file atomic_and_64_cas.c was initially added on branch thorpej-atomic.
 1.1.2.4 22-Apr-2007  thorpej Make sure namespace-cleansed aliases are avaialble for all atomic ops.
 1.1.2.3 17-Apr-2007  thorpej Work around pesky macro argument expansion rules.
 1.1.2.2 16-Apr-2007  thorpej Namespace protection for atomic operations.
 1.1.2.1 13-Apr-2007  thorpej Add generic atomic_and routines built on atomic_cas.
 1.4.8.1 18-May-2008  yamt sync with head.
 1.4.4.2 09-Jan-2008  matt sync with HEAD
 1.4.4.1 29-Nov-2007  matt file atomic_and_64_cas.c was added on branch matt-armv6 on 2008-01-09 01:21:20 +0000
 1.5.26.1 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.5.4.1 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.9.2.1 10-Aug-2014  tls Rebase.
 1.10.24.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.10.24.1 21-Apr-2020  martin Sync with HEAD
 1.8 29-May-2022  rin Fix typo; this is "and", not "add".
 1.7 22-Feb-2014  martin branches: 1.7.26;
Try to hide the C runtime implementation specific __sync_* ops from librump,
to avoid duplicates.
 1.6 27-Jan-2014  matt Add __sync_<OP>_and_fetch_<N> aliases
 1.5 28-Apr-2008  martin branches: 1.5.4; 1.5.26;
Remove clause 3 and 4 from TNF licenses
 1.4 29-Nov-2007  ad branches: 1.4.4; 1.4.8;
Only build the 64-bit versions if __HAVE_ATOMIC64_OPS.
 1.3 28-Nov-2007  ad Memory barrier in respect of the CAS isn't necessary.
 1.2 28-Nov-2007  ad Support files from the thorpej-atomic branch.
 1.1 13-Apr-2007  thorpej branches: 1.1.2;
file atomic_and_64_nv_cas.c was initially added on branch thorpej-atomic.
 1.1.2.4 22-Apr-2007  thorpej Make sure namespace-cleansed aliases are avaialble for all atomic ops.
 1.1.2.3 17-Apr-2007  thorpej Work around pesky macro argument expansion rules.
 1.1.2.2 16-Apr-2007  thorpej Namespace protection for atomic operations.
 1.1.2.1 13-Apr-2007  thorpej Add generic atomic_and routines built on atomic_cas.
 1.4.8.1 18-May-2008  yamt sync with head.
 1.4.4.2 09-Jan-2008  matt sync with HEAD
 1.4.4.1 29-Nov-2007  matt file atomic_and_64_nv_cas.c was added on branch matt-armv6 on 2008-01-09 01:21:21 +0000
 1.5.26.1 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.5.4.1 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.7.26.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.7.26.1 21-Apr-2020  martin Sync with HEAD
 1.3 23-Jun-2014  joerg branches: 1.3.4; 1.3.26;
Add aliases for the builtins used to implement C11/C++11 atomics.
 1.2 21-Feb-2014  martin branches: 1.2.2;
Provide 8 and 16 bit sync ops (using 16 bit and 8 bit cas)
 1.1 13-Apr-2007  thorpej branches: 1.1.2; 1.1.24;
file atomic_and_8_cas.c was initially added on branch thorpej-atomic.
 1.1.24.1 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.1.2.4 22-Apr-2007  thorpej Make sure namespace-cleansed aliases are avaialble for all atomic ops.
 1.1.2.3 17-Apr-2007  thorpej Work around pesky macro argument expansion rules.
 1.1.2.2 16-Apr-2007  thorpej Namespace protection for atomic operations.
 1.1.2.1 13-Apr-2007  thorpej Add generic atomic_and routines built on atomic_cas.
 1.2.2.1 10-Aug-2014  tls Rebase.
 1.3.26.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.3.26.1 21-Apr-2020  martin Sync with HEAD
 1.3.4.2 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.3.4.1 23-Jun-2014  tls file atomic_and_8_cas.c was added on branch tls-maxphys on 2014-08-19 23:45:14 +0000
 1.2 13-Oct-2014  martin branches: 1.2.2; 1.2.18;
Provide __sync_and_and_fetch_2 and __sync_and_and_fetch_1 for pre-ARMv6,
they are needed for the C++ 2011 <atomic> stuff.
 1.1 13-Apr-2007  thorpej branches: 1.1.2;
file atomic_and_8_nv_cas.c was initially added on branch thorpej-atomic.
 1.1.2.4 22-Apr-2007  thorpej Make sure namespace-cleansed aliases are avaialble for all atomic ops.
 1.1.2.3 17-Apr-2007  thorpej Work around pesky macro argument expansion rules.
 1.1.2.2 16-Apr-2007  thorpej Namespace protection for atomic operations.
 1.1.2.1 13-Apr-2007  thorpej Add generic atomic_and routines built on atomic_cas.
 1.2.18.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.2.18.1 21-Apr-2020  martin Sync with HEAD
 1.2.2.2 12-Nov-2014  snj Pull up following revision(s) (requested by martin in ticket #218):
common/lib/libc/arch/arm/atomic/Makefile.inc: revision 1.24-1.26
common/lib/libc/arch/hppa/atomic/Makefile.inc: revision 1.13
common/lib/libc/arch/mips/atomic/Makefile.inc: revision 1.13
common/lib/libc/arch/sh3/atomic/Makefile.inc: revision 1.7
common/lib/libc/arch/sparc/atomic/Makefile.inc: revision 1.18
common/lib/libc/arch/vax/atomic/Makefile.inc: revision 1.7
common/lib/libc/atomic/atomic_and_16_nv_cas.c: revision 1.2
common/lib/libc/atomic/atomic_and_8_nv_cas.c: revision 1.2
common/lib/libc/atomic/atomic_c11_compare_exchange_cas_16.c: revision 1.1-1.2
common/lib/libc/atomic/atomic_c11_compare_exchange_cas_32.c: revision 1.1-1.2
common/lib/libc/atomic/atomic_c11_compare_exchange_cas_8.c: revision 1.1-1.2
common/lib/libc/atomic/atomic_cas_by_cas32.c: revision 1.4
common/lib/libc/atomic/atomic_op_namespace.h: revision 1.7
Add __sync_val_compare_and_swap_{1,2} aliases for _atomic_cas_{8,16}
--
Provide __atomic_compare_exchange_N (as needed for the C11 2011 <atomic>
ops) via the corresponding CAS.
--
Hook __atomic_compare_exchange_N into vax libc.
--
Provide __sync_and_and_fetch_2 and __sync_and_and_fetch_1 for pre-ARMv6,
they are needed for the C++ 2011 <atomic> stuff.
--
Add C++ 2011 <atomic> support functions.
--
Move the and_{16,8}_nv sources into the right (libc only) block.
--
Provide <atomic> C++ 2011 support functions for mips and sh3.
--
Provide C++ 2011 <atomic> support functions for hppa and arm.
--
Provide prototypes to fix build with clang.
 1.2.2.1 13-Oct-2014  snj file atomic_and_8_nv_cas.c was added on branch netbsd-7 on 2014-11-12 19:20:06 +0000
 1.5 29-Jan-2024  christos sprinkle argsused
 1.4 14-May-2022  skrll *** empty log message ***
 1.3 07-Sep-2020  mrg make some prototypes match the builtin properly. GCC 9 complains
with the old version, GCC 8 is happy with this version.

tested on sparc.
 1.2 04-Nov-2014  joerg branches: 1.2.2; 1.2.18; 1.2.20;
Provide prototypes to fix build with clang.
 1.1 12-Oct-2014  martin Provide __atomic_compare_exchange_N (as needed for the C11 2011 <atomic>
ops) via the corresponding CAS.
 1.2.20.1 15-May-2022  martin Pull up following revision(s) (requested by skrll in ticket #1451):

common/lib/libc/atomic/atomic_c11_compare_exchange_cas_8.c: revision 1.4
common/lib/libc/atomic/atomic_c11_compare_exchange_cas_32.c: revision 1.4
common/lib/libc/atomic/atomic_c11_compare_exchange_cas_16.c: revision 1.4

PR 56832:
fix C implementations of __atomic_compare_exchange*
 1.2.18.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.2.18.1 21-Apr-2020  martin Sync with HEAD
 1.2.2.2 12-Nov-2014  snj Pull up following revision(s) (requested by martin in ticket #218):
common/lib/libc/arch/arm/atomic/Makefile.inc: revision 1.24-1.26
common/lib/libc/arch/hppa/atomic/Makefile.inc: revision 1.13
common/lib/libc/arch/mips/atomic/Makefile.inc: revision 1.13
common/lib/libc/arch/sh3/atomic/Makefile.inc: revision 1.7
common/lib/libc/arch/sparc/atomic/Makefile.inc: revision 1.18
common/lib/libc/arch/vax/atomic/Makefile.inc: revision 1.7
common/lib/libc/atomic/atomic_and_16_nv_cas.c: revision 1.2
common/lib/libc/atomic/atomic_and_8_nv_cas.c: revision 1.2
common/lib/libc/atomic/atomic_c11_compare_exchange_cas_16.c: revision 1.1-1.2
common/lib/libc/atomic/atomic_c11_compare_exchange_cas_32.c: revision 1.1-1.2
common/lib/libc/atomic/atomic_c11_compare_exchange_cas_8.c: revision 1.1-1.2
common/lib/libc/atomic/atomic_cas_by_cas32.c: revision 1.4
common/lib/libc/atomic/atomic_op_namespace.h: revision 1.7
Add __sync_val_compare_and_swap_{1,2} aliases for _atomic_cas_{8,16}
--
Provide __atomic_compare_exchange_N (as needed for the C11 2011 <atomic>
ops) via the corresponding CAS.
--
Hook __atomic_compare_exchange_N into vax libc.
--
Provide __sync_and_and_fetch_2 and __sync_and_and_fetch_1 for pre-ARMv6,
they are needed for the C++ 2011 <atomic> stuff.
--
Add C++ 2011 <atomic> support functions.
--
Move the and_{16,8}_nv sources into the right (libc only) block.
--
Provide <atomic> C++ 2011 support functions for mips and sh3.
--
Provide C++ 2011 <atomic> support functions for hppa and arm.
--
Provide prototypes to fix build with clang.
 1.2.2.1 04-Nov-2014  snj file atomic_c11_compare_exchange_cas_16.c was added on branch netbsd-7 on 2014-11-12 19:20:06 +0000
 1.6 29-Jan-2024  christos sprinkle argsused
 1.5 16-May-2022  skrll Fix the copy&paste botch from previous. Spotted by Tom Lane.
 1.4 14-May-2022  skrll *** empty log message ***
 1.3 07-Sep-2020  mrg make some prototypes match the builtin properly. GCC 9 complains
with the old version, GCC 8 is happy with this version.

tested on sparc.
 1.2 04-Nov-2014  joerg branches: 1.2.2; 1.2.18; 1.2.20;
Provide prototypes to fix build with clang.
 1.1 12-Oct-2014  martin Provide __atomic_compare_exchange_N (as needed for the C11 2011 <atomic>
ops) via the corresponding CAS.
 1.2.20.2 17-May-2022  martin Pull up following revision(s) (requested by skrll in ticket #1453):

common/lib/libc/atomic/atomic_c11_compare_exchange_cas_32.c: revision 1.5

Fix the copy&paste botch from previous. Spotted by Tom Lane.
 1.2.20.1 15-May-2022  martin Pull up following revision(s) (requested by skrll in ticket #1451):

common/lib/libc/atomic/atomic_c11_compare_exchange_cas_8.c: revision 1.4
common/lib/libc/atomic/atomic_c11_compare_exchange_cas_32.c: revision 1.4
common/lib/libc/atomic/atomic_c11_compare_exchange_cas_16.c: revision 1.4

PR 56832:
fix C implementations of __atomic_compare_exchange*
 1.2.18.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.2.18.1 21-Apr-2020  martin Sync with HEAD
 1.2.2.2 12-Nov-2014  snj Pull up following revision(s) (requested by martin in ticket #218):
common/lib/libc/arch/arm/atomic/Makefile.inc: revision 1.24-1.26
common/lib/libc/arch/hppa/atomic/Makefile.inc: revision 1.13
common/lib/libc/arch/mips/atomic/Makefile.inc: revision 1.13
common/lib/libc/arch/sh3/atomic/Makefile.inc: revision 1.7
common/lib/libc/arch/sparc/atomic/Makefile.inc: revision 1.18
common/lib/libc/arch/vax/atomic/Makefile.inc: revision 1.7
common/lib/libc/atomic/atomic_and_16_nv_cas.c: revision 1.2
common/lib/libc/atomic/atomic_and_8_nv_cas.c: revision 1.2
common/lib/libc/atomic/atomic_c11_compare_exchange_cas_16.c: revision 1.1-1.2
common/lib/libc/atomic/atomic_c11_compare_exchange_cas_32.c: revision 1.1-1.2
common/lib/libc/atomic/atomic_c11_compare_exchange_cas_8.c: revision 1.1-1.2
common/lib/libc/atomic/atomic_cas_by_cas32.c: revision 1.4
common/lib/libc/atomic/atomic_op_namespace.h: revision 1.7
Add __sync_val_compare_and_swap_{1,2} aliases for _atomic_cas_{8,16}
--
Provide __atomic_compare_exchange_N (as needed for the C11 2011 <atomic>
ops) via the corresponding CAS.
--
Hook __atomic_compare_exchange_N into vax libc.
--
Provide __sync_and_and_fetch_2 and __sync_and_and_fetch_1 for pre-ARMv6,
they are needed for the C++ 2011 <atomic> stuff.
--
Add C++ 2011 <atomic> support functions.
--
Move the and_{16,8}_nv sources into the right (libc only) block.
--
Provide <atomic> C++ 2011 support functions for mips and sh3.
--
Provide C++ 2011 <atomic> support functions for hppa and arm.
--
Provide prototypes to fix build with clang.
 1.2.2.1 04-Nov-2014  snj file atomic_c11_compare_exchange_cas_32.c was added on branch netbsd-7 on 2014-11-12 19:20:06 +0000
 1.5 29-Jan-2024  christos sprinkle argsused
 1.4 14-May-2022  skrll *** empty log message ***
 1.3 07-Sep-2020  mrg make some prototypes match the builtin properly. GCC 9 complains
with the old version, GCC 8 is happy with this version.

tested on sparc.
 1.2 04-Nov-2014  joerg branches: 1.2.2; 1.2.18; 1.2.20;
Provide prototypes to fix build with clang.
 1.1 12-Oct-2014  martin Provide __atomic_compare_exchange_N (as needed for the C11 2011 <atomic>
ops) via the corresponding CAS.
 1.2.20.1 15-May-2022  martin Pull up following revision(s) (requested by skrll in ticket #1451):

common/lib/libc/atomic/atomic_c11_compare_exchange_cas_8.c: revision 1.4
common/lib/libc/atomic/atomic_c11_compare_exchange_cas_32.c: revision 1.4
common/lib/libc/atomic/atomic_c11_compare_exchange_cas_16.c: revision 1.4

PR 56832:
fix C implementations of __atomic_compare_exchange*
 1.2.18.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.2.18.1 21-Apr-2020  martin Sync with HEAD
 1.2.2.2 12-Nov-2014  snj Pull up following revision(s) (requested by martin in ticket #218):
common/lib/libc/arch/arm/atomic/Makefile.inc: revision 1.24-1.26
common/lib/libc/arch/hppa/atomic/Makefile.inc: revision 1.13
common/lib/libc/arch/mips/atomic/Makefile.inc: revision 1.13
common/lib/libc/arch/sh3/atomic/Makefile.inc: revision 1.7
common/lib/libc/arch/sparc/atomic/Makefile.inc: revision 1.18
common/lib/libc/arch/vax/atomic/Makefile.inc: revision 1.7
common/lib/libc/atomic/atomic_and_16_nv_cas.c: revision 1.2
common/lib/libc/atomic/atomic_and_8_nv_cas.c: revision 1.2
common/lib/libc/atomic/atomic_c11_compare_exchange_cas_16.c: revision 1.1-1.2
common/lib/libc/atomic/atomic_c11_compare_exchange_cas_32.c: revision 1.1-1.2
common/lib/libc/atomic/atomic_c11_compare_exchange_cas_8.c: revision 1.1-1.2
common/lib/libc/atomic/atomic_cas_by_cas32.c: revision 1.4
common/lib/libc/atomic/atomic_op_namespace.h: revision 1.7
Add __sync_val_compare_and_swap_{1,2} aliases for _atomic_cas_{8,16}
--
Provide __atomic_compare_exchange_N (as needed for the C11 2011 <atomic>
ops) via the corresponding CAS.
--
Hook __atomic_compare_exchange_N into vax libc.
--
Provide __sync_and_and_fetch_2 and __sync_and_and_fetch_1 for pre-ARMv6,
they are needed for the C++ 2011 <atomic> stuff.
--
Add C++ 2011 <atomic> support functions.
--
Move the and_{16,8}_nv sources into the right (libc only) block.
--
Provide <atomic> C++ 2011 support functions for mips and sh3.
--
Provide C++ 2011 <atomic> support functions for hppa and arm.
--
Provide prototypes to fix build with clang.
 1.2.2.1 04-Nov-2014  snj file atomic_c11_compare_exchange_cas_8.c was added on branch netbsd-7 on 2014-11-12 19:20:06 +0000
 1.4 23-Jun-2008  ad Install manual pages for the atomic ops.
 1.3 30-Apr-2008  martin branches: 1.3.2;
Convert TNF licenses to new 2 clause variant
 1.2 28-Nov-2007  ad branches: 1.2.4; 1.2.8;
Support files from the thorpej-atomic branch.
 1.1 12-Apr-2007  thorpej branches: 1.1.2;
file atomic_cas.3 was initially added on branch thorpej-atomic.
 1.1.2.1 12-Apr-2007  thorpej Man pages for the atomic ops API.
 1.2.8.1 18-May-2008  yamt sync with head.
 1.2.4.2 09-Jan-2008  matt sync with HEAD
 1.2.4.1 28-Nov-2007  matt file atomic_cas.3 was added on branch matt-armv6 on 2008-01-09 01:21:21 +0000
 1.3.2.1 07-Sep-2008  wrstuden Delete a bunch of things that are now gone in head.
 1.3 21-Feb-2014  martin branches: 1.3.6; 1.3.28;
Add a few missing namespace includes
 1.2 21-Feb-2014  martin Provide 8 and 16 bit sync ops (using 16 bit and 8 bit cas)
 1.1 13-Apr-2007  thorpej branches: 1.1.2; 1.1.24;
file atomic_cas_16_cas.c was initially added on branch thorpej-atomic.
 1.1.24.1 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.1.2.4 22-Apr-2007  thorpej Make sure namespace-cleansed aliases are avaialble for all atomic ops.
 1.1.2.3 17-Apr-2007  thorpej Work around pesky macro argument expansion rules.
 1.1.2.2 16-Apr-2007  thorpej Namespace protection for atomic operations.
 1.1.2.1 13-Apr-2007  thorpej Add generic atomic_cas_8() and atomic_cas_16() routines built on
atomic_cas_32().
 1.3.28.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.3.28.1 21-Apr-2020  martin Sync with HEAD
 1.3.6.2 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.3.6.1 21-Feb-2014  tls file atomic_cas_16_cas.c was added on branch tls-maxphys on 2014-08-19 23:45:14 +0000
 1.1 21-Feb-2014  martin branches: 1.1.4; 1.1.8; 1.1.30;
Provide 32bit __sync ops based on atomic_cas_32.
 1.1.30.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.1.30.1 21-Apr-2020  martin Sync with HEAD
 1.1.8.2 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.1.8.1 21-Feb-2014  tls file atomic_cas_32_cas.c was added on branch tls-maxphys on 2014-08-19 23:45:14 +0000
 1.1.4.2 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.1.4.1 21-Feb-2014  yamt file atomic_cas_32_cas.c was added on branch yamt-pagecache on 2014-05-22 11:26:30 +0000
 1.2 18-Feb-2014  martin branches: 1.2.4; 1.2.8; 1.2.30;
Do not include <stdbool.h> in the kernel
 1.1 18-Feb-2014  martin Provide most missing __sync_*64 primitives for i386
 1.2.30.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.2.30.1 21-Apr-2020  martin Sync with HEAD
 1.2.8.2 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.2.8.1 18-Feb-2014  tls file atomic_cas_64_cas.c was added on branch tls-maxphys on 2014-08-19 23:45:14 +0000
 1.2.4.2 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.2.4.1 18-Feb-2014  yamt file atomic_cas_64_cas.c was added on branch yamt-pagecache on 2014-05-22 11:26:30 +0000
 1.3 21-Feb-2014  martin branches: 1.3.6; 1.3.28;
Add a few missing namespace includes
 1.2 21-Feb-2014  martin Provide 8 and 16 bit sync ops (using 16 bit and 8 bit cas)
 1.1 13-Apr-2007  thorpej branches: 1.1.2; 1.1.24;
file atomic_cas_8_cas.c was initially added on branch thorpej-atomic.
 1.1.24.1 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.1.2.4 22-Apr-2007  thorpej Make sure namespace-cleansed aliases are avaialble for all atomic ops.
 1.1.2.3 17-Apr-2007  thorpej Work around pesky macro argument expansion rules.
 1.1.2.2 16-Apr-2007  thorpej Namespace protection for atomic operations.
 1.1.2.1 13-Apr-2007  thorpej Add generic atomic_cas_8() and atomic_cas_16() routines built on
atomic_cas_32().
 1.3.28.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.3.28.1 21-Apr-2020  martin Sync with HEAD
 1.3.6.2 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.3.6.1 21-Feb-2014  tls file atomic_cas_8_cas.c was added on branch tls-maxphys on 2014-08-19 23:45:14 +0000
 1.4 03-Sep-2014  matt branches: 1.4.16;
Add __sync_val_compare_and_swap_{1,2} aliases for _atomic_cas_{8,16}
 1.3 09-Mar-2014  riastradh branches: 1.3.4; 1.3.6; 1.3.8;
Fix subword atomic_cas.

- Don't loop forever on failure.
- Don't shift away the result and return zero on success.

Evidently we lack automatic tests for these!
 1.2 07-Mar-2014  martin Fix return value (typo, noticed by Nick Hudson)
 1.1 24-Feb-2014  martin Provide cas_16 and cas_8 emulation via cas_32 and use that for mips64
 1.3.8.2 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.3.8.1 09-Mar-2014  tls file atomic_cas_by_cas32.c was added on branch tls-maxphys on 2014-08-19 23:45:14 +0000
 1.3.6.1 12-Nov-2014  snj Pull up following revision(s) (requested by martin in ticket #218):
common/lib/libc/arch/arm/atomic/Makefile.inc: revision 1.24-1.26
common/lib/libc/arch/hppa/atomic/Makefile.inc: revision 1.13
common/lib/libc/arch/mips/atomic/Makefile.inc: revision 1.13
common/lib/libc/arch/sh3/atomic/Makefile.inc: revision 1.7
common/lib/libc/arch/sparc/atomic/Makefile.inc: revision 1.18
common/lib/libc/arch/vax/atomic/Makefile.inc: revision 1.7
common/lib/libc/atomic/atomic_and_16_nv_cas.c: revision 1.2
common/lib/libc/atomic/atomic_and_8_nv_cas.c: revision 1.2
common/lib/libc/atomic/atomic_c11_compare_exchange_cas_16.c: revision 1.1-1.2
common/lib/libc/atomic/atomic_c11_compare_exchange_cas_32.c: revision 1.1-1.2
common/lib/libc/atomic/atomic_c11_compare_exchange_cas_8.c: revision 1.1-1.2
common/lib/libc/atomic/atomic_cas_by_cas32.c: revision 1.4
common/lib/libc/atomic/atomic_op_namespace.h: revision 1.7
Add __sync_val_compare_and_swap_{1,2} aliases for _atomic_cas_{8,16}
--
Provide __atomic_compare_exchange_N (as needed for the C11 2011 <atomic>
ops) via the corresponding CAS.
--
Hook __atomic_compare_exchange_N into vax libc.
--
Provide __sync_and_and_fetch_2 and __sync_and_and_fetch_1 for pre-ARMv6,
they are needed for the C++ 2011 <atomic> stuff.
--
Add C++ 2011 <atomic> support functions.
--
Move the and_{16,8}_nv sources into the right (libc only) block.
--
Provide <atomic> C++ 2011 support functions for mips and sh3.
--
Provide C++ 2011 <atomic> support functions for hppa and arm.
--
Provide prototypes to fix build with clang.
 1.3.4.2 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.3.4.1 09-Mar-2014  yamt file atomic_cas_by_cas32.c was added on branch yamt-pagecache on 2014-05-22 11:26:30 +0000
 1.4.16.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.4.16.1 21-Apr-2020  martin Sync with HEAD
 1.4 23-Jun-2008  ad Install manual pages for the atomic ops.
 1.3 30-Apr-2008  martin branches: 1.3.2;
Convert TNF licenses to new 2 clause variant
 1.2 28-Nov-2007  ad branches: 1.2.4; 1.2.8;
Support files from the thorpej-atomic branch.
 1.1 12-Apr-2007  thorpej branches: 1.1.2;
file atomic_dec.3 was initially added on branch thorpej-atomic.
 1.1.2.1 12-Apr-2007  thorpej Man pages for the atomic ops API.
 1.2.8.1 18-May-2008  yamt sync with head.
 1.2.4.2 09-Jan-2008  matt sync with HEAD
 1.2.4.1 28-Nov-2007  matt file atomic_dec.3 was added on branch matt-armv6 on 2008-01-09 01:21:21 +0000
 1.3.2.1 07-Sep-2008  wrstuden Delete a bunch of things that are now gone in head.
 1.1 13-Apr-2007  thorpej branches: 1.1.2;
file atomic_dec_16_add.c was initially added on branch thorpej-atomic.
 1.1.2.5 22-Apr-2007  thorpej Make sure namespace-cleansed aliases are avaialble for all atomic ops.
 1.1.2.4 18-Apr-2007  thorpej Fix a typo.
 1.1.2.3 17-Apr-2007  thorpej Work around pesky macro argument expansion rules.
 1.1.2.2 16-Apr-2007  thorpej Namespace protection for atomic operations.
 1.1.2.1 13-Apr-2007  thorpej Add generic atomic_inc and atomic_dec routines built on atomic_add.
 1.1 13-Apr-2007  thorpej branches: 1.1.2;
file atomic_dec_16_nv_add.c was initially added on branch thorpej-atomic.
 1.1.2.4 22-Apr-2007  thorpej Make sure namespace-cleansed aliases are avaialble for all atomic ops.
 1.1.2.3 17-Apr-2007  thorpej Work around pesky macro argument expansion rules.
 1.1.2.2 16-Apr-2007  thorpej Namespace protection for atomic operations.
 1.1.2.1 13-Apr-2007  thorpej Add generic atomic_inc and atomic_dec routines built on atomic_add.
 1.3 28-Apr-2008  martin branches: 1.3.64;
Remove clause 3 and 4 from TNF licenses
 1.2 28-Nov-2007  ad branches: 1.2.4; 1.2.8;
Support files from the thorpej-atomic branch.
 1.1 13-Apr-2007  thorpej branches: 1.1.2;
file atomic_dec_32_add.c was initially added on branch thorpej-atomic.
 1.1.2.5 22-Apr-2007  thorpej Make sure namespace-cleansed aliases are avaialble for all atomic ops.
 1.1.2.4 18-Apr-2007  thorpej Fix a typo.
 1.1.2.3 17-Apr-2007  thorpej Work around pesky macro argument expansion rules.
 1.1.2.2 16-Apr-2007  thorpej Namespace protection for atomic operations.
 1.1.2.1 13-Apr-2007  thorpej Add generic atomic_inc and atomic_dec routines built on atomic_add.
 1.2.8.1 18-May-2008  yamt sync with head.
 1.2.4.2 09-Jan-2008  matt sync with HEAD
 1.2.4.1 28-Nov-2007  matt file atomic_dec_32_add.c was added on branch matt-armv6 on 2008-01-09 01:21:22 +0000
 1.3.64.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.3.64.1 21-Apr-2020  martin Sync with HEAD
 1.3 28-Apr-2008  martin branches: 1.3.64;
Remove clause 3 and 4 from TNF licenses
 1.2 29-Nov-2007  ad branches: 1.2.4; 1.2.8;
atomic_add_* takes signed integers, the others take unsigned.
 1.1 29-Nov-2007  ad Add inc/dec variants that use atomic_cas in preference to atomic_add.
 1.2.8.1 18-May-2008  yamt sync with head.
 1.2.4.2 09-Jan-2008  matt sync with HEAD
 1.2.4.1 29-Nov-2007  matt file atomic_dec_32_cas.c was added on branch matt-armv6 on 2008-01-09 01:21:22 +0000
 1.3.64.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.3.64.1 21-Apr-2020  martin Sync with HEAD
 1.3 28-Apr-2008  martin branches: 1.3.64;
Remove clause 3 and 4 from TNF licenses
 1.2 28-Nov-2007  ad branches: 1.2.4; 1.2.8;
Support files from the thorpej-atomic branch.
 1.1 13-Apr-2007  thorpej branches: 1.1.2;
file atomic_dec_32_nv_add.c was initially added on branch thorpej-atomic.
 1.1.2.4 22-Apr-2007  thorpej Make sure namespace-cleansed aliases are avaialble for all atomic ops.
 1.1.2.3 17-Apr-2007  thorpej Work around pesky macro argument expansion rules.
 1.1.2.2 16-Apr-2007  thorpej Namespace protection for atomic operations.
 1.1.2.1 13-Apr-2007  thorpej Add generic atomic_inc and atomic_dec routines built on atomic_add.
 1.2.8.1 18-May-2008  yamt sync with head.
 1.2.4.2 09-Jan-2008  matt sync with HEAD
 1.2.4.1 28-Nov-2007  matt file atomic_dec_32_nv_add.c was added on branch matt-armv6 on 2008-01-09 01:21:23 +0000
 1.3.64.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.3.64.1 21-Apr-2020  martin Sync with HEAD
 1.3 28-Apr-2008  martin branches: 1.3.64;
Remove clause 3 and 4 from TNF licenses
 1.2 29-Nov-2007  ad branches: 1.2.4; 1.2.8;
atomic_add_* takes signed integers, the others take unsigned.
 1.1 29-Nov-2007  ad Add inc/dec variants that use atomic_cas in preference to atomic_add.
 1.2.8.1 18-May-2008  yamt sync with head.
 1.2.4.2 09-Jan-2008  matt sync with HEAD
 1.2.4.1 29-Nov-2007  matt file atomic_dec_32_nv_cas.c was added on branch matt-armv6 on 2008-01-09 01:21:23 +0000
 1.3.64.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.3.64.1 21-Apr-2020  martin Sync with HEAD
 1.4 28-Apr-2008  martin branches: 1.4.64;
Remove clause 3 and 4 from TNF licenses
 1.3 29-Nov-2007  ad branches: 1.3.4; 1.3.8;
Only build the 64-bit versions if __HAVE_ATOMIC64_OPS.
 1.2 28-Nov-2007  ad Support files from the thorpej-atomic branch.
 1.1 13-Apr-2007  thorpej branches: 1.1.2;
file atomic_dec_64_add.c was initially added on branch thorpej-atomic.
 1.1.2.5 22-Apr-2007  thorpej Make sure namespace-cleansed aliases are avaialble for all atomic ops.
 1.1.2.4 18-Apr-2007  thorpej Fix a typo.
 1.1.2.3 17-Apr-2007  thorpej Work around pesky macro argument expansion rules.
 1.1.2.2 16-Apr-2007  thorpej Namespace protection for atomic operations.
 1.1.2.1 13-Apr-2007  thorpej Add generic atomic_inc and atomic_dec routines built on atomic_add.
 1.3.8.1 18-May-2008  yamt sync with head.
 1.3.4.2 09-Jan-2008  matt sync with HEAD
 1.3.4.1 29-Nov-2007  matt file atomic_dec_64_add.c was added on branch matt-armv6 on 2008-01-09 01:21:24 +0000
 1.4.64.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.4.64.1 21-Apr-2020  martin Sync with HEAD
 1.3 28-Apr-2008  martin branches: 1.3.64;
Remove clause 3 and 4 from TNF licenses
 1.2 29-Nov-2007  ad branches: 1.2.4; 1.2.8;
atomic_add_* takes signed integers, the others take unsigned.
 1.1 29-Nov-2007  ad Add inc/dec variants that use atomic_cas in preference to atomic_add.
 1.2.8.1 18-May-2008  yamt sync with head.
 1.2.4.2 09-Jan-2008  matt sync with HEAD
 1.2.4.1 29-Nov-2007  matt file atomic_dec_64_cas.c was added on branch matt-armv6 on 2008-01-09 01:21:24 +0000
 1.3.64.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.3.64.1 21-Apr-2020  martin Sync with HEAD
 1.4 28-Apr-2008  martin branches: 1.4.64;
Remove clause 3 and 4 from TNF licenses
 1.3 29-Nov-2007  ad branches: 1.3.4; 1.3.8;
Only build the 64-bit versions if __HAVE_ATOMIC64_OPS.
 1.2 28-Nov-2007  ad Support files from the thorpej-atomic branch.
 1.1 13-Apr-2007  thorpej branches: 1.1.2;
file atomic_dec_64_nv_add.c was initially added on branch thorpej-atomic.
 1.1.2.4 22-Apr-2007  thorpej Make sure namespace-cleansed aliases are avaialble for all atomic ops.
 1.1.2.3 17-Apr-2007  thorpej Work around pesky macro argument expansion rules.
 1.1.2.2 16-Apr-2007  thorpej Namespace protection for atomic operations.
 1.1.2.1 13-Apr-2007  thorpej Add generic atomic_inc and atomic_dec routines built on atomic_add.
 1.3.8.1 18-May-2008  yamt sync with head.
 1.3.4.2 09-Jan-2008  matt sync with HEAD
 1.3.4.1 29-Nov-2007  matt file atomic_dec_64_nv_add.c was added on branch matt-armv6 on 2008-01-09 01:21:25 +0000
 1.4.64.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.4.64.1 21-Apr-2020  martin Sync with HEAD
 1.3 28-Apr-2008  martin branches: 1.3.64;
Remove clause 3 and 4 from TNF licenses
 1.2 29-Nov-2007  ad branches: 1.2.4; 1.2.8;
atomic_add_* takes signed integers, the others take unsigned.
 1.1 29-Nov-2007  ad Add inc/dec variants that use atomic_cas in preference to atomic_add.
 1.2.8.1 18-May-2008  yamt sync with head.
 1.2.4.2 09-Jan-2008  matt sync with HEAD
 1.2.4.1 29-Nov-2007  matt file atomic_dec_64_nv_cas.c was added on branch matt-armv6 on 2008-01-09 01:21:25 +0000
 1.3.64.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.3.64.1 21-Apr-2020  martin Sync with HEAD
 1.1 13-Apr-2007  thorpej branches: 1.1.2;
file atomic_dec_8_add.c was initially added on branch thorpej-atomic.
 1.1.2.4 22-Apr-2007  thorpej Make sure namespace-cleansed aliases are avaialble for all atomic ops.
 1.1.2.3 17-Apr-2007  thorpej Work around pesky macro argument expansion rules.
 1.1.2.2 16-Apr-2007  thorpej Namespace protection for atomic operations.
 1.1.2.1 13-Apr-2007  thorpej Add generic atomic_inc and atomic_dec routines built on atomic_add.
 1.1 13-Apr-2007  thorpej branches: 1.1.2;
file atomic_dec_8_nv_add.c was initially added on branch thorpej-atomic.
 1.1.2.4 22-Apr-2007  thorpej Make sure namespace-cleansed aliases are avaialble for all atomic ops.
 1.1.2.3 17-Apr-2007  thorpej Work around pesky macro argument expansion rules.
 1.1.2.2 16-Apr-2007  thorpej Namespace protection for atomic operations.
 1.1.2.1 13-Apr-2007  thorpej Add generic atomic_inc and atomic_dec routines built on atomic_add.
 1.4 23-Jun-2008  ad Install manual pages for the atomic ops.
 1.3 30-Apr-2008  martin branches: 1.3.2;
Convert TNF licenses to new 2 clause variant
 1.2 28-Nov-2007  ad branches: 1.2.4; 1.2.8;
Support files from the thorpej-atomic branch.
 1.1 12-Apr-2007  thorpej branches: 1.1.2;
file atomic_inc.3 was initially added on branch thorpej-atomic.
 1.1.2.1 12-Apr-2007  thorpej Man pages for the atomic ops API.
 1.2.8.1 18-May-2008  yamt sync with head.
 1.2.4.2 09-Jan-2008  matt sync with HEAD
 1.2.4.1 28-Nov-2007  matt file atomic_inc.3 was added on branch matt-armv6 on 2008-01-09 01:21:26 +0000
 1.3.2.1 07-Sep-2008  wrstuden Delete a bunch of things that are now gone in head.
 1.1 13-Apr-2007  thorpej branches: 1.1.2;
file atomic_inc_16_add.c was initially added on branch thorpej-atomic.
 1.1.2.5 22-Apr-2007  thorpej Make sure namespace-cleansed aliases are avaialble for all atomic ops.
 1.1.2.4 18-Apr-2007  thorpej Fix a typo.
 1.1.2.3 17-Apr-2007  thorpej Work around pesky macro argument expansion rules.
 1.1.2.2 16-Apr-2007  thorpej Namespace protection for atomic operations.
 1.1.2.1 13-Apr-2007  thorpej Add generic atomic_inc and atomic_dec routines built on atomic_add.
 1.1 13-Apr-2007  thorpej branches: 1.1.2;
file atomic_inc_16_nv_add.c was initially added on branch thorpej-atomic.
 1.1.2.4 22-Apr-2007  thorpej Make sure namespace-cleansed aliases are avaialble for all atomic ops.
 1.1.2.3 17-Apr-2007  thorpej Work around pesky macro argument expansion rules.
 1.1.2.2 16-Apr-2007  thorpej Namespace protection for atomic operations.
 1.1.2.1 13-Apr-2007  thorpej Add generic atomic_inc and atomic_dec routines built on atomic_add.
 1.3 28-Apr-2008  martin branches: 1.3.64;
Remove clause 3 and 4 from TNF licenses
 1.2 28-Nov-2007  ad branches: 1.2.4; 1.2.8;
Support files from the thorpej-atomic branch.
 1.1 13-Apr-2007  thorpej branches: 1.1.2;
file atomic_inc_32_add.c was initially added on branch thorpej-atomic.
 1.1.2.5 22-Apr-2007  thorpej Make sure namespace-cleansed aliases are avaialble for all atomic ops.
 1.1.2.4 18-Apr-2007  thorpej Fix a typo.
 1.1.2.3 17-Apr-2007  thorpej Work around pesky macro argument expansion rules.
 1.1.2.2 16-Apr-2007  thorpej Namespace protection for atomic operations.
 1.1.2.1 13-Apr-2007  thorpej Add generic atomic_inc and atomic_dec routines built on atomic_add.
 1.2.8.1 18-May-2008  yamt sync with head.
 1.2.4.2 09-Jan-2008  matt sync with HEAD
 1.2.4.1 28-Nov-2007  matt file atomic_inc_32_add.c was added on branch matt-armv6 on 2008-01-09 01:21:26 +0000
 1.3.64.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.3.64.1 21-Apr-2020  martin Sync with HEAD
 1.3 28-Apr-2008  martin branches: 1.3.64;
Remove clause 3 and 4 from TNF licenses
 1.2 29-Nov-2007  ad branches: 1.2.4; 1.2.8;
atomic_add_* takes signed integers, the others take unsigned.
 1.1 29-Nov-2007  ad Add inc/dec variants that use atomic_cas in preference to atomic_add.
 1.2.8.1 18-May-2008  yamt sync with head.
 1.2.4.2 09-Jan-2008  matt sync with HEAD
 1.2.4.1 29-Nov-2007  matt file atomic_inc_32_cas.c was added on branch matt-armv6 on 2008-01-09 01:21:27 +0000
 1.3.64.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.3.64.1 21-Apr-2020  martin Sync with HEAD
 1.3 28-Apr-2008  martin branches: 1.3.64;
Remove clause 3 and 4 from TNF licenses
 1.2 28-Nov-2007  ad branches: 1.2.4; 1.2.8;
Support files from the thorpej-atomic branch.
 1.1 13-Apr-2007  thorpej branches: 1.1.2;
file atomic_inc_32_nv_add.c was initially added on branch thorpej-atomic.
 1.1.2.4 22-Apr-2007  thorpej Make sure namespace-cleansed aliases are avaialble for all atomic ops.
 1.1.2.3 17-Apr-2007  thorpej Work around pesky macro argument expansion rules.
 1.1.2.2 16-Apr-2007  thorpej Namespace protection for atomic operations.
 1.1.2.1 13-Apr-2007  thorpej Add generic atomic_inc and atomic_dec routines built on atomic_add.
 1.2.8.1 18-May-2008  yamt sync with head.
 1.2.4.2 09-Jan-2008  matt sync with HEAD
 1.2.4.1 28-Nov-2007  matt file atomic_inc_32_nv_add.c was added on branch matt-armv6 on 2008-01-09 01:21:27 +0000
 1.3.64.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.3.64.1 21-Apr-2020  martin Sync with HEAD
 1.3 28-Apr-2008  martin branches: 1.3.64;
Remove clause 3 and 4 from TNF licenses
 1.2 29-Nov-2007  ad branches: 1.2.4; 1.2.8;
atomic_add_* takes signed integers, the others take unsigned.
 1.1 29-Nov-2007  ad Add inc/dec variants that use atomic_cas in preference to atomic_add.
 1.2.8.1 18-May-2008  yamt sync with head.
 1.2.4.2 09-Jan-2008  matt sync with HEAD
 1.2.4.1 29-Nov-2007  matt file atomic_inc_32_nv_cas.c was added on branch matt-armv6 on 2008-01-09 01:21:27 +0000
 1.3.64.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.3.64.1 21-Apr-2020  martin Sync with HEAD
 1.4 28-Apr-2008  martin branches: 1.4.64;
Remove clause 3 and 4 from TNF licenses
 1.3 29-Nov-2007  ad branches: 1.3.4; 1.3.8;
Only build the 64-bit versions if __HAVE_ATOMIC64_OPS.
 1.2 28-Nov-2007  ad Support files from the thorpej-atomic branch.
 1.1 13-Apr-2007  thorpej branches: 1.1.2;
file atomic_inc_64_add.c was initially added on branch thorpej-atomic.
 1.1.2.5 22-Apr-2007  thorpej Make sure namespace-cleansed aliases are avaialble for all atomic ops.
 1.1.2.4 18-Apr-2007  thorpej Fix a typo.
 1.1.2.3 17-Apr-2007  thorpej Work around pesky macro argument expansion rules.
 1.1.2.2 16-Apr-2007  thorpej Namespace protection for atomic operations.
 1.1.2.1 13-Apr-2007  thorpej Add generic atomic_inc and atomic_dec routines built on atomic_add.
 1.3.8.1 18-May-2008  yamt sync with head.
 1.3.4.2 09-Jan-2008  matt sync with HEAD
 1.3.4.1 29-Nov-2007  matt file atomic_inc_64_add.c was added on branch matt-armv6 on 2008-01-09 01:21:28 +0000
 1.4.64.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.4.64.1 21-Apr-2020  martin Sync with HEAD
 1.3 28-Apr-2008  martin branches: 1.3.64;
Remove clause 3 and 4 from TNF licenses
 1.2 29-Nov-2007  ad branches: 1.2.4; 1.2.8;
atomic_add_* takes signed integers, the others take unsigned.
 1.1 29-Nov-2007  ad Add inc/dec variants that use atomic_cas in preference to atomic_add.
 1.2.8.1 18-May-2008  yamt sync with head.
 1.2.4.2 09-Jan-2008  matt sync with HEAD
 1.2.4.1 29-Nov-2007  matt file atomic_inc_64_cas.c was added on branch matt-armv6 on 2008-01-09 01:21:28 +0000
 1.3.64.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.3.64.1 21-Apr-2020  martin Sync with HEAD
 1.4 28-Apr-2008  martin branches: 1.4.64;
Remove clause 3 and 4 from TNF licenses
 1.3 29-Nov-2007  ad branches: 1.3.4; 1.3.8;
Only build the 64-bit versions if __HAVE_ATOMIC64_OPS.
 1.2 28-Nov-2007  ad Support files from the thorpej-atomic branch.
 1.1 13-Apr-2007  thorpej branches: 1.1.2;
file atomic_inc_64_nv_add.c was initially added on branch thorpej-atomic.
 1.1.2.4 22-Apr-2007  thorpej Make sure namespace-cleansed aliases are avaialble for all atomic ops.
 1.1.2.3 17-Apr-2007  thorpej Work around pesky macro argument expansion rules.
 1.1.2.2 16-Apr-2007  thorpej Namespace protection for atomic operations.
 1.1.2.1 13-Apr-2007  thorpej Add generic atomic_inc and atomic_dec routines built on atomic_add.
 1.3.8.1 18-May-2008  yamt sync with head.
 1.3.4.2 09-Jan-2008  matt sync with HEAD
 1.3.4.1 29-Nov-2007  matt file atomic_inc_64_nv_add.c was added on branch matt-armv6 on 2008-01-09 01:21:28 +0000
 1.4.64.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.4.64.1 21-Apr-2020  martin Sync with HEAD
 1.4 28-Apr-2008  martin branches: 1.4.64;
Remove clause 3 and 4 from TNF licenses
 1.3 04-Dec-2007  he branches: 1.3.4; 1.3.8;
Correct what must obviously be a cut+paste-induced set of typos.
 1.2 29-Nov-2007  ad atomic_add_* takes signed integers, the others take unsigned.
 1.1 29-Nov-2007  ad Add inc/dec variants that use atomic_cas in preference to atomic_add.
 1.3.8.1 18-May-2008  yamt sync with head.
 1.3.4.2 09-Jan-2008  matt sync with HEAD
 1.3.4.1 04-Dec-2007  matt file atomic_inc_64_nv_cas.c was added on branch matt-armv6 on 2008-01-09 01:21:29 +0000
 1.4.64.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.4.64.1 21-Apr-2020  martin Sync with HEAD
 1.1 13-Apr-2007  thorpej branches: 1.1.2;
file atomic_inc_8_add.c was initially added on branch thorpej-atomic.
 1.1.2.4 22-Apr-2007  thorpej Make sure namespace-cleansed aliases are avaialble for all atomic ops.
 1.1.2.3 17-Apr-2007  thorpej Work around pesky macro argument expansion rules.
 1.1.2.2 16-Apr-2007  thorpej Namespace protection for atomic operations.
 1.1.2.1 13-Apr-2007  thorpej Add generic atomic_inc and atomic_dec routines built on atomic_add.
 1.1 13-Apr-2007  thorpej branches: 1.1.2;
file atomic_inc_8_nv_add.c was initially added on branch thorpej-atomic.
 1.1.2.4 22-Apr-2007  thorpej Make sure namespace-cleansed aliases are avaialble for all atomic ops.
 1.1.2.3 17-Apr-2007  thorpej Work around pesky macro argument expansion rules.
 1.1.2.2 16-Apr-2007  thorpej Namespace protection for atomic operations.
 1.1.2.1 13-Apr-2007  thorpej Add generic atomic_inc and atomic_dec routines built on atomic_add.
 1.5 19-Jan-2024  christos make decls consistent
 1.4 21-Aug-2013  matt branches: 1.4.26;
Use __section(".test.startup") for the init routines
 1.3 28-Apr-2008  martin branches: 1.3.4; 1.3.26;
Remove clause 3 and 4 from TNF licenses
 1.2 10-Feb-2008  ad branches: 1.2.4; 1.2.6;
Fix name of the init function.
 1.1 09-Feb-2008  ad Some glue for atomic ops in userspace.
 1.2.6.1 18-May-2008  yamt sync with head.
 1.2.4.2 23-Mar-2008  matt sync with HEAD
 1.2.4.1 10-Feb-2008  matt file atomic_init_cas.c was added on branch matt-armv6 on 2008-03-23 00:12:44 +0000
 1.3.26.1 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.3.4.1 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.4.26.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.4.26.1 21-Apr-2020  martin Sync with HEAD
 1.19 21-Jan-2024  thorpej Catch up with revision 1.5 of atomic_init_cas.c. Fixes building libc
on sun2, and probably others.
 1.18 19-Jan-2024  christos make decls consistent
 1.17 15-May-2020  martin PR 55239: initialize all RAS sections for non-MP configurations
 1.16 18-Feb-2019  martin Add some atomic_cas_64_ni aliases
 1.15 27-Feb-2014  matt branches: 1.15.26;
Add atomic_cas_64 support for ARM EABI on V5TE and V5TEJ cpus.
(strd is atomic).
 1.14 24-Feb-2014  martin Provide CAS variants for 16 and 8 bit when running with more that 1 cpu
 1.13 22-Feb-2014  martin Try to hide the C runtime implementation specific __sync_* ops from librump,
to avoid duplicates.
 1.12 29-Jan-2014  martin Fix a few more copy&pastos - now it actually builds.
 1.11 29-Jan-2014  martin Fix copy&pasto
 1.10 27-Jan-2014  matt Add _atomic_cas_16 and _atomic_cas_8 and their aliases
__sync_val_compare_and_swap_2 & __sync_val_compare_and_swap_1
 1.9 21-Aug-2013  matt Use __section(".test.startup") for the init routines
 1.8 28-Feb-2012  joerg branches: 1.8.2;
Make sure to create the right aliases.
 1.7 14-Dec-2009  matt branches: 1.7.6;
Merge from matt-nb5-mips64
 1.6 30-Jan-2009  skrll Make sure _atomic_cas_fn is initialised for "weirdo environments" such
as ramdisks where the constructor doesn't get called.

Fixes ifconfig in hp700 sysinst.

OK'ed by ad.
 1.5 29-Apr-2008  scw branches: 1.5.10;
Implement _atomic_cas_up() in assembly code as the compiler cannot be
trusted to generate fully restartable code sequences.

Addresses lib/38482 for ARM and m68000.
 1.4 28-Apr-2008  martin Remove clause 3 and 4 from TNF licenses
 1.3 11-Feb-2008  ad branches: 1.3.4; 1.3.6;
Put back a __predict_false removed during testing and fix atomic_cas_mp().
 1.2 10-Feb-2008  ad Add atomic_cas_foo_ni().
 1.1 09-Feb-2008  ad Some glue for atomic ops in userspace.
 1.3.6.1 18-May-2008  yamt sync with head.
 1.3.4.2 23-Mar-2008  matt sync with HEAD
 1.3.4.1 11-Feb-2008  matt file atomic_init_testset.c was added on branch matt-armv6 on 2008-03-23 00:12:44 +0000
 1.5.10.1 05-Sep-2009  matt Don't cast a ptr to uint32_t, use uintptr_t instead.
 1.7.6.2 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.7.6.1 17-Apr-2012  yamt sync with head
 1.8.2.1 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.15.26.3 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.15.26.2 21-Apr-2020  martin Sync with HEAD
 1.15.26.1 10-Jun-2019  christos Sync with HEAD
 1.3 03-Apr-2023  riastradh branches: 1.3.2;
libc atomic: Make previous work a little less accidentally.
 1.2 03-Apr-2023  riastradh libc __atomic_is_lock_free: Fix clang build with symbol hacks.
 1.1 30-Mar-2023  riastradh libc: Define __atomic_is_lock_free.

Limited to architectures where it is actually needed by gcc for any
calls to stdatomic.h atomic_is_lock_free for now.

We should also add it to other architectures too, along with lockful
atomic r/m/w operations for sizes that can't be handled natively, but
that's a lot more work. It is also necessary for -fno-inline-atomics
but we're missing a lot of other symbols for that too, to be fixed.
For now, this should enable the OpenSSL build to complete on these
architectures again after I reverted a local change.

XXX pullup-10
 1.3.2.3 14-Dec-2023  martin Pull up following revision(s) (requested by rin in ticket #496):

common/lib/libc/atomic/atomic_is_lock_free.c: revision 1.2
common/lib/libc/atomic/atomic_is_lock_free.c: revision 1.3

libc __atomic_is_lock_free: Fix clang build with symbol hacks.

libc atomic: Make previous work a little less accidentally.
 1.3.2.2 31-Jul-2023  martin Pull up following revision(s) (requested by riastradh in ticket #275):

common/lib/libc/arch/sparc/atomic/Makefile.inc: revision 1.24
common/lib/libc/arch/m68k/atomic/Makefile.inc: revision 1.16
common/lib/libc/arch/mips/atomic/Makefile.inc: revision 1.16
common/lib/libc/arch/hppa/atomic/Makefile.inc: revision 1.15
common/lib/libc/arch/vax/atomic/Makefile.inc: revision 1.9
common/lib/libc/atomic/atomic_is_lock_free.c: revision 1.1
common/lib/libc/arch/sh3/atomic/Makefile.inc: revision 1.9

libc: Define __atomic_is_lock_free.

Limited to architectures where it is actually needed by gcc for any
calls to stdatomic.h atomic_is_lock_free for now.

We should also add it to other architectures too, along with lockful
atomic r/m/w operations for sizes that can't be handled natively, but
that's a lot more work. It is also necessary for -fno-inline-atomics
but we're missing a lot of other symbols for that too, to be fixed.

For now, this should enable the OpenSSL build to complete on these
architectures again after I reverted a local change.
 1.3.2.1 03-Apr-2023  martin file atomic_is_lock_free.c was added on branch netbsd-10 on 2023-07-31 16:10:26 +0000
 1.5 14-Feb-2024  christos sprinkle unused.
 1.4 09-Apr-2022  riastradh libc/atomic: Fix membars in __atomic_load/store_* stubs.

- membar_enter/exit ordering was backwards.
- membar_enter doesn't make any sense for load anyway.
- Switch to membar_release for store and membar_acquire for load.

The only sensible orderings for a simple load or store are acquire or
release, respectively, or sequential consistency. This never
provided correct sequential consistency before -- we should really
make it conditional on memmodel but I don't know offhand what the
values of memmodel might be and this is at least better than before.
 1.3 07-Sep-2020  mrg make some prototypes match the builtin properly. GCC 9 complains
with the old version, GCC 8 is happy with this version.

tested on sparc.
 1.2 06-Jul-2014  joerg branches: 1.2.2; 1.2.6; 1.2.28;
Add unused memory model argument to make GCC happy.
 1.1 05-Jul-2014  joerg Provide a basic implementation of __atomic_load_* and __atomic_store_*,
used by GCC and LLVM as backing for C11/C++11 atomics, if the hardware
is not known to have corresponding features. Include it on ARM as LLVM
and libc++ hit it when compiled for ARMv4.
 1.2.28.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.2.28.1 21-Apr-2020  martin Sync with HEAD
 1.2.6.2 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.2.6.1 06-Jul-2014  tls file atomic_load.c was added on branch tls-maxphys on 2014-08-19 23:45:14 +0000
 1.2.2.2 10-Aug-2014  tls Rebase.
 1.2.2.1 06-Jul-2014  tls file atomic_load.c was added on branch tls-earlyentropy on 2014-08-10 06:47:06 +0000
 1.3 01-Mar-2019  isaki Add missing alias for __atomic_fetch_nand_N.
XXX I'm not sure why does llvm/sparc call __atomic instead of
__sync though.
XXX atomic_*_cas.c should be more symmetric...
 1.2 21-Feb-2014  martin branches: 1.2.4; 1.2.8; 1.2.30;
Add a few missing namespace includes
 1.1 21-Feb-2014  martin Provide 8 and 16 bit sync ops (using 16 bit and 8 bit cas)
 1.2.30.3 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.2.30.2 21-Apr-2020  martin Sync with HEAD
 1.2.30.1 10-Jun-2019  christos Sync with HEAD
 1.2.8.2 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.2.8.1 21-Feb-2014  tls file atomic_nand_16_cas.c was added on branch tls-maxphys on 2014-08-19 23:45:14 +0000
 1.2.4.2 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.2.4.1 21-Feb-2014  yamt file atomic_nand_16_cas.c was added on branch yamt-pagecache on 2014-05-22 11:26:30 +0000
 1.2 01-Mar-2019  isaki Add missing alias for __atomic_fetch_nand_N.
XXX I'm not sure why does llvm/sparc call __atomic instead of
__sync though.
XXX atomic_*_cas.c should be more symmetric...
 1.1 21-Feb-2014  martin branches: 1.1.4; 1.1.8; 1.1.30;
Provide 32bit __sync ops based on atomic_cas_32.
 1.1.30.3 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.1.30.2 21-Apr-2020  martin Sync with HEAD
 1.1.30.1 10-Jun-2019  christos Sync with HEAD
 1.1.8.2 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.1.8.1 21-Feb-2014  tls file atomic_nand_32_cas.c was added on branch tls-maxphys on 2014-08-19 23:45:14 +0000
 1.1.4.2 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.1.4.1 21-Feb-2014  yamt file atomic_nand_32_cas.c was added on branch yamt-pagecache on 2014-05-22 11:26:30 +0000
 1.2 18-Feb-2014  martin branches: 1.2.4; 1.2.8; 1.2.30;
Make the _and_fetch_8 primitives return the new value they calculated
and set - not whatever is found in memory later.
 1.1 18-Feb-2014  martin Provide most missing __sync_*64 primitives for i386
 1.2.30.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.2.30.1 21-Apr-2020  martin Sync with HEAD
 1.2.8.2 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.2.8.1 18-Feb-2014  tls file atomic_nand_64_cas.c was added on branch tls-maxphys on 2014-08-19 23:45:14 +0000
 1.2.4.2 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.2.4.1 18-Feb-2014  yamt file atomic_nand_64_cas.c was added on branch yamt-pagecache on 2014-05-22 11:26:30 +0000
 1.3 01-Mar-2019  isaki Add missing alias for __atomic_fetch_nand_N.
XXX I'm not sure why does llvm/sparc call __atomic instead of
__sync though.
XXX atomic_*_cas.c should be more symmetric...
 1.2 21-Feb-2014  martin branches: 1.2.4; 1.2.8; 1.2.30;
Add a few missing namespace includes
 1.1 21-Feb-2014  martin Provide 8 and 16 bit sync ops (using 16 bit and 8 bit cas)
 1.2.30.3 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.2.30.2 21-Apr-2020  martin Sync with HEAD
 1.2.30.1 10-Jun-2019  christos Sync with HEAD
 1.2.8.2 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.2.8.1 21-Feb-2014  tls file atomic_nand_8_cas.c was added on branch tls-maxphys on 2014-08-19 23:45:14 +0000
 1.2.4.2 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.2.4.1 21-Feb-2014  yamt file atomic_nand_8_cas.c was added on branch yamt-pagecache on 2014-05-22 11:26:30 +0000
 1.1 13-Apr-2007  thorpej branches: 1.1.2;
file atomic_op_cas_impl.h was initially added on branch thorpej-atomic.
 1.1.2.3 20-Apr-2007  thorpej Use membar_enter() (W-RW) as the memory barrier in CAS-based atomic ops.
 1.1.2.2 13-Apr-2007  thorpej Make this work for big-endian with a trick provided by Matt Thomas.
 1.1.2.1 13-Apr-2007  thorpej Add a header file that contains convenience macros for implementing
other types of atomic operations in terms of atomic_cas_32() (including
sub-32-bit operations).

XXX Probably does not work on big-endian yet.
 1.8 04-Mar-2025  riastradh libc atomic_op_namespace.h: Add membar_release and membar_acquire.

These namespace macros will be needed so that libpthread can call
NetBSD's nonstandard membar_release/acquire functions even if linked
against an application that defines symbols of the same name.

Preparation for:

PR lib/59126: pthread_once(3): missing memory ordering
 1.7 13-Oct-2014  martin branches: 1.7.16; 1.7.36;
Provide __sync_and_and_fetch_2 and __sync_and_and_fetch_1 for pre-ARMv6,
they are needed for the C++ 2011 <atomic> stuff.
 1.6 22-Feb-2014  martin branches: 1.6.4;
Try to hide the C runtime implementation specific __sync_* ops from librump,
to avoid duplicates.
 1.5 21-Feb-2014  martin Provide 8 and 16 bit sync ops (using 16 bit and 8 bit cas)
 1.4 23-Jun-2008  ad branches: 1.4.2; 1.4.24;
+atomic_cas_*_ni
 1.3 28-Apr-2008  martin branches: 1.3.2;
Remove clause 3 and 4 from TNF licenses
 1.2 28-Nov-2007  ad branches: 1.2.4; 1.2.8;
Support files from the thorpej-atomic branch.
 1.1 16-Apr-2007  thorpej branches: 1.1.2;
file atomic_op_namespace.h was initially added on branch thorpej-atomic.
 1.1.2.1 16-Apr-2007  thorpej Namespace protection for atomic operations.
 1.2.8.1 18-May-2008  yamt sync with head.
 1.2.4.2 09-Jan-2008  matt sync with HEAD
 1.2.4.1 28-Nov-2007  matt file atomic_op_namespace.h was added on branch matt-armv6 on 2008-01-09 01:21:29 +0000
 1.3.2.1 18-Sep-2008  wrstuden Sync with wrstuden-revivesa-base-2.
 1.4.24.1 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.4.2.1 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.6.4.1 12-Nov-2014  snj Pull up following revision(s) (requested by martin in ticket #218):
common/lib/libc/arch/arm/atomic/Makefile.inc: revision 1.24-1.26
common/lib/libc/arch/hppa/atomic/Makefile.inc: revision 1.13
common/lib/libc/arch/mips/atomic/Makefile.inc: revision 1.13
common/lib/libc/arch/sh3/atomic/Makefile.inc: revision 1.7
common/lib/libc/arch/sparc/atomic/Makefile.inc: revision 1.18
common/lib/libc/arch/vax/atomic/Makefile.inc: revision 1.7
common/lib/libc/atomic/atomic_and_16_nv_cas.c: revision 1.2
common/lib/libc/atomic/atomic_and_8_nv_cas.c: revision 1.2
common/lib/libc/atomic/atomic_c11_compare_exchange_cas_16.c: revision 1.1-1.2
common/lib/libc/atomic/atomic_c11_compare_exchange_cas_32.c: revision 1.1-1.2
common/lib/libc/atomic/atomic_c11_compare_exchange_cas_8.c: revision 1.1-1.2
common/lib/libc/atomic/atomic_cas_by_cas32.c: revision 1.4
common/lib/libc/atomic/atomic_op_namespace.h: revision 1.7
Add __sync_val_compare_and_swap_{1,2} aliases for _atomic_cas_{8,16}
--
Provide __atomic_compare_exchange_N (as needed for the C11 2011 <atomic>
ops) via the corresponding CAS.
--
Hook __atomic_compare_exchange_N into vax libc.
--
Provide __sync_and_and_fetch_2 and __sync_and_and_fetch_1 for pre-ARMv6,
they are needed for the C++ 2011 <atomic> stuff.
--
Add C++ 2011 <atomic> support functions.
--
Move the and_{16,8}_nv sources into the right (libc only) block.
--
Provide <atomic> C++ 2011 support functions for mips and sh3.
--
Provide C++ 2011 <atomic> support functions for hppa and arm.
--
Provide prototypes to fix build with clang.
 1.7.36.1 02-Aug-2025  perseant Sync with HEAD
 1.7.16.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.7.16.1 21-Apr-2020  martin Sync with HEAD
 1.6 23-Jun-2008  ad Install manual pages for the atomic ops.
 1.5 30-Apr-2008  martin branches: 1.5.2;
Convert TNF licenses to new 2 clause variant
 1.4 11-Feb-2008  ad branches: 1.4.4;
Note many of the restrictions, and clarify the section on global
visibility.
 1.3 02-Dec-2007  wiz branches: 1.3.4;
Typo fixes.
 1.2 28-Nov-2007  ad Support files from the thorpej-atomic branch.
 1.1 12-Apr-2007  thorpej branches: 1.1.2;
file atomic_ops.3 was initially added on branch thorpej-atomic.
 1.1.2.3 27-Apr-2007  thorpej Note that a given memory location should be modified either entirely
with atomic operations or entirely with some other synchronization
strategy, else the results are undefined.

This is critical for correctness on platforms that use interlocks
or restartable atomic sequences.
 1.1.2.2 26-Apr-2007  thorpej Make a note that sub-32-bit atomic ops are extremely expensive on some
architectures.
 1.1.2.1 12-Apr-2007  thorpej Man pages for the atomic ops API.
 1.3.4.3 23-Mar-2008  matt sync with HEAD
 1.3.4.2 09-Jan-2008  matt sync with HEAD
 1.3.4.1 02-Dec-2007  matt file atomic_ops.3 was added on branch matt-armv6 on 2008-01-09 01:21:30 +0000
 1.4.4.1 18-May-2008  yamt sync with head.
 1.5.2.1 07-Sep-2008  wrstuden Delete a bunch of things that are now gone in head.
 1.4 23-Jun-2008  ad Install manual pages for the atomic ops.
 1.3 30-Apr-2008  martin branches: 1.3.2;
Convert TNF licenses to new 2 clause variant
 1.2 28-Nov-2007  ad branches: 1.2.4; 1.2.8;
Support files from the thorpej-atomic branch.
 1.1 12-Apr-2007  thorpej branches: 1.1.2;
file atomic_or.3 was initially added on branch thorpej-atomic.
 1.1.2.1 12-Apr-2007  thorpej Man pages for the atomic ops API.
 1.2.8.1 18-May-2008  yamt sync with head.
 1.2.4.2 09-Jan-2008  matt sync with HEAD
 1.2.4.1 28-Nov-2007  matt file atomic_or.3 was added on branch matt-armv6 on 2008-01-09 01:21:30 +0000
 1.3.2.1 07-Sep-2008  wrstuden Delete a bunch of things that are now gone in head.
 1.4 01-Mar-2019  isaki Fix return value. or_and_fetch should return new value.
 1.3 23-Jun-2014  joerg branches: 1.3.4; 1.3.26;
Add aliases for the builtins used to implement C11/C++11 atomics.
 1.2 21-Feb-2014  martin branches: 1.2.2;
Provide 8 and 16 bit sync ops (using 16 bit and 8 bit cas)
 1.1 13-Apr-2007  thorpej branches: 1.1.2; 1.1.24;
file atomic_or_16_cas.c was initially added on branch thorpej-atomic.
 1.1.24.1 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.1.2.4 22-Apr-2007  thorpej Make sure namespace-cleansed aliases are avaialble for all atomic ops.
 1.1.2.3 17-Apr-2007  thorpej Work around pesky macro argument expansion rules.
 1.1.2.2 16-Apr-2007  thorpej Namespace protection for atomic operations.
 1.1.2.1 13-Apr-2007  thorpej Add generic atomic_or routines built on atomic_cas.
 1.2.2.1 10-Aug-2014  tls Rebase.
 1.3.26.3 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.3.26.2 21-Apr-2020  martin Sync with HEAD
 1.3.26.1 10-Jun-2019  christos Sync with HEAD
 1.3.4.2 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.3.4.1 23-Jun-2014  tls file atomic_or_16_cas.c was added on branch tls-maxphys on 2014-08-19 23:45:14 +0000
 1.1 13-Apr-2007  thorpej branches: 1.1.2;
file atomic_or_16_nv_cas.c was initially added on branch thorpej-atomic.
 1.1.2.4 22-Apr-2007  thorpej Make sure namespace-cleansed aliases are avaialble for all atomic ops.
 1.1.2.3 17-Apr-2007  thorpej Work around pesky macro argument expansion rules.
 1.1.2.2 16-Apr-2007  thorpej Namespace protection for atomic operations.
 1.1.2.1 13-Apr-2007  thorpej Add generic atomic_or routines built on atomic_cas.
 1.10 23-Jun-2014  joerg branches: 1.10.24;
Add aliases for the builtins used to implement C11/C++11 atomics.
 1.9 22-Feb-2014  martin branches: 1.9.2;
Try to hide the C runtime implementation specific __sync_* ops from librump,
to avoid duplicates.
 1.8 29-Jan-2014  joerg Fix up the remaining __sync_* functions to build with Clang on ARM.
 1.7 27-Jan-2014  matt fix tpyos.
 1.6 27-Jan-2014  matt Rework so that __sync_fetch_and_<OP>_<N> actually returns a value.
 1.5 27-Jan-2014  matt Add __sync_fetch_and_<OP>_<N> aliases
 1.4 28-Apr-2008  martin branches: 1.4.4; 1.4.26;
Remove clause 3 and 4 from TNF licenses
 1.3 28-Nov-2007  ad branches: 1.3.4; 1.3.8;
Memory barrier in respect of the CAS isn't necessary.
 1.2 28-Nov-2007  ad Support files from the thorpej-atomic branch.
 1.1 13-Apr-2007  thorpej branches: 1.1.2;
file atomic_or_32_cas.c was initially added on branch thorpej-atomic.
 1.1.2.4 22-Apr-2007  thorpej Make sure namespace-cleansed aliases are avaialble for all atomic ops.
 1.1.2.3 17-Apr-2007  thorpej Work around pesky macro argument expansion rules.
 1.1.2.2 16-Apr-2007  thorpej Namespace protection for atomic operations.
 1.1.2.1 13-Apr-2007  thorpej Add generic atomic_or routines built on atomic_cas.
 1.3.8.1 18-May-2008  yamt sync with head.
 1.3.4.2 09-Jan-2008  matt sync with HEAD
 1.3.4.1 28-Nov-2007  matt file atomic_or_32_cas.c was added on branch matt-armv6 on 2008-01-09 01:21:30 +0000
 1.4.26.1 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.4.4.1 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.9.2.1 10-Aug-2014  tls Rebase.
 1.10.24.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.10.24.1 21-Apr-2020  martin Sync with HEAD
 1.6 22-Feb-2014  martin branches: 1.6.26;
Try to hide the C runtime implementation specific __sync_* ops from librump,
to avoid duplicates.
 1.5 27-Jan-2014  matt Add __sync_<OP>_and_fetch_<N> aliases
 1.4 28-Apr-2008  martin branches: 1.4.4; 1.4.26;
Remove clause 3 and 4 from TNF licenses
 1.3 28-Nov-2007  ad branches: 1.3.4; 1.3.8;
Memory barrier in respect of the CAS isn't necessary.
 1.2 28-Nov-2007  ad Support files from the thorpej-atomic branch.
 1.1 13-Apr-2007  thorpej branches: 1.1.2;
file atomic_or_32_nv_cas.c was initially added on branch thorpej-atomic.
 1.1.2.4 22-Apr-2007  thorpej Make sure namespace-cleansed aliases are avaialble for all atomic ops.
 1.1.2.3 17-Apr-2007  thorpej Work around pesky macro argument expansion rules.
 1.1.2.2 16-Apr-2007  thorpej Namespace protection for atomic operations.
 1.1.2.1 13-Apr-2007  thorpej Add generic atomic_or routines built on atomic_cas.
 1.3.8.1 18-May-2008  yamt sync with head.
 1.3.4.2 09-Jan-2008  matt sync with HEAD
 1.3.4.1 28-Nov-2007  matt file atomic_or_32_nv_cas.c was added on branch matt-armv6 on 2008-01-09 01:21:31 +0000
 1.4.26.1 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.4.4.1 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.6.26.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.6.26.1 21-Apr-2020  martin Sync with HEAD
 1.10 23-Jun-2014  joerg branches: 1.10.24;
Add aliases for the builtins used to implement C11/C++11 atomics.
 1.9 28-Jan-2014  joerg branches: 1.9.2;
Allow building with clang.
XXX __RENAME is currently not allowed for the kernel, so use plain asm.
Clang rejects definitions of builtins, so it doesn't work without.
 1.8 27-Jan-2014  matt fix c&p bug.
 1.7 27-Jan-2014  matt Rework so that __sync_fetch_and_<OP>_<N> actually returns a value.
 1.6 27-Jan-2014  matt Add __sync_fetch_and_<OP>_<N> aliases
 1.5 28-Apr-2008  martin branches: 1.5.4; 1.5.26;
Remove clause 3 and 4 from TNF licenses
 1.4 29-Nov-2007  ad branches: 1.4.4; 1.4.8;
Only build the 64-bit versions if __HAVE_ATOMIC64_OPS.
 1.3 28-Nov-2007  ad Memory barrier in respect of the CAS isn't necessary.
 1.2 28-Nov-2007  ad Support files from the thorpej-atomic branch.
 1.1 13-Apr-2007  thorpej branches: 1.1.2;
file atomic_or_64_cas.c was initially added on branch thorpej-atomic.
 1.1.2.4 22-Apr-2007  thorpej Make sure namespace-cleansed aliases are avaialble for all atomic ops.
 1.1.2.3 17-Apr-2007  thorpej Work around pesky macro argument expansion rules.
 1.1.2.2 16-Apr-2007  thorpej Namespace protection for atomic operations.
 1.1.2.1 13-Apr-2007  thorpej Add generic atomic_or routines built on atomic_cas.
 1.4.8.1 18-May-2008  yamt sync with head.
 1.4.4.2 09-Jan-2008  matt sync with HEAD
 1.4.4.1 29-Nov-2007  matt file atomic_or_64_cas.c was added on branch matt-armv6 on 2008-01-09 01:21:31 +0000
 1.5.26.1 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.5.4.1 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.9.2.1 10-Aug-2014  tls Rebase.
 1.10.24.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.10.24.1 21-Apr-2020  martin Sync with HEAD
 1.7 22-Feb-2014  martin branches: 1.7.26;
Try to hide the C runtime implementation specific __sync_* ops from librump,
to avoid duplicates.
 1.6 27-Jan-2014  matt Add __sync_<OP>_and_fetch_<N> aliases
 1.5 28-Apr-2008  martin branches: 1.5.4; 1.5.26;
Remove clause 3 and 4 from TNF licenses
 1.4 29-Nov-2007  ad branches: 1.4.4; 1.4.8;
Only build the 64-bit versions if __HAVE_ATOMIC64_OPS.
 1.3 28-Nov-2007  ad Memory barrier in respect of the CAS isn't necessary.
 1.2 28-Nov-2007  ad Support files from the thorpej-atomic branch.
 1.1 13-Apr-2007  thorpej branches: 1.1.2;
file atomic_or_64_nv_cas.c was initially added on branch thorpej-atomic.
 1.1.2.4 22-Apr-2007  thorpej Make sure namespace-cleansed aliases are avaialble for all atomic ops.
 1.1.2.3 17-Apr-2007  thorpej Work around pesky macro argument expansion rules.
 1.1.2.2 16-Apr-2007  thorpej Namespace protection for atomic operations.
 1.1.2.1 13-Apr-2007  thorpej Add generic atomic_or routines built on atomic_cas.
 1.4.8.1 18-May-2008  yamt sync with head.
 1.4.4.2 09-Jan-2008  matt sync with HEAD
 1.4.4.1 29-Nov-2007  matt file atomic_or_64_nv_cas.c was added on branch matt-armv6 on 2008-01-09 01:21:32 +0000
 1.5.26.1 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.5.4.1 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.7.26.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.7.26.1 21-Apr-2020  martin Sync with HEAD
 1.4 01-Mar-2019  isaki Fix return value. or_and_fetch should return new value.
 1.3 23-Jun-2014  joerg branches: 1.3.4; 1.3.26;
Add aliases for the builtins used to implement C11/C++11 atomics.
 1.2 21-Feb-2014  martin branches: 1.2.2;
Provide 8 and 16 bit sync ops (using 16 bit and 8 bit cas)
 1.1 13-Apr-2007  thorpej branches: 1.1.2; 1.1.24;
file atomic_or_8_cas.c was initially added on branch thorpej-atomic.
 1.1.24.1 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.1.2.4 22-Apr-2007  thorpej Make sure namespace-cleansed aliases are avaialble for all atomic ops.
 1.1.2.3 17-Apr-2007  thorpej Work around pesky macro argument expansion rules.
 1.1.2.2 16-Apr-2007  thorpej Namespace protection for atomic operations.
 1.1.2.1 13-Apr-2007  thorpej Add generic atomic_or routines built on atomic_cas.
 1.2.2.1 10-Aug-2014  tls Rebase.
 1.3.26.3 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.3.26.2 21-Apr-2020  martin Sync with HEAD
 1.3.26.1 10-Jun-2019  christos Sync with HEAD
 1.3.4.2 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.3.4.1 23-Jun-2014  tls file atomic_or_8_cas.c was added on branch tls-maxphys on 2014-08-19 23:45:14 +0000
 1.1 13-Apr-2007  thorpej branches: 1.1.2;
file atomic_or_8_nv_cas.c was initially added on branch thorpej-atomic.
 1.1.2.4 22-Apr-2007  thorpej Make sure namespace-cleansed aliases are avaialble for all atomic ops.
 1.1.2.3 17-Apr-2007  thorpej Work around pesky macro argument expansion rules.
 1.1.2.2 16-Apr-2007  thorpej Namespace protection for atomic operations.
 1.1.2.1 13-Apr-2007  thorpej Add generic atomic_or routines built on atomic_cas.
 1.5 04-Feb-2024  christos mark unused argument
 1.4 09-Apr-2022  riastradh libc/atomic: Fix membars in __atomic_load/store_* stubs.

- membar_enter/exit ordering was backwards.
- membar_enter doesn't make any sense for load anyway.
- Switch to membar_release for store and membar_acquire for load.

The only sensible orderings for a simple load or store are acquire or
release, respectively, or sequential consistency. This never
provided correct sequential consistency before -- we should really
make it conditional on memmodel but I don't know offhand what the
values of memmodel might be and this is at least better than before.
 1.3 07-Sep-2020  mrg make some prototypes match the builtin properly. GCC 9 complains
with the old version, GCC 8 is happy with this version.

tested on sparc.
 1.2 06-Jul-2014  joerg branches: 1.2.2; 1.2.6; 1.2.28;
Add unused memory model argument to make GCC happy.
 1.1 05-Jul-2014  joerg Provide a basic implementation of __atomic_load_* and __atomic_store_*,
used by GCC and LLVM as backing for C11/C++11 atomics, if the hardware
is not known to have corresponding features. Include it on ARM as LLVM
and libc++ hit it when compiled for ARMv4.
 1.2.28.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.2.28.1 21-Apr-2020  martin Sync with HEAD
 1.2.6.2 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.2.6.1 06-Jul-2014  tls file atomic_store.c was added on branch tls-maxphys on 2014-08-19 23:45:14 +0000
 1.2.2.2 10-Aug-2014  tls Rebase.
 1.2.2.1 06-Jul-2014  tls file atomic_store.c was added on branch tls-earlyentropy on 2014-08-10 06:47:06 +0000
 1.3 23-Jun-2014  joerg branches: 1.3.4; 1.3.26;
Add aliases for the builtins used to implement C11/C++11 atomics.
 1.2 21-Feb-2014  martin branches: 1.2.2; 1.2.4;
Add a few missing namespace includes
 1.1 21-Feb-2014  martin Provide 8 and 16 bit sync ops (using 16 bit and 8 bit cas)
 1.2.4.2 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.2.4.1 21-Feb-2014  yamt file atomic_sub_16_cas.c was added on branch yamt-pagecache on 2014-05-22 11:26:30 +0000
 1.2.2.1 10-Aug-2014  tls Rebase.
 1.3.26.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.3.26.1 21-Apr-2020  martin Sync with HEAD
 1.3.4.2 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.3.4.1 23-Jun-2014  tls file atomic_sub_16_cas.c was added on branch tls-maxphys on 2014-08-19 23:45:14 +0000
 1.2 23-Jun-2014  joerg branches: 1.2.4; 1.2.26;
Add aliases for the builtins used to implement C11/C++11 atomics.
 1.1 21-Feb-2014  martin branches: 1.1.2; 1.1.4;
Provide 32bit __sync ops based on atomic_cas_32.
 1.1.4.2 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.1.4.1 21-Feb-2014  yamt file atomic_sub_32_cas.c was added on branch yamt-pagecache on 2014-05-22 11:26:30 +0000
 1.1.2.1 10-Aug-2014  tls Rebase.
 1.2.26.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.2.26.1 21-Apr-2020  martin Sync with HEAD
 1.2.4.2 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.2.4.1 23-Jun-2014  tls file atomic_sub_32_cas.c was added on branch tls-maxphys on 2014-08-19 23:45:14 +0000
 1.3 23-Jun-2014  joerg branches: 1.3.4; 1.3.26;
Add aliases for the builtins used to implement C11/C++11 atomics.
 1.2 18-Feb-2014  martin branches: 1.2.2; 1.2.4;
Make the _and_fetch_8 primitives return the new value they calculated
and set - not whatever is found in memory later.
 1.1 18-Feb-2014  martin Provide most missing __sync_*64 primitives for i386
 1.2.4.2 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.2.4.1 18-Feb-2014  yamt file atomic_sub_64_cas.c was added on branch yamt-pagecache on 2014-05-22 11:26:30 +0000
 1.2.2.1 10-Aug-2014  tls Rebase.
 1.3.26.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.3.26.1 21-Apr-2020  martin Sync with HEAD
 1.3.4.2 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.3.4.1 23-Jun-2014  tls file atomic_sub_64_cas.c was added on branch tls-maxphys on 2014-08-19 23:45:14 +0000
 1.3 23-Jun-2014  joerg branches: 1.3.4; 1.3.26;
Add aliases for the builtins used to implement C11/C++11 atomics.
 1.2 21-Feb-2014  martin branches: 1.2.2; 1.2.4;
Add a few missing namespace includes
 1.1 21-Feb-2014  martin Provide 8 and 16 bit sync ops (using 16 bit and 8 bit cas)
 1.2.4.2 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.2.4.1 21-Feb-2014  yamt file atomic_sub_8_cas.c was added on branch yamt-pagecache on 2014-05-22 11:26:30 +0000
 1.2.2.1 10-Aug-2014  tls Rebase.
 1.3.26.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.3.26.1 21-Apr-2020  martin Sync with HEAD
 1.3.4.2 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.3.4.1 23-Jun-2014  tls file atomic_sub_8_cas.c was added on branch tls-maxphys on 2014-08-19 23:45:14 +0000
 1.4 23-Jun-2008  ad Install manual pages for the atomic ops.
 1.3 30-Apr-2008  martin branches: 1.3.2;
Convert TNF licenses to new 2 clause variant
 1.2 28-Nov-2007  ad branches: 1.2.4; 1.2.8;
Support files from the thorpej-atomic branch.
 1.1 12-Apr-2007  thorpej branches: 1.1.2;
file atomic_swap.3 was initially added on branch thorpej-atomic.
 1.1.2.1 12-Apr-2007  thorpej Man pages for the atomic ops API.
 1.2.8.1 18-May-2008  yamt sync with head.
 1.2.4.2 09-Jan-2008  matt sync with HEAD
 1.2.4.1 28-Nov-2007  matt file atomic_swap.3 was added on branch matt-armv6 on 2008-01-09 01:21:32 +0000
 1.3.2.1 07-Sep-2008  wrstuden Delete a bunch of things that are now gone in head.
 1.3 28-Jun-2014  joerg branches: 1.3.4; 1.3.26;
Add aliases for the C11/C++11 spelling of the CAS primitives.
 1.2 21-Feb-2014  martin branches: 1.2.2;
Provide 8 and 16 bit sync ops (using 16 bit and 8 bit cas)
 1.1 13-Apr-2007  thorpej branches: 1.1.2; 1.1.24;
file atomic_swap_16_cas.c was initially added on branch thorpej-atomic.
 1.1.24.1 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.1.2.4 22-Apr-2007  thorpej Make sure namespace-cleansed aliases are avaialble for all atomic ops.
 1.1.2.3 17-Apr-2007  thorpej Work around pesky macro argument expansion rules.
 1.1.2.2 16-Apr-2007  thorpej Namespace protection for atomic operations.
 1.1.2.1 13-Apr-2007  thorpej Add generic atomic_swap routines built on atomic_cas.
 1.2.2.1 10-Aug-2014  tls Rebase.
 1.3.26.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.3.26.1 21-Apr-2020  martin Sync with HEAD
 1.3.4.2 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.3.4.1 28-Jun-2014  tls file atomic_swap_16_cas.c was added on branch tls-maxphys on 2014-08-19 23:45:14 +0000
 1.7 28-Jun-2014  joerg branches: 1.7.24;
Add aliases for the C11/C++11 spelling of the CAS primitives.
 1.6 22-Feb-2014  martin branches: 1.6.2;
Try to hide the C runtime implementation specific __sync_* ops from librump,
to avoid duplicates.
 1.5 21-Feb-2014  martin Add missing __sync_lock_test_and_set_* aliases for PR lib/48601
 1.4 28-Apr-2008  martin branches: 1.4.4; 1.4.26;
Remove clause 3 and 4 from TNF licenses
 1.3 28-Nov-2007  ad branches: 1.3.4; 1.3.8;
Memory barrier in respect of the CAS isn't necessary.
 1.2 28-Nov-2007  ad Support files from the thorpej-atomic branch.
 1.1 13-Apr-2007  thorpej branches: 1.1.2;
file atomic_swap_32_cas.c was initially added on branch thorpej-atomic.
 1.1.2.4 22-Apr-2007  thorpej Make sure namespace-cleansed aliases are avaialble for all atomic ops.
 1.1.2.3 17-Apr-2007  thorpej Work around pesky macro argument expansion rules.
 1.1.2.2 16-Apr-2007  thorpej Namespace protection for atomic operations.
 1.1.2.1 13-Apr-2007  thorpej Add generic atomic_swap routines built on atomic_cas.
 1.3.8.1 18-May-2008  yamt sync with head.
 1.3.4.2 09-Jan-2008  matt sync with HEAD
 1.3.4.1 28-Nov-2007  matt file atomic_swap_32_cas.c was added on branch matt-armv6 on 2008-01-09 01:21:32 +0000
 1.4.26.1 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.4.4.1 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.6.2.1 10-Aug-2014  tls Rebase.
 1.7.24.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.7.24.1 21-Apr-2020  martin Sync with HEAD
 1.10 29-May-2022  rin One more typo; this is called _atomic_swap_64, not _8.
 1.9 28-Jun-2014  joerg branches: 1.9.24;
Add aliases for the C11/C++11 spelling of the CAS primitives.
 1.8 22-Feb-2014  martin branches: 1.8.2;
Try to hide the C runtime implementation specific __sync_* ops from librump,
to avoid duplicates.
 1.7 21-Feb-2014  martin Fix previous: move the alias declaration outside of a #ifdef _LP64
 1.6 21-Feb-2014  martin Add missing __sync_lock_test_and_set_* aliases for PR lib/48601
 1.5 28-Apr-2008  martin branches: 1.5.4; 1.5.26;
Remove clause 3 and 4 from TNF licenses
 1.4 29-Nov-2007  ad branches: 1.4.4; 1.4.8;
Only build the 64-bit versions if __HAVE_ATOMIC64_OPS.
 1.3 28-Nov-2007  ad Memory barrier in respect of the CAS isn't necessary.
 1.2 28-Nov-2007  ad Support files from the thorpej-atomic branch.
 1.1 13-Apr-2007  thorpej branches: 1.1.2;
file atomic_swap_64_cas.c was initially added on branch thorpej-atomic.
 1.1.2.4 22-Apr-2007  thorpej Make sure namespace-cleansed aliases are avaialble for all atomic ops.
 1.1.2.3 17-Apr-2007  thorpej Work around pesky macro argument expansion rules.
 1.1.2.2 16-Apr-2007  thorpej Namespace protection for atomic operations.
 1.1.2.1 13-Apr-2007  thorpej Add generic atomic_swap routines built on atomic_cas.
 1.4.8.1 18-May-2008  yamt sync with head.
 1.4.4.2 09-Jan-2008  matt sync with HEAD
 1.4.4.1 29-Nov-2007  matt file atomic_swap_64_cas.c was added on branch matt-armv6 on 2008-01-09 01:21:33 +0000
 1.5.26.1 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.5.4.1 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.8.2.1 10-Aug-2014  tls Rebase.
 1.9.24.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.9.24.1 21-Apr-2020  martin Sync with HEAD
 1.3 28-Jun-2014  joerg branches: 1.3.4; 1.3.26;
Add aliases for the C11/C++11 spelling of the CAS primitives.
 1.2 21-Feb-2014  martin branches: 1.2.2;
Provide 8 and 16 bit sync ops (using 16 bit and 8 bit cas)
 1.1 13-Apr-2007  thorpej branches: 1.1.2; 1.1.24;
file atomic_swap_8_cas.c was initially added on branch thorpej-atomic.
 1.1.24.1 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.1.2.4 22-Apr-2007  thorpej Make sure namespace-cleansed aliases are avaialble for all atomic ops.
 1.1.2.3 17-Apr-2007  thorpej Work around pesky macro argument expansion rules.
 1.1.2.2 16-Apr-2007  thorpej Namespace protection for atomic operations.
 1.1.2.1 13-Apr-2007  thorpej Add generic atomic_swap routines built on atomic_cas.
 1.2.2.1 10-Aug-2014  tls Rebase.
 1.3.26.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.3.26.1 21-Apr-2020  martin Sync with HEAD
 1.3.4.2 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.3.4.1 28-Jun-2014  tls file atomic_swap_8_cas.c was added on branch tls-maxphys on 2014-08-19 23:45:14 +0000
 1.3 23-Jun-2014  joerg branches: 1.3.4; 1.3.26;
Add aliases for the builtins used to implement C11/C++11 atomics.
 1.2 21-Feb-2014  martin branches: 1.2.2; 1.2.4;
Add a few missing namespace includes
 1.1 21-Feb-2014  martin Provide 8 and 16 bit sync ops (using 16 bit and 8 bit cas)
 1.2.4.2 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.2.4.1 21-Feb-2014  yamt file atomic_xor_16_cas.c was added on branch yamt-pagecache on 2014-05-22 11:26:30 +0000
 1.2.2.1 10-Aug-2014  tls Rebase.
 1.3.26.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.3.26.1 21-Apr-2020  martin Sync with HEAD
 1.3.4.2 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.3.4.1 23-Jun-2014  tls file atomic_xor_16_cas.c was added on branch tls-maxphys on 2014-08-19 23:45:14 +0000
 1.2 23-Jun-2014  joerg branches: 1.2.4; 1.2.26;
Add aliases for the builtins used to implement C11/C++11 atomics.
 1.1 21-Feb-2014  martin branches: 1.1.2; 1.1.4;
Provide 32bit __sync ops based on atomic_cas_32.
 1.1.4.2 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.1.4.1 21-Feb-2014  yamt file atomic_xor_32_cas.c was added on branch yamt-pagecache on 2014-05-22 11:26:30 +0000
 1.1.2.1 10-Aug-2014  tls Rebase.
 1.2.26.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.2.26.1 21-Apr-2020  martin Sync with HEAD
 1.2.4.2 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.2.4.1 23-Jun-2014  tls file atomic_xor_32_cas.c was added on branch tls-maxphys on 2014-08-19 23:45:14 +0000
 1.3 23-Jun-2014  joerg branches: 1.3.4; 1.3.26;
Add aliases for the builtins used to implement C11/C++11 atomics.
 1.2 18-Feb-2014  martin branches: 1.2.2; 1.2.4;
Make the _and_fetch_8 primitives return the new value they calculated
and set - not whatever is found in memory later.
 1.1 18-Feb-2014  martin Provide most missing __sync_*64 primitives for i386
 1.2.4.2 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.2.4.1 18-Feb-2014  yamt file atomic_xor_64_cas.c was added on branch yamt-pagecache on 2014-05-22 11:26:30 +0000
 1.2.2.1 10-Aug-2014  tls Rebase.
 1.3.26.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.3.26.1 21-Apr-2020  martin Sync with HEAD
 1.3.4.2 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.3.4.1 23-Jun-2014  tls file atomic_xor_64_cas.c was added on branch tls-maxphys on 2014-08-19 23:45:14 +0000
 1.3 23-Jun-2014  joerg branches: 1.3.4; 1.3.26;
Add aliases for the builtins used to implement C11/C++11 atomics.
 1.2 21-Feb-2014  martin branches: 1.2.2; 1.2.4;
Add a few missing namespace includes
 1.1 21-Feb-2014  martin Provide 8 and 16 bit sync ops (using 16 bit and 8 bit cas)
 1.2.4.2 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.2.4.1 21-Feb-2014  yamt file atomic_xor_8_cas.c was added on branch yamt-pagecache on 2014-05-22 11:26:30 +0000
 1.2.2.1 10-Aug-2014  tls Rebase.
 1.3.26.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.3.26.1 21-Apr-2020  martin Sync with HEAD
 1.3.4.2 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.3.4.1 23-Jun-2014  tls file atomic_xor_8_cas.c was added on branch tls-maxphys on 2014-08-19 23:45:14 +0000
 1.6 23-Jun-2008  ad Install manual pages for the atomic ops.
 1.5 30-Apr-2008  martin branches: 1.5.2;
Convert TNF licenses to new 2 clause variant
 1.4 11-Feb-2008  ad branches: 1.4.4;
Clarify what membar_enter/membar_exit provide.
 1.3 02-Dec-2007  wiz branches: 1.3.4;
Typo fixes.
 1.2 28-Nov-2007  ad Support files from the thorpej-atomic branch.
 1.1 12-Apr-2007  thorpej branches: 1.1.2;
file membar_ops.3 was initially added on branch thorpej-atomic.
 1.1.2.1 12-Apr-2007  thorpej Man pages for the atomic ops API.
 1.3.4.3 23-Mar-2008  matt sync with HEAD
 1.3.4.2 09-Jan-2008  matt sync with HEAD
 1.3.4.1 02-Dec-2007  matt file membar_ops.3 was added on branch matt-armv6 on 2008-01-09 01:21:34 +0000
 1.4.4.1 18-May-2008  yamt sync with head.
 1.5.2.1 07-Sep-2008  wrstuden Delete a bunch of things that are now gone in head.
 1.8 09-Apr-2022  riastradh Introduce membar_acquire/release. Deprecate membar_enter/exit.

The names membar_enter/exit were unclear, and the documentation of
membar_enter has disagreed with the implementations on sparc,
powerpc, and even x86(!) for the entire time it has been in NetBSD.

The terms `acquire' and `release' are ubiquitous in the literature
today, and have been adopted in the C and C++ standards to mean
load-before-load/store and load/store-before-store, respectively,
which are exactly the orderings required by acquiring and releasing a
mutex, as well as other useful applications like decrementing a
reference count and then freeing the underlying object if it went to
zero.

Originally I proposed changing one word in the documentation for
membar_enter to make it load-before-load/store instead of
store-before-load/store, i.e., to make it an acquire barrier. I
proposed this on the grounds that

(a) all implementations guarantee load-before-load/store,
(b) some implementations fail to guarantee store-before-load/store,
and
(c) all uses in-tree assume load-before-load/store.

I verified parts (a) and (b) (except, for (a), powerpc didn't even
guarantee load-before-load/store -- isync isn't necessarily enough;
need lwsync in general -- but it _almost_ did, and it certainly didn't
guarantee store-before-load/store).

Part (c) might not be correct, however: under the mistaken assumption
that atomic-r/m/w then membar-w/rw is equivalent to atomic-r/m/w then
membar-r/rw, I only audited the cases of membar_enter that _aren't_
immediately after an atomic-r/m/w. All of those cases assume
load-before-load/store. But my assumption was wrong -- there are
cases of atomic-r/m/w then membar-w/rw that would be broken by
changing to atomic-r/m/w then membar-r/rw:

https://mail-index.netbsd.org/tech-kern/2022/03/29/msg028044.html

Furthermore, the name membar_enter has been adopted in other places
like OpenBSD where it actually does follow the documentation and
guarantee store-before-load/store, even if that order is not useful.
So the name membar_enter currently lives in a bad place where it
means either of two things -- r/rw or w/rw.

With this change, we deprecate membar_enter/exit, introduce
membar_acquire/release as better names for the useful pair (r/rw and
rw/w), and make sure the implementation of membar_enter guarantees
both what was documented _and_ what was implemented, making it an
alias for membar_sync.

While here, rework all of the membar_* definitions and aliases. The
new logic follows a rule to make it easier to audit:

membar_X is defined as an alias for membar_Y iff membar_X is
guaranteed by membar_Y.

The `no stronger than' relation is (the transitive closure of):

- membar_consumer (r/r) is guaranteed by membar_acquire (r/rw)
- membar_producer (w/w) is guaranteed by membar_release (rw/w)
- membar_acquire (r/rw) is guaranteed by membar_sync (rw/rw)
- membar_release (rw/w) is guaranteed by membar_sync (rw/rw)

And, for the deprecated membars:

- membar_enter (whether r/rw, w/rw, or rw/rw) is guaranteed by
membar_sync (rw/rw)
- membar_exit (rw/w) is guaranteed by membar_release (rw/w)

(membar_exit is identical to membar_release, but the name is
deprecated.)

Finally, while here, annotate some of the instructions with their
semantics. For powerpc, leave an essay with citations on the
unfortunate but -- as far as I can tell -- necessary decision to use
lwsync, not isync, for membar_acquire and membar_consumer.

Also add membar(3) and atomic(3) man page links.
 1.7 22-Feb-2014  martin branches: 1.7.26;
Try to hide the C runtime implementation specific __sync_* ops from librump,
to avoid duplicates.
 1.6 07-Dec-2013  skrll Add __sync_synchronize alias.
 1.5 21-Aug-2009  skrll branches: 1.5.6; 1.5.12;
__strong_alias(foo,foo) is... pointless.
 1.4 28-Apr-2008  martin branches: 1.4.12;
Remove clause 3 and 4 from TNF licenses
 1.3 29-Nov-2007  ad branches: 1.3.4; 1.3.8;
membar_read/write are gone.
 1.2 28-Nov-2007  ad Support files from the thorpej-atomic branch.
 1.1 18-Apr-2007  thorpej branches: 1.1.2;
file membar_ops_nop.c was initially added on branch thorpej-atomic.
 1.1.2.2 22-Apr-2007  thorpej Make sure namespace-cleansed aliases are avaialble for all atomic ops.
 1.1.2.1 18-Apr-2007  thorpej Generic membar_*() routines for platforms that don't require memory
barriers.
 1.3.8.1 18-May-2008  yamt sync with head.
 1.3.4.2 09-Jan-2008  matt sync with HEAD
 1.3.4.1 29-Nov-2007  matt file membar_ops_nop.c was added on branch matt-armv6 on 2008-01-09 01:21:34 +0000
 1.4.12.1 26-Aug-2009  matt Don't make a strong alias to the same name.
 1.5.12.1 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.5.6.1 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.7.26.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.7.26.1 21-Apr-2020  martin Sync with HEAD
 1.2 10-Jan-2017  christos branches: 1.2.14;
mark parameter as unused.
 1.1 11-Dec-2013  joerg branches: 1.1.4; 1.1.8; 1.1.12;
Allow kernel code to access constant databases by moving cdbr(3) and the
required mi_vector_hash(3) into src/common.
 1.1.12.1 20-Mar-2017  pgoyette Sync with HEAD
 1.1.8.2 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.1.8.1 11-Dec-2013  tls file cdbr.c was added on branch tls-maxphys on 2014-08-19 23:45:14 +0000
 1.1.4.2 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.1.4.1 11-Dec-2013  yamt file cdbr.c was added on branch yamt-pagecache on 2014-05-22 11:26:30 +0000
 1.2.14.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.2.14.1 21-Apr-2020  martin Sync with HEAD
 1.4 17-Mar-2012  martin branches: 1.4.34;
Mark linted
 1.3 04-Jul-2011  joerg branches: 1.3.2;
ANSIfy
 1.2 16-Feb-2008  apb For all files under src/common, as well as related man pages under
src/lib/libc, or related headers under src/sys/sys: change u_intNN_t to
uintNN_t.
 1.1 20-Dec-2005  christos branches: 1.1.6; 1.1.18;
Merge libkern + libc common files. As requested by core.
 1.1.18.1 23-Mar-2008  matt sync with HEAD
 1.1.6.1 18-Nov-2008  snj Pull up following revision(s) (requested by tsutsui in ticket #1224):
common/lib/libc/gen/bswap64.c: revision 1.2
common/lib/libprop/prop_data.c: revision 1.10
common/lib/libc/inet/inet_addr.c: revision 1.2
common/lib/libc/hash/rmd160/rmd160.c: revision 1.4
common/lib/libc/gen/bswap32.c: revision 1.2
common/lib/libc/gen/bswap16.c: revision 1.2
common/lib/libc/arch/alpha/gen/byte_swap_4.S: revision 1.3
common/lib/libc/arch/sh3/string/memset.S: revision 1.2
common/lib/libc/arch/alpha/gen/byte_swap_2.S: revision 1.3
common/lib/libc/md/md5c.c: revision 1.3
common/lib/libc/hash/sha1/sha1.c: revision 1.3
common/lib/libc/md/md4c.c: revision 1.3
sys/sys/rmd160.h: revision 1.2
sys/sys/sha2.h: revision 1.2
For all files under src/common, as well as related man pages under
src/lib/libc, or related headers under src/sys/sys: change u_intNN_t to
uintNN_t.
 1.3.2.1 17-Apr-2012  yamt sync with head
 1.4.34.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.4.34.1 21-Apr-2020  martin Sync with HEAD
 1.3 04-Jul-2011  joerg branches: 1.3.44;
ANSIfy
 1.2 16-Feb-2008  apb For all files under src/common, as well as related man pages under
src/lib/libc, or related headers under src/sys/sys: change u_intNN_t to
uintNN_t.
 1.1 20-Dec-2005  christos branches: 1.1.6; 1.1.18;
Merge libkern + libc common files. As requested by core.
 1.1.18.1 23-Mar-2008  matt sync with HEAD
 1.1.6.1 18-Nov-2008  snj Pull up following revision(s) (requested by tsutsui in ticket #1224):
common/lib/libc/gen/bswap64.c: revision 1.2
common/lib/libprop/prop_data.c: revision 1.10
common/lib/libc/inet/inet_addr.c: revision 1.2
common/lib/libc/hash/rmd160/rmd160.c: revision 1.4
common/lib/libc/gen/bswap32.c: revision 1.2
common/lib/libc/gen/bswap16.c: revision 1.2
common/lib/libc/arch/alpha/gen/byte_swap_4.S: revision 1.3
common/lib/libc/arch/sh3/string/memset.S: revision 1.2
common/lib/libc/arch/alpha/gen/byte_swap_2.S: revision 1.3
common/lib/libc/md/md5c.c: revision 1.3
common/lib/libc/hash/sha1/sha1.c: revision 1.3
common/lib/libc/md/md4c.c: revision 1.3
sys/sys/rmd160.h: revision 1.2
sys/sys/sha2.h: revision 1.2
For all files under src/common, as well as related man pages under
src/lib/libc, or related headers under src/sys/sys: change u_intNN_t to
uintNN_t.
 1.3.44.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.3.44.1 21-Apr-2020  martin Sync with HEAD
 1.3 16-Mar-2009  cegger branches: 1.3.48;
ansify function definitions
 1.2 16-Feb-2008  apb branches: 1.2.12;
For all files under src/common, as well as related man pages under
src/lib/libc, or related headers under src/sys/sys: change u_intNN_t to
uintNN_t.
 1.1 20-Dec-2005  christos branches: 1.1.6; 1.1.18;
Merge libkern + libc common files. As requested by core.
 1.1.18.1 23-Mar-2008  matt sync with HEAD
 1.1.6.1 18-Nov-2008  snj Pull up following revision(s) (requested by tsutsui in ticket #1224):
common/lib/libc/gen/bswap64.c: revision 1.2
common/lib/libprop/prop_data.c: revision 1.10
common/lib/libc/inet/inet_addr.c: revision 1.2
common/lib/libc/hash/rmd160/rmd160.c: revision 1.4
common/lib/libc/gen/bswap32.c: revision 1.2
common/lib/libc/gen/bswap16.c: revision 1.2
common/lib/libc/arch/alpha/gen/byte_swap_4.S: revision 1.3
common/lib/libc/arch/sh3/string/memset.S: revision 1.2
common/lib/libc/arch/alpha/gen/byte_swap_2.S: revision 1.3
common/lib/libc/md/md5c.c: revision 1.3
common/lib/libc/hash/sha1/sha1.c: revision 1.3
common/lib/libc/md/md4c.c: revision 1.3
sys/sys/rmd160.h: revision 1.2
sys/sys/sha2.h: revision 1.2
For all files under src/common, as well as related man pages under
src/lib/libc, or related headers under src/sys/sys: change u_intNN_t to
uintNN_t.
 1.2.12.1 13-May-2009  jym Sync with HEAD.

Commit is split, to avoid a "too many arguments" protocol error.
 1.3.48.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.3.48.1 21-Apr-2020  martin Sync with HEAD
 1.15 25-Apr-2025  andvar s/abadoned/abandoned/ and s/fobidden/forbidden/ in comments.
 1.14 25-Apr-2025  andvar s/identiy/identity/ and s/chang /change/ in comments.
 1.13 20-Jan-2024  christos branches: 1.13.2;
move local decls to headers
 1.12 26-May-2022  rillig libc/ptree: remove CONSTCOND comments

Since 2021-01-31, lint does not need them anymore.
 1.11 24-May-2022  andvar fix various typos in comment, documentation and log messages.
 1.10 06-Oct-2012  matt branches: 1.10.32;
Add a ptree_mask_node_p to determine if an item is a mask node, and what
its non-mask length is.
 1.9 15-Jul-2012  rmind branches: 1.9.2;
ptree_find_filtered_node: make key argument const.
 1.8 14-Jul-2012  matt Add code to protect the ptree from multiple insertions of the same node.
 1.7 13-Jul-2012  matt Don't bother testing 0 length keys since they can only have one possible value.
 1.6 11-Jul-2012  rmind ptree: add context argument. OK matt@
 1.5 07-Jun-2009  yamt branches: 1.5.6; 1.5.8;
fix comment typos.
 1.4 18-Jan-2009  lukem branches: 1.4.2;
fix -Wsign-compare issues
 1.3 21-Nov-2008  jnemeth fix build in _STANDALONE case
 1.2 21-Nov-2008  matt Make lint happy (stupid lint).
 1.1 20-Nov-2008  matt Add new ptree (Patricia / RADIX tree) implementation to NetBSD.
 1.4.2.1 23-Jul-2009  jym Sync with HEAD.
 1.5.8.3 24-Nov-2012  jdc Pull up revisions:
src/common/lib/libc/gen/ptree.c revision 1.10
src/sys/sys/ptree.h revision 1.8
(requested by rmind in ticket #717).

Add a ptree_mask_node_p to determine if an item is a mask node, and what
its non-mask length is.
 1.5.8.2 16-Jul-2012  riz Pull up following revision(s) (requested by rmind in ticket #420):
common/lib/libc/gen/ptree.c: revision 1.7
common/lib/libc/gen/ptree.c: revision 1.8
common/lib/libc/gen/ptree.c: revision 1.9
sys/sys/ptree.h: revision 1.7
Don't bother testing 0 length keys since they can only have one possible value.
Add code to protect the ptree from multiple insertions of the same node.
ptree_find_filtered_node: make key argument const.
 1.5.8.1 12-Jul-2012  riz Pull up following revision(s) (requested by rmind in ticket #413):
common/lib/libc/gen/ptree.c: revision 1.6
sys/sys/ptree.h: revision 1.6
ptree: add context argument. OK matt@
 1.5.6.1 30-Oct-2012  yamt sync with head
 1.9.2.1 20-Nov-2012  tls Resync to 2012-11-19 00:00:00 UTC
 1.10.32.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.10.32.1 21-Apr-2020  martin Sync with HEAD
 1.13.2.1 02-Aug-2025  perseant Sync with HEAD
 1.34 04-May-2024  chs radixtree: allocate memory with KM_NOSLEEP to prevent pagedaemon hangs

Revert the part of rev 1.32 (reapplying "Do away with separate pool_cache
for some kernel objects") that changed the memory allocation for radixtree
nodes from PR_NOWAIT to KM_SLEEP as part of changing from a pool to kmem.
uvm_pageinsert_tree() calls into the radixtree code while holding
the object's vmobjlock, but that same lock is taken by the pagedaemon
in the process of reclaiming pages, and if the pagedaemon happens to
choose the same object to reclaim from that uvm_pageinsert_tree()
is being called on, then these two threads will deadlock.
The previous code already handled memory allocation failures
in uvm_pageinsert_tree() so we can simply change it back to nosleep.

Fixes a hang reported by simonb@, and the fix was also tested by him.
 1.33 23-Sep-2023  ad kmem_free() -> kmem_intr_free(). Spotted by rin@.
 1.32 23-Sep-2023  ad Repply this change with a couple of bugs fixed:

- Do away with separate pool_cache for some kernel objects that have no special
requirements and use the general purpose allocator instead. On one of my
test systems this makes for a small (~1%) but repeatable reduction in system
time during builds presumably because it decreases the kernel's cache /
memory bandwidth footprint a little.
- vfs_lockf: cache a pointer to the uidinfo and put mutex in the data segment.
 1.31 12-Sep-2023  ad Back out recent change to replace pool_cache with then general allocator.
Will return to this when I have time again.
 1.30 10-Sep-2023  ad - Do away with separate pool_cache for some kernel objects that have no special
requirements and use the general purpose allocator instead. On one of my
test systems this makes for a small (~1%) but repeatable reduction in system
time during builds presumably because it decreases the kernel's cache /
memory bandwidth footprint a little.
- vfs_lockf: cache a pointer to the uidinfo and put mutex in the data segment.
 1.29 06-Mar-2023  andvar fix few typos in comments and log messages.
 1.28 24-May-2022  andvar fix various typos in comment, documentation and log messages.
 1.27 14-May-2020  msaitoh Remove extra semicolon.
 1.26 11-Apr-2020  ad Match the naming convention in the file.
 1.25 10-Apr-2020  ad PR kern/54979 (radixtree might misbehave if ENOMEM)

- radix_tree_insert_node(): if the insert failed due to ENOMEM, roll back
any updates made to the tree.

- radix_tree_grow(): either succeed or fail, never make partial adjustments
to the tree.

- radix_tree_await_memory(): allocate & free the maximum possible number of
nodes required by any insertion.
 1.24 10-Apr-2020  ad Rename radix_tree_node_clean_p() to radix_tree_node_sum() and have it return
the computed sum. Use to replace any_children_tagmask(). Simpler & faster.
 1.23 28-Jan-2020  ad gang_lookup_scan(): if a dense scan and the first sibling doesn't match,
the scan is finished.
 1.22 28-Jan-2020  ad Add a radix_tree_await_memory(), for kernel use.
 1.21 12-Jan-2020  para initialize radix_tree_node_cache with PR_LARGECACHE

this increases the cache groups from 15 to 63 items in order
to reduce traffic between pool cache layers
this is the same as for other highly frequented pool caches as the pvpool and anonpool
 1.20 05-Dec-2019  ad branches: 1.20.2;
Fix warning that appears when compiling in kernel.
 1.19 05-Dec-2019  ad Delete the counter from "struct radix_tree_node", and in the one place we
need a non-zero check, substitute with a deterministic bitwise OR of all
values in the node. The structure then becomes cache line aligned.

For each node we now need only touch 2 cache lines instead of 3, which makes
all the operations faster (measured), amortises the cost of not having a
counter, and will avoid intra-pool-page false sharing on MP.
 1.18 05-Dec-2019  ad Merge radixtree changes from yamt-pagecache.
 1.17 02-Nov-2011  yamt branches: 1.17.2; 1.17.44;
comments
 1.16 25-Oct-2011  yamt add radix_tree_empty_tagged_tree_p, a "tagged" variant of
radix_tree_empty_tree_p.
 1.15 14-Oct-2011  yamt - add functions to scan the tree in the reverse order
(i wonder if it's the longest function name in the tree)
- assertions
- comments
- fix and update unittest
 1.14 14-Oct-2011  yamt unwarp a short line
 1.13 14-Oct-2011  yamt constify
 1.12 14-Oct-2011  yamt fix "get_tag" result of unittest
 1.11 14-Oct-2011  yamt make the output of unittest a little machine-readable
 1.10 14-Oct-2011  yamt int -> unsigned int where appropriate
 1.9 14-Oct-2011  yamt add a function to check if a tree is empty.
 1.8 14-Oct-2011  yamt include string.h for memset
 1.7 19-May-2011  yamt radix_tree_clear_tag:
- fix a bug which errornously clears tags on intermediate nodes.
- add comments.
 1.6 19-May-2011  yamt radixtree: assertions
 1.5 19-May-2011  yamt radixtree: comments
 1.4 19-May-2011  yamt radixtree: comments
 1.3 26-Apr-2011  yamt fix _STANDALONE build
 1.2 14-Apr-2011  yamt - fix _STANDALONE build.
- use __CTASSERT instead of CTASSERT. enable it for userland.
- __read_mostly.
 1.1 22-Feb-2011  yamt branches: 1.1.2;
an implementation of radix tree. the idea from linux.
 1.1.2.2 05-Mar-2011  bouyer Sync with HEAD
 1.1.2.1 22-Feb-2011  bouyer file radixtree.c was added on branch bouyer-quota2 on 2011-03-05 15:08:32 +0000
 1.17.44.4 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.17.44.3 21-Apr-2020  martin Sync with HEAD
 1.17.44.2 13-Apr-2020  martin Mostly merge changes from HEAD upto 20200411
 1.17.44.1 08-Apr-2020  martin Merge changes from current as of 20200406
 1.17.2.6 22-May-2014  yamt suppress gcc warnings
 1.17.2.5 25-Mar-2014  yamt comments. some ascii arts to explain memory consumption.
 1.17.2.4 01-Aug-2012  yamt make tag-variants of radix tree functions take and return a mask of tags
rather than tag ids so that they can deal with multiple tags at once.
 1.17.2.3 13-Jun-2012  yamt comment
 1.17.2.2 17-Feb-2012  yamt comments
 1.17.2.1 25-Nov-2011  yamt radix_tree_gang_lookup_node and its variants: add a option to stop on a hole.
 1.20.2.1 29-Feb-2020  ad Sync with head.
 1.16 16-Sep-2021  andvar fix typos in word "successfully", mainly s/succesfully/successfully/.
 1.15 09-May-2019  skrll toolify

PR/54182: Cross-building on Linux fails in rb.c
 1.14 08-Mar-2019  roy rbtree: Add a define to mark function arguments as unused for non debug

This allows rbtree to be used outside of NetBSD without any compile
warnings and removes the need for the lint comment.
 1.13 22-Aug-2014  matt branches: 1.13.16;
Remove enclosing parens on return.
 1.12 22-Aug-2014  riastradh Fix failure case in rb_tree_find_node_leq/geq.

Return NULL, not `NULL - offset'.

XXX pullup to netbsd-5, netbsd-6, netbsd-7
 1.11 20-Jun-2011  mrg branches: 1.11.4; 1.11.22;
various build fixes for gcc 4.5. from chuq. XXX i'm not sure all of
these work properly wtf pointer aliasing, but there are no casts at
least...

the lib/libpuffs/puffs_priv.h is definately a real bug fix.

from chuq.
 1.10 12-Apr-2011  matt branches: 1.10.2;
Add RCSIDs
 1.9 17-Nov-2010  tron Provide proper namespace protection for rbtree(3) because it is now
used by "libc" internally.
 1.8 25-Sep-2010  matt Rename rb.h to rbtree.h, as it is more appropriate (c.f. ptree.h). Also
helps find code that hasn't been updated to use the new rbtree API.
 1.7 24-Sep-2010  rmind Fixes/improvements to RB-tree implementation:
1. Fix inverted node order, so that negative value from comparison operator
would represent lower (left) node, and positive - higher (right) node.
2. Add an argument (i.e. "context"), passed to comparison operators.
3. Change rb_tree_insert_node() to return a node - either inserted one or
already existing one.
4. Amend the interface to manipulate the actual object, instead of the
rb_node (in a similar way as Patricia-tree interface does).
5. Update all RB-tree users accordingly.

XXX: Perhaps rename rb.h to rbtree.h, since cleaning-up..

1-3 address the PR/43488 by Jeremy Huddleston.

Passes RB-tree regression tests.
Reviewed by: matt@, christos@
 1.6 30-Apr-2010  joerg Fix rb_tree_iterate to get the min/max element of the tree for the
RBSMALL case.
 1.5 28-Apr-2010  joerg Fix warnings for -DRBSMALL
 1.4 19-May-2009  yamt don't include sys/cdefs.h twice.
 1.3 30-Jun-2008  matt branches: 1.3.2; 1.3.8;
Changes to make lint STFU.
 1.2 30-Jun-2008  matt Change tree op members/typedefs to rbto_compare_* from rb_compare_*
 1.1 30-Jun-2008  matt Move rb.c from sys/lib/libkern to common so it can be in both libc and libkern.
 1.3.8.1 23-Jul-2009  jym Sync with HEAD.
 1.3.2.2 18-Sep-2008  wrstuden Sync with wrstuden-revivesa-base-2.
 1.3.2.1 30-Jun-2008  wrstuden file rb.c was added on branch wrstuden-revivesa on 2008-09-18 04:54:18 +0000
 1.10.2.1 23-Jun-2011  cherry Catchup with rmind-uvmplock merge.
 1.11.22.1 29-Aug-2014  martin Pull up following revision(s) (requested by riastradh):
common/lib/libc/gen/rb.c: revision 1.12
common/lib/libc/gen/rb.c: revision 1.13
Fix failure case in rb_tree_find_node_leq/geq.
Return NULL, not `NULL - offset'.
XXX pullup to netbsd-5, netbsd-6, netbsd-7
Remove enclosing parens on return.
 1.11.4.1 27-Oct-2014  msaitoh Pull up following revision(s) (requested by riastradh in ticket #1142):
common/lib/libc/gen/rb.c: revision 1.12
common/lib/libc/gen/rb.c: revision 1.13
Fix failure case in rb_tree_find_node_leq/geq.
Return NULL, not `NULL - offset'.
XXX pullup to netbsd-5, netbsd-6, netbsd-7
Remove enclosing parens on return.
 1.13.16.3 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.13.16.2 21-Apr-2020  martin Sync with HEAD
 1.13.16.1 10-Jun-2019  christos Sync with HEAD
 1.12 14-Nov-2021  andvar fix few typos in comments and log message.
 1.11 26-Apr-2011  yamt branches: 1.11.46;
fix _STANDALONE build
 1.10 14-Apr-2011  yamt fix _STANDALONE build
 1.9 26-May-2009  yamt branches: 1.9.2;
fix a comment
 1.8 26-May-2009  yamt rpst_iterate_next: remove an unnecessary assignment.
 1.7 26-May-2009  yamt rpst_iterate_first: fix a bug which causes unnecessary scan.
 1.6 26-May-2009  yamt sprinkle some inline
 1.5 25-May-2009  yamt maintain parent node pointers to speed up search and node removal.
 1.4 25-May-2009  yamt add comments.
 1.3 22-May-2009  yamt rpst_insert_node1: fix an inverted condition.
 1.2 20-May-2009  yamt - fix various bugs in the iteration code.
- add assertions.
- unittest: more tests. verify query results by comparing with linear search.
 1.1 19-May-2009  yamt radix priority search tree.
 1.9.2.2 23-Jul-2009  jym Sync with HEAD.
 1.9.2.1 26-May-2009  jym file rpst.c was added on branch jym-xensuspend on 2009-07-23 23:31:34 +0000
 1.11.46.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.11.46.1 21-Apr-2020  martin Sync with HEAD
 1.18 23-Feb-2024  christos fix static unused issue until lint understands attributes better.
 1.17 30-Aug-2021  christos remove lint exclusion
 1.16 14-Aug-2021  ryo Improved the performance of kernel profiling on MULTIPROCESSOR, and possible to get profiling data for each CPU.

In the current implementation, locks are acquired at the entrance of the mcount
internal function, so the higher the number of cores, the more lock conflict
occurs, making profiling performance in a MULTIPROCESSOR environment unusable
and slow. Profiling buffers has been changed to be reserved for each CPU,
improving profiling performance in MP by several to several dozen times.

- Eliminated cpu_simple_lock in mcount internal function, using per-CPU buffers.
- Add ci_gmon member to struct cpu_info of each MP arch.
- Add kern.profiling.percpu node in sysctl tree.
- Add new -c <cpuid> option to kgmon(8) to specify the cpuid, like openbsd.
For compatibility, if the -c option is not specified, the entire system can be
operated as before, and the -p option will get the total profiling data for
all CPUs.
 1.15 14-Aug-2021  ryo don't include "opt_multiprocessor.h" inside an ifdef to work "make depend" properly.
 1.14 27-Aug-2019  kamil Enhance the support of LLVM sanitizers

Define _REENTRANT for MKSANITIZER build. This is needed for at least stdio
code. This caused new build issued with duplicated symbols in few places
and rump kernel code picking different code paths borrowed from libc.
Handle all this in one go.

Add bsd.sanitizer.mk to share common code used by programs and libraries.

Switch from realall to beforeinstall target in .syms files. This is more
reliable in MKSANITIZER.
 1.13 28-Feb-2016  christos branches: 1.13.16; 1.13.18;
fix infinite recursion through thr_getspecific
 1.12 11-Jan-2016  christos Put back the rump-disabling of mcount for librump. Otherwise the world
breaks because the mcount assembly code is written to call __mcount via
the PLT.
 1.11 10-Jan-2016  ryo __mcount_lock is moved to MI from MD.
because it is needed for all MULTIPROCESSOR arch, but it is exists only in i386 and amd64.

ok christos@, on tech-kern@
 1.10 20-Mar-2012  matt Remove __P usage.
Convert to C89 prototypes.
 1.9 17-Mar-2012  martin shut up lint
 1.8 05-Jan-2009  pooka branches: 1.8.8;
Temporarily define MCOUNT as a dummy for _RUMPKERNEL to allow build
to proceed - the MD macros use processor instructions unavailable
to userspace and should/could be provided as interfaces.
 1.7 27-Oct-2006  uwe branches: 1.7.2;
Simplify the __attribute__ ifdef mess now that __used does the right
thing for older gccs.
 1.6 26-Oct-2006  uwe Do the used/unused dance under #ifdef KERNEL too. Prevents mcount
from being optimized away when compiling sh3 kernels with profiling
enabled (gcc doesn't see that __mcount, which is written in asm,
refers to it).
 1.5 08-Jan-2006  christos always compile in the userland portion. XXX: We might want to change this
to __no_instrument_function__ like the kernel does.
 1.4 21-Dec-2005  christos conditionally build this on GPROF again.
 1.3 21-Dec-2005  christos make this compile again.
 1.2 21-Dec-2005  christos Build also when GPROF is not defined.
 1.1 20-Dec-2005  christos Merge libkern + libc common files. As requested by core.
 1.7.2.2 27-Oct-2006  uwe Simplify the __attribute__ ifdef mess now that __used does the right
thing for older gccs.
 1.7.2.1 27-Oct-2006  uwe file mcount.c was added on branch newlock2 on 2006-10-27 22:14:14 +0000
 1.8.8.1 17-Apr-2012  yamt sync with head
 1.13.18.1 01-Sep-2019  martin Pull up following revision(s) (requested by kamil in ticket #126):

external/bsd/compiler_rt/lib/clang/lib/netbsd/ubsan_minimal-m64/Makefile: revision 1.3
external/bsd/compiler_rt/lib/clang/lib/netbsd/msan_cxx-m64/Makefile: revision 1.3
external/gpl2/lvm2/lvm2tools.mk: revision 1.5
common/lib/libc/gmon/mcount.c: revision 1.14
Makefile: revision 1.331
share/mk/Makefile: revision 1.51
external/bsd/fetch/lib/Makefile: revision 1.12
external/bsd/compiler_rt/lib/clang/lib/netbsd/asan_cxx-m64/Makefile: revision 1.3
tests/lib/libc/net/Makefile: revision 1.12
external/bsd/compiler_rt/lib/clang/lib/netbsd/ubsan_standalone_cxx-m64/Makefile: revision 1.3
external/bsd/compiler_rt/lib/clang/lib/netbsd/msan-m64/Makefile: revision 1.3
distrib/sets/lists/base/mi: revision 1.1213
share/mk/bsd.lib.mk: revision 1.380
external/bsd/compiler_rt/lib/clang/lib/netbsd/ubsan_standalone-m64/Makefile: revision 1.3
external/bsd/compiler_rt/lib/clang/lib/netbsd/asan-m64/Makefile: revision 1.3
share/mk/bsd.sanitizer.mk: revision 1.1
share/mk/bsd.prog.mk: revision 1.323
external/bsd/compiler_rt/lib/clang/lib/netbsd/tsan-m64/Makefile: revision 1.3
external/bsd/compiler_rt/lib/clang/lib/netbsd/tsan_cxx-m64/Makefile: revision 1.3
lib/libc/gen/sysctlgetmibinfo.c: revision 1.14

Enhance the support of LLVM sanitizers

Define _REENTRANT for MKSANITIZER build. This is needed for at least stdio
code. This caused new build issued with duplicated symbols in few places
and rump kernel code picking different code paths borrowed from libc.

Handle all this in one go.

Add bsd.sanitizer.mk to share common code used by programs and libraries.

Switch from realall to beforeinstall target in .syms files. This is more
reliable in MKSANITIZER.
 1.13.16.3 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.13.16.2 21-Apr-2020  martin Sync with HEAD
 1.13.16.1 13-Apr-2020  martin Mostly merge changes from HEAD upto 20200411
 1.9 15-Sep-2025  christos pacify lint
 1.8 20-Aug-2019  para add now required includes for memcpy prototypes analogue to other hash functions

(fix the build)
 1.7 20-Aug-2019  riastradh Fix byte order bug in murmurhash and pacify sanitizers.
 1.6 26-Oct-2013  rmind branches: 1.6.26; 1.6.28;
murmurhash2: add an optimised path for the aligned pointer case.
 1.5 30-Jun-2013  rmind RUMP fix: ifdef weak alias. Pointed out by martin@.
 1.4 10-Jul-2012  christos branches: 1.4.2; 1.4.4;
Fix kernel compilation: no namespace.h
Fix libc RCSID protection.
Remove useless cast.
 1.3 09-Jul-2012  rmind Add weak symbol for murmurhash2.
 1.2 08-Jul-2012  rmind Shut up lint.
 1.1 08-Jul-2012  rmind Add MurmurHash2 -- a non-cryptographic hash function by Austin Appleby.
The code is taken from the upstream and is in the public domain.

OK christos@
 1.4.4.3 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.4.4.2 30-Oct-2012  yamt sync with head
 1.4.4.1 10-Jul-2012  yamt file murmurhash.c was added on branch yamt-pagecache on 2012-10-30 18:46:13 +0000
 1.4.2.1 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.6.28.1 08-Dec-2019  martin Pull up following revision(s) (requested by riastradh in ticket #505):

common/lib/libc/hash/murmurhash/murmurhash.c: revision 1.7
common/lib/libc/hash/murmurhash/murmurhash.c: revision 1.8
sys/sys/param.h: revision 1.610
sys/arch/amd64/include/param.h: revision 1.31
sys/arch/i386/include/param.h: revision 1.85

New macro ALIGNED_POINTER_LOAD.

To be used with ALIGNED_POINTER(p,t) instead of writing *(const t *)p
directly. This way, on machines without strict alignment, we can use
memcpy to pacify sanitizers, while getting the same compiled code in
the end with a single (say) MOV instruction.

Fix byte order bug in murmurhash and pacify sanitizers.
add now required includes for memcpy prototypes analogue to other hash functions
(fix the build)
 1.6.26.3 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.6.26.2 21-Apr-2020  martin Sync with HEAD
 1.6.26.1 13-Apr-2020  martin Mostly merge changes from HEAD upto 20200411
 1.7 21-Apr-2015  riastradh branches: 1.7.16;
Sort includes.
 1.6 15-Apr-2015  justin On big endian machines needs to include sys/endian.h for le32dec

Mainly affects cross builds on big endian; tested on FreeBSD mips
and Linux ppc.
 1.5 21-Aug-2009  skrll Don't define weak aliases if _STANDALONE.
 1.4 16-Feb-2008  apb For all files under src/common, as well as related man pages under
src/lib/libc, or related headers under src/sys/sys: change u_intNN_t to
uintNN_t.
 1.3 18-Jul-2007  joerg branches: 1.3.4; 1.3.8;
Fix SIGBUS issues on strict alignment issues. Use le32dec in RMD160
as the data pointer to RMD160_Update doesn't have to be aligned.
In SHA256_Update and SHA512_Update, only operate directly on the passed
in data if no left-over in the context exists and the data is correctly
aligned. The problem was exposed by the audit-packages rewrite in C
and reported for the libnbcompat version in PR pkg/36662.
 1.2 01-Nov-2006  he branches: 1.2.2; 1.2.4; 1.2.12;
Add include of <sys/param.h>, to allow vax kernels to build again.
For vax, this causes <machine/macros.h> to be included and the
redefine of memset() to take effect.
 1.1 27-Oct-2006  christos this is shared with the kernel now.
 1.2.12.1 03-Sep-2007  wrstuden Sync w/ NetBSD-4-RC_1
 1.2.4.2 18-Nov-2008  snj Pull up following revision(s) (requested by tsutsui in ticket #1224):
common/lib/libc/gen/bswap64.c: revision 1.2
common/lib/libprop/prop_data.c: revision 1.10
common/lib/libc/inet/inet_addr.c: revision 1.2
common/lib/libc/hash/rmd160/rmd160.c: revision 1.4
common/lib/libc/gen/bswap32.c: revision 1.2
common/lib/libc/gen/bswap16.c: revision 1.2
common/lib/libc/arch/alpha/gen/byte_swap_4.S: revision 1.3
common/lib/libc/arch/sh3/string/memset.S: revision 1.2
common/lib/libc/arch/alpha/gen/byte_swap_2.S: revision 1.3
common/lib/libc/md/md5c.c: revision 1.3
common/lib/libc/hash/sha1/sha1.c: revision 1.3
common/lib/libc/md/md4c.c: revision 1.3
sys/sys/rmd160.h: revision 1.2
sys/sys/sha2.h: revision 1.2
For all files under src/common, as well as related man pages under
src/lib/libc, or related headers under src/sys/sys: change u_intNN_t to
uintNN_t.
 1.2.4.1 18-Jul-2007  liamjfoy Pull up following revision(s) (requested by joerg in ticket #776):
common/lib/libc/hash/sha2/sha2.c: revision 1.5
common/lib/libc/hash/sha2/sha2.c: revision 1.6
common/lib/libc/hash/rmd160/rmd160.c: revision 1.3
Fix SIGBUS issues on strict alignment issues. Use le32dec in RMD160
as the data pointer to RMD160_Update doesn't have to be aligned.
In SHA256_Update and SHA512_Update, only operate directly on the passed
in data if no left-over in the context exists and the data is correctly
aligned. The problem was exposed by the audit-packages rewrite in C
and reported for the libnbcompat version in PR pkg/36662.
cast to void* to avoid a fatal warning
 1.2.2.2 01-Nov-2006  he Add include of <sys/param.h>, to allow vax kernels to build again.
For vax, this causes <machine/macros.h> to be included and the
redefine of memset() to take effect.
 1.2.2.1 01-Nov-2006  he file rmd160.c was added on branch newlock2 on 2006-11-01 11:29:09 +0000
 1.3.8.2 18-Jul-2007  joerg Fix SIGBUS issues on strict alignment issues. Use le32dec in RMD160
as the data pointer to RMD160_Update doesn't have to be aligned.
In SHA256_Update and SHA512_Update, only operate directly on the passed
in data if no left-over in the context exists and the data is correctly
aligned. The problem was exposed by the audit-packages rewrite in C
and reported for the libnbcompat version in PR pkg/36662.
 1.3.8.1 18-Jul-2007  joerg file rmd160.c was added on branch matt-mips64 on 2007-07-18 13:57:55 +0000
 1.3.4.1 23-Mar-2008  matt sync with HEAD
 1.7.16.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.7.16.1 21-Apr-2020  martin Sync with HEAD
 1.7 28-Oct-2021  christos Use defined constant instead of direct value (Etienne Brateau)
 1.6 06-Nov-2009  joerg branches: 1.6.48;
Improve portability and clarity by using uint8_t for the byte data
and expanding u_int to not depend on the !POSIX types.
 1.5 21-Aug-2009  skrll Don't define weak aliases if _STANDALONE.
 1.4 16-Mar-2009  cegger ansify function definitions
 1.3 16-Feb-2008  apb branches: 1.3.12;
For all files under src/common, as well as related man pages under
src/lib/libc, or related headers under src/sys/sys: change u_intNN_t to
uintNN_t.
 1.2 27-Oct-2006  christos branches: 1.2.2; 1.2.4; 1.2.16;
restruct the include files to look like the other hash functions.
 1.1 20-Dec-2005  christos add files I forgot to commit.
 1.2.16.1 23-Mar-2008  matt sync with HEAD
 1.2.4.1 18-Nov-2008  snj Pull up following revision(s) (requested by tsutsui in ticket #1224):
common/lib/libc/gen/bswap64.c: revision 1.2
common/lib/libprop/prop_data.c: revision 1.10
common/lib/libc/inet/inet_addr.c: revision 1.2
common/lib/libc/hash/rmd160/rmd160.c: revision 1.4
common/lib/libc/gen/bswap32.c: revision 1.2
common/lib/libc/gen/bswap16.c: revision 1.2
common/lib/libc/arch/alpha/gen/byte_swap_4.S: revision 1.3
common/lib/libc/arch/sh3/string/memset.S: revision 1.2
common/lib/libc/arch/alpha/gen/byte_swap_2.S: revision 1.3
common/lib/libc/md/md5c.c: revision 1.3
common/lib/libc/hash/sha1/sha1.c: revision 1.3
common/lib/libc/md/md4c.c: revision 1.3
sys/sys/rmd160.h: revision 1.2
sys/sys/sha2.h: revision 1.2
For all files under src/common, as well as related man pages under
src/lib/libc, or related headers under src/sys/sys: change u_intNN_t to
uintNN_t.
 1.2.2.2 27-Oct-2006  christos restruct the include files to look like the other hash functions.
 1.2.2.1 27-Oct-2006  christos file sha1.c was added on branch newlock2 on 2006-10-27 21:25:22 +0000
 1.3.12.1 13-May-2009  jym Sync with HEAD.

Commit is split, to avoid a "too many arguments" protocol error.
 1.6.48.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.6.48.1 21-Apr-2020  martin Sync with HEAD
 1.26 20-Jan-2024  christos move local decls to headers
 1.25 28-Oct-2021  christos Fix build with -Werror=array-parameter (Etienne Brateau)
 1.24 09-Jun-2013  christos branches: 1.24.28;
no need for the local defs of he enc functions; compat_defs.h provides them.
 1.23 09-Jun-2013  christos Use be{32,64}enc, suggested by joerg.
 1.22 07-Jun-2013  christos PR/47908: Gary Grebus: SHA256_Transform and SHA512_Transform are called
by openssl with unaligned buffers. All other Transforms can handle unaligned
buffers so make these handle them too.
XXX[1]: any better fixes are welcome
XXX[2]: pullup-5, pullup-6
 1.21 24-Jan-2010  joerg branches: 1.21.6; 1.21.12;
Fix unaligned access in *_Final for SHA224/SHA256/SHA384.
Remaining part of PR 42273. Tested by snj.
 1.20 06-Nov-2009  joerg Fix unaligned access as reported in PR port-sparc/42273
 1.19 21-Aug-2009  skrll Don't define weak aliases if _STANDALONE.
 1.18 25-Jun-2009  joerg Fix initialisation of SHA224.
 1.17 19-Jun-2009  tsutsui - move #include <sys/param.h> inside #if defined(_KERNEL) section
where it's actually required
- add XXX comments that notes <sys/param.h> is required for vax memset(9)
 1.16 18-Jun-2009  he Back out the addition of memset.c to the vax libkern, and instead do
as tsutsui@ suggested, and include <sys/param.h> in sha2.c instead.
On the vax, this causes <machine/macros.h> to be included, and it contains
that machine's memset() macro+inline.
 1.15 14-Jun-2009  martin Fix copy&paste errors
 1.14 11-Jun-2009  joerg Provide fallback definitions of be[32|64]toh and htobe[32|64] for
systems without sys/endian.h.
 1.13 11-Jun-2009  christos use memcpy for bitcount writes to the buffer to avoid type punning issues.
 1.12 11-Jun-2009  christos - fix sha224 functions that used sha256 ctx, and add casts.
- fix sha224 function lengths.
- wrap long lines
- use NULL
- fix whitespace
- put all 224 functions together in the same spirit as 384
 1.11 11-Jun-2009  joerg KNF
 1.10 11-Jun-2009  joerg De-uglify:
- drop useless asserts
- use uintXX_t directly
- use NULL
 1.9 11-Jun-2009  joerg Simplify by using sys/endian.h functions. Fixes Big Endian case for SHA2.
 1.8 26-May-2009  joerg Add SHA224 implementation to libc.
Make libcrypto use the SHA2 implementation of libc.
Bump minor versions of libc and libcrypto.
 1.7 16-Feb-2008  apb branches: 1.7.10; 1.7.12; 1.7.14;
C99 is no longer "very recent". Adjust a comment accordingly, and
start using uintNN_t instead of u_intNN_t in the definitions of
sha2_{byte,word32,word64}.
 1.6 18-Jul-2007  drochner branches: 1.6.4; 1.6.8;
cast to void* to avoid a fatal warning
 1.5 18-Jul-2007  joerg Fix SIGBUS issues on strict alignment issues. Use le32dec in RMD160
as the data pointer to RMD160_Update doesn't have to be aligned.
In SHA256_Update and SHA512_Update, only operate directly on the passed
in data if no left-over in the context exists and the data is correctly
aligned. The problem was exposed by the audit-packages rewrite in C
and reported for the libnbcompat version in PR pkg/36662.
 1.4 18-Feb-2007  christos Make SHA512_Last static since it is not part of the API. Suggested by
Matthias Scheler
 1.3 17-Feb-2007  christos make SHA512_Last weak.
 1.2 01-Nov-2006  he branches: 1.2.2; 1.2.4;
Add include of <sys/param.h>, to allow vax kernels to build again.
For vax, this causes <machine/macros.h> to be included and the
redefine of memset() to take effect.
 1.1 27-Oct-2006  christos this is shared with the kernel now.
 1.2.4.7 27-Jan-2010  bouyer Pull up following revision(s) (requested by joerg in ticket #1382):
common/lib/libc/hash/sha2/sha2.c: revision 1.21
Fix unaligned access in *_Final for SHA224/SHA256/SHA384.
Remaining part of PR 42273. Tested by snj.
 1.2.4.6 27-Jan-2010  bouyer Pull up following revision(s) (requested by joerg in ticket #1381):
common/lib/libc/hash/sha2/sha2.c: revision 1.20
Fix unaligned access as reported in PR port-sparc/42273
 1.2.4.5 22-Jul-2009  snj Apply patch (requested by joerg in ticket #1338):
- Add support for SHA224 to libc
- Fix Big Endian support in some cases
- Use libc support in libcrypto
 1.2.4.4 18-Nov-2008  snj Pull up following revision(s) (requested by tsutsui in ticket #1224):
common/lib/libc/hash/sha2/sha2.c: revision 1.7
C99 is no longer "very recent". Adjust a comment accordingly, and
start using uintNN_t instead of u_intNN_t in the definitions of
sha2_{byte,word32,word64}.
 1.2.4.3 18-Jul-2007  liamjfoy branches: 1.2.4.3.2;
Pull up following revision(s) (requested by joerg in ticket #776):
common/lib/libc/hash/sha2/sha2.c: revision 1.5
common/lib/libc/hash/sha2/sha2.c: revision 1.6
common/lib/libc/hash/rmd160/rmd160.c: revision 1.3
Fix SIGBUS issues on strict alignment issues. Use le32dec in RMD160
as the data pointer to RMD160_Update doesn't have to be aligned.
In SHA256_Update and SHA512_Update, only operate directly on the passed
in data if no left-over in the context exists and the data is correctly
aligned. The problem was exposed by the audit-packages rewrite in C
and reported for the libnbcompat version in PR pkg/36662.
cast to void* to avoid a fatal warning
 1.2.4.2 19-Feb-2007  tron branches: 1.2.4.2.2;
Pull up following revision(s) (requested by christos in ticket #456):
common/lib/libc/hash/sha2/sha2.c: revision 1.4
Make SHA512_Last static since it is not part of the API. Suggested by
Matthias Scheler
 1.2.4.1 19-Feb-2007  tron Pull up following revision(s) (requested by christos in ticket #456):
common/lib/libc/hash/sha2/sha2.c: revision 1.3
make SHA512_Last weak.
 1.2.4.3.2.1 22-Jul-2009  snj Apply patch (requested by joerg in ticket #1338):
- Add support for SHA224 to libc
- Fix Big Endian support in some cases
- Use libc support in libcrypto
 1.2.4.2.2.1 03-Sep-2007  wrstuden Sync w/ NetBSD-4-RC_1
 1.2.2.2 01-Nov-2006  he Add include of <sys/param.h>, to allow vax kernels to build again.
For vax, this causes <machine/macros.h> to be included and the
redefine of memset() to take effect.
 1.2.2.1 01-Nov-2006  he file sha2.c was added on branch newlock2 on 2006-11-01 11:29:09 +0000
 1.6.8.2 18-Jul-2007  drochner cast to void* to avoid a fatal warning
 1.6.8.1 18-Jul-2007  drochner file sha2.c was added on branch matt-mips64 on 2007-07-18 16:58:15 +0000
 1.6.4.1 23-Mar-2008  matt sync with HEAD
 1.7.14.3 30-Jan-2010  snj Pull up following revision(s) (requested by joerg in ticket #1276):
common/lib/libc/hash/sha2/sha2.c: revision 1.21
Fix unaligned access in *_Final for SHA224/SHA256/SHA384.
Remaining part of PR 42273. Tested by snj.
 1.7.14.2 30-Jan-2010  snj Pull up following revision(s) (requested by joerg in ticket #1275):
common/lib/libc/hash/sha2/sha2.c: revision 1.20
Fix unaligned access as reported in PR port-sparc/42273
 1.7.14.1 14-Jul-2009  snj branches: 1.7.14.1.2;
Pull up following revisions (requested by joerg in ticket #855):
common/lib/libc/hash/sha2/sha2.c: revisions 1.8-1.18 via patch
crypto/dist/openssl/crypto/evp/m_sha1.c: revisions 1.2-1.3 via patch
distrib/sets/lists/base/md.amd64: revision 1.51 via patch
distrib/sets/lists/base/md.sparc64: revision 1.46 via patch
distrib/sets/lists/base/shl.mi: revision 1.474 via patch
distrib/sets/lists/comp/mi: revision 1.1263 via patch
lib/libc/shlib_version: patch
lib/libc/hash/sha2/Makefile.inc: revision 1.4 via patch
lib/libc/hash/sha2/sha2.3: revision 1.5 via patch
lib/libc/hash/sha2/sha224hl.c: revision 1.1 via patch
lib/libc/include/namespace.h: revision 1.138 via patch
lib/libcrypto/libc-sha256.c: revision 1.1 via patch
lib/libcrypto/libc-sha512.c: revision 1.1 via patch
lib/libcrypto/sha.inc: revision 1.10 via patch
lib/libcrypto/shlib_version: patch
sys/lib/libkern/arch/vax/Makefile.inc: revision 1.21 via patch
sys/sys/sha2.h: revision 1.3 via patch
Add SHA224 implementation to libc.
Make libcrypto use the SHA2 implementation of libc.
Bump minor versions of libc and libcrypto.
 1.7.14.1.2.1 21-Apr-2010  matt sync to netbsd-5
 1.7.12.1 23-Jul-2009  jym Sync with HEAD.
 1.7.10.3 30-Jan-2010  snj Pull up following revision(s) (requested by joerg in ticket #1276):
common/lib/libc/hash/sha2/sha2.c: revision 1.21
Fix unaligned access in *_Final for SHA224/SHA256/SHA384.
Remaining part of PR 42273. Tested by snj.
 1.7.10.2 30-Jan-2010  snj Pull up following revision(s) (requested by joerg in ticket #1275):
common/lib/libc/hash/sha2/sha2.c: revision 1.20
Fix unaligned access as reported in PR port-sparc/42273.
 1.7.10.1 14-Jul-2009  snj Pull up following revisions (requested by joerg in ticket #855):
common/lib/libc/hash/sha2/sha2.c: revisions 1.8-1.18 via patch
crypto/dist/openssl/crypto/evp/m_sha1.c: revisions 1.2-1.3 via patch
distrib/sets/lists/base/md.amd64: revision 1.51 via patch
distrib/sets/lists/base/md.sparc64: revision 1.46 via patch
distrib/sets/lists/base/shl.mi: revision 1.474 via patch
distrib/sets/lists/comp/mi: revision 1.1263 via patch
lib/libc/shlib_version: patch
lib/libc/hash/sha2/Makefile.inc: revision 1.4 via patch
lib/libc/hash/sha2/sha2.3: revision 1.5 via patch
lib/libc/hash/sha2/sha224hl.c: revision 1.1 via patch
lib/libc/include/namespace.h: revision 1.138 via patch
lib/libcrypto/libc-sha256.c: revision 1.1 via patch
lib/libcrypto/libc-sha512.c: revision 1.1 via patch
lib/libcrypto/sha.inc: revision 1.10 via patch
lib/libcrypto/shlib_version: patch
sys/sys/sha2.h: revision 1.3 via patch
Add SHA224 implementation to libc.
Make libcrypto use the SHA2 implementation of libc.
Bump minor versions of libc and libcrypto.
 1.21.12.1 23-Jun-2013  tls resync from head
 1.21.6.1 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.24.28.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.24.28.1 21-Apr-2020  martin Sync with HEAD
 1.1 30-Nov-2017  riastradh branches: 1.1.4;
Import SHA-3 code into libc and libkern.

No new public symbols in libc, but publishing the symbols is a simple
matter if/when we decide to do so.

Proposed on tech-kern and tech-userlevel with no objections:

https://mail-index.NetBSD.org/tech-kern/2017/11/11/msg022581.html
https://mail-index.NetBSD.org/tech-userlevel/2017/11/11/msg010968.html
 1.1.4.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.1.4.1 21-Apr-2020  martin Sync with HEAD
 1.1 30-Nov-2017  riastradh branches: 1.1.4;
Import SHA-3 code into libc and libkern.

No new public symbols in libc, but publishing the symbols is a simple
matter if/when we decide to do so.

Proposed on tech-kern and tech-userlevel with no objections:

https://mail-index.NetBSD.org/tech-kern/2017/11/11/msg022581.html
https://mail-index.NetBSD.org/tech-userlevel/2017/11/11/msg010968.html
 1.1.4.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.1.4.1 21-Apr-2020  martin Sync with HEAD
 1.4 19-Jan-2024  christos use size_t
 1.3 31-Jul-2021  andvar fix more typos in style found one in file - check/fix them all.
 1.2 30-May-2020  riastradh Merge updates from upstream to reduce stack usage of SHA3_Selftest.
 1.1 30-Nov-2017  riastradh branches: 1.1.4;
Import SHA-3 code into libc and libkern.

No new public symbols in libc, but publishing the symbols is a simple
matter if/when we decide to do so.

Proposed on tech-kern and tech-userlevel with no objections:

https://mail-index.NetBSD.org/tech-kern/2017/11/11/msg022581.html
https://mail-index.NetBSD.org/tech-userlevel/2017/11/11/msg010968.html
 1.1.4.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.1.4.1 21-Apr-2020  martin Sync with HEAD
 1.4 26-Jul-2018  kamil Avoid undefined behavior in an inet_addr.c

Do not change the signedness bit with a left shift operation.
Cast to unsigned integer to prevent this.

inet_addr.c:218:20, left shift of 131 by 24 places cannot be represented in type 'int'

Detected with micro-UBSan in the user mode.
 1.3 09-Mar-2012  christos branches: 1.3.32; 1.3.34;
Casts and type changes to fix portability issues.
- int -> size_t
- adjust width of RHS of shift
- adjust widths of types
 1.2 16-Feb-2008  apb branches: 1.2.26;
For all files under src/common, as well as related man pages under
src/lib/libc, or related headers under src/sys/sys: change u_intNN_t to
uintNN_t.
 1.1 20-Dec-2005  christos branches: 1.1.6; 1.1.18;
Merge libkern + libc common files. As requested by core.
 1.1.18.1 23-Mar-2008  matt sync with HEAD
 1.1.6.1 18-Nov-2008  snj Pull up following revision(s) (requested by tsutsui in ticket #1224):
common/lib/libc/gen/bswap64.c: revision 1.2
common/lib/libprop/prop_data.c: revision 1.10
common/lib/libc/inet/inet_addr.c: revision 1.2
common/lib/libc/hash/rmd160/rmd160.c: revision 1.4
common/lib/libc/gen/bswap32.c: revision 1.2
common/lib/libc/gen/bswap16.c: revision 1.2
common/lib/libc/arch/alpha/gen/byte_swap_4.S: revision 1.3
common/lib/libc/arch/sh3/string/memset.S: revision 1.2
common/lib/libc/arch/alpha/gen/byte_swap_2.S: revision 1.3
common/lib/libc/md/md5c.c: revision 1.3
common/lib/libc/hash/sha1/sha1.c: revision 1.3
common/lib/libc/md/md4c.c: revision 1.3
sys/sys/rmd160.h: revision 1.2
sys/sys/sha2.h: revision 1.2
For all files under src/common, as well as related man pages under
src/lib/libc, or related headers under src/sys/sys: change u_intNN_t to
uintNN_t.
 1.2.26.1 17-Apr-2012  yamt sync with head
 1.3.34.3 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.3.34.2 21-Apr-2020  martin Sync with HEAD
 1.3.34.1 10-Jun-2019  christos Sync with HEAD
 1.3.32.1 28-Jul-2018  pgoyette Sync with HEAD
 1.5 20-Mar-2012  matt branches: 1.5.34;
Remove __P usage.
Convert to C89 prototypes.
 1.4 16-Mar-2009  cegger branches: 1.4.6;
ansify function definitions
 1.3 16-Feb-2008  apb branches: 1.3.12;
For all files under src/common, as well as related man pages under
src/lib/libc, or related headers under src/sys/sys: change u_intNN_t to
uintNN_t.
 1.2 24-Dec-2005  matt branches: 1.2.6; 1.2.18;
Include <sys/param.h> first to make VAX port happy.
 1.1 20-Dec-2005  christos Merge libkern + libc common files. As requested by core.
 1.2.18.1 23-Mar-2008  matt sync with HEAD
 1.2.6.1 18-Nov-2008  snj Pull up following revision(s) (requested by tsutsui in ticket #1224):
common/lib/libc/gen/bswap64.c: revision 1.2
common/lib/libprop/prop_data.c: revision 1.10
common/lib/libc/inet/inet_addr.c: revision 1.2
common/lib/libc/hash/rmd160/rmd160.c: revision 1.4
common/lib/libc/gen/bswap32.c: revision 1.2
common/lib/libc/gen/bswap16.c: revision 1.2
common/lib/libc/arch/alpha/gen/byte_swap_4.S: revision 1.3
common/lib/libc/arch/sh3/string/memset.S: revision 1.2
common/lib/libc/arch/alpha/gen/byte_swap_2.S: revision 1.3
common/lib/libc/md/md5c.c: revision 1.3
common/lib/libc/hash/sha1/sha1.c: revision 1.3
common/lib/libc/md/md4c.c: revision 1.3
sys/sys/rmd160.h: revision 1.2
sys/sys/sha2.h: revision 1.2
For all files under src/common, as well as related man pages under
src/lib/libc, or related headers under src/sys/sys: change u_intNN_t to
uintNN_t.
 1.3.12.1 13-May-2009  jym Sync with HEAD.

Commit is split, to avoid a "too many arguments" protocol error.
 1.4.6.1 17-Apr-2012  yamt sync with head
 1.5.34.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.5.34.1 21-Apr-2020  martin Sync with HEAD
 1.5 20-Mar-2012  matt branches: 1.5.34;
Remove __P usage.
Convert to C89 prototypes.
 1.4 16-Mar-2009  cegger branches: 1.4.6;
ansify function definitions
 1.3 16-Feb-2008  apb branches: 1.3.12;
For all files under src/common, as well as related man pages under
src/lib/libc, or related headers under src/sys/sys: change u_intNN_t to
uintNN_t.
 1.2 24-Dec-2005  matt branches: 1.2.6; 1.2.18;
Include <sys/param.h> first to make VAX port happy.
 1.1 20-Dec-2005  christos Merge libkern + libc common files. As requested by core.
 1.2.18.1 23-Mar-2008  matt sync with HEAD
 1.2.6.1 18-Nov-2008  snj Pull up following revision(s) (requested by tsutsui in ticket #1224):
common/lib/libc/gen/bswap64.c: revision 1.2
common/lib/libprop/prop_data.c: revision 1.10
common/lib/libc/inet/inet_addr.c: revision 1.2
common/lib/libc/hash/rmd160/rmd160.c: revision 1.4
common/lib/libc/gen/bswap32.c: revision 1.2
common/lib/libc/gen/bswap16.c: revision 1.2
common/lib/libc/arch/alpha/gen/byte_swap_4.S: revision 1.3
common/lib/libc/arch/sh3/string/memset.S: revision 1.2
common/lib/libc/arch/alpha/gen/byte_swap_2.S: revision 1.3
common/lib/libc/md/md5c.c: revision 1.3
common/lib/libc/hash/sha1/sha1.c: revision 1.3
common/lib/libc/md/md4c.c: revision 1.3
sys/sys/rmd160.h: revision 1.2
sys/sys/sha2.h: revision 1.2
For all files under src/common, as well as related man pages under
src/lib/libc, or related headers under src/sys/sys: change u_intNN_t to
uintNN_t.
 1.3.12.1 13-May-2009  jym Sync with HEAD.

Commit is split, to avoid a "too many arguments" protocol error.
 1.4.6.1 17-Apr-2012  yamt sync with head
 1.5.34.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.5.34.1 21-Apr-2020  martin Sync with HEAD
 1.12 07-Dec-2023  andvar s/miltiple/multiple/ in comment.
 1.11 19-Sep-2021  andvar fix various typos in comments, messages and documentation.
 1.10 08-Mar-2020  kamil Add support for alignment_assumptions in uubsan

Cherry-pick from FreeBSD:

From 7c1bc5ffc2fa68ddc76e5ea8a3a1a6fdfeee57f0 Mon Sep 17 00:00:00 2001
From: andrew <andrew@FreeBSD.org>
Date: Tue, 28 May 2019 09:12:15 +0000
Subject: [PATCH] Teach the kernel KUBSAN runtime about alignment_assumption

This checks the alignment of a given pointer is sufficient for the
requested alignment asked for. This fixes the build with a recent
llvm/clang.

Sponsored by: DARPA, AFRL
 1.9 01-Nov-2019  kamil uubsan: Implement function_type_mismatch_v1

RTTI is not supported by micro-UBSan (by design) and this is now a stub
handler.
 1.8 30-Oct-2019  kamil uubsan: Handle implicit_conversion
 1.7 04-Oct-2019  mrg workaround a GCC 8 warning:

- code that will be unreachable on platforms with
sizeof(double) != sizeof(unsigned long) triggered an valid out
of bounds warning. avoid the error by using sizeof ul.
- also assert that the sizes are the same if entering here.

both from kamil@.
 1.6 17-Jun-2019  kamil Introduce UBSAN_ALWAYS_FATAL in micro-UBSan

UBSAN_ALWAYS_FATAL is a CPP option to mark all reports fatal always.
 1.5 13-Feb-2019  kamil branches: 1.5.2;
Fix kUBSan build with GCC7

Add missing __unreachable() and FALLTHROUGH keywords.

Reported by <Akul Pillai>
 1.4 04-Feb-2019  mrg - use __unreachable() and move 'break's around to increase consistency
and correctness

ok kamil@
 1.3 03-Aug-2018  kamil branches: 1.3.2;
Try to fix the evbppc-powerpc64 build

Avoid "comparison between signed and unsigned integer expressions" on
Big-Endian hosts.
 1.2 03-Aug-2018  kamil Tidy up the comment in ubsan.c

As noted, style has no impact on the comparison of a similar code.
This version is a reimplementation from scratch with no code and no
algirithm (whenever possible) reuse.

Public symbols and struct layout must be kept in sync with the code
generation part. It casues violation of the style like with long filenames.

My previous comment was 'too perfect' and could trigger some unnecessary
attention.
 1.1 03-Aug-2018  kamil Import micro-UBSan (ubsan.c)

This is a reimplementation of the Undefined Behavior Sanitizer with the
following properties:
- pure and clean-room C implementation,
- no -fsanitize=vpts support, as it requires RTTI support and C++
low-level routies to validate whether C++ objects are compatible
- designed to be used inside libc and known as uUBSan or user-UBSan
- designed to be shared with kernel and known as kUBSan or kernel-UBSan
- designed to be usable with ATF tests as a standalone runtime,
reachable without any MK* switches
- designed to be safer for hardening as it does not have side effects on
executables like writing to a selected location on demand
- controllable with environment variable LIBC_UBSAN with options:
* a - abort on report
* A - do not abort on a report (unless a failure is unrecoverable)
* e - output report to stderr
* E - do not output report on stderr
* l - output report on syslog (LOG_DEBUG | LOG_USER)
* L - do not output report on syslog
* o - output report on stdout
* O - do not output report on stdout
The default options are: "AeLO".
- compatible with Clang (3.8, 7.x) and GCC (6.x) code generation
- all handlers (except =vptr) from Clang/LLVM up to 7svn are supported

Tested with Clang amd64+i386 and GCC amd64+i386.
 1.3.2.2 06-Sep-2018  pgoyette Sync with HEAD

Resolve a couple of conflicts (result of the uimin/uimax changes)
 1.3.2.1 03-Aug-2018  pgoyette file ubsan.c was added on branch pgoyette-compat on 2018-09-06 06:51:32 +0000
 1.5.2.5 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.5.2.4 21-Apr-2020  martin Sync with HEAD
 1.5.2.3 13-Apr-2020  martin Mostly merge changes from HEAD upto 20200411
 1.5.2.2 10-Jun-2019  christos Sync with HEAD
 1.5.2.1 13-Feb-2019  christos file ubsan.c was added on branch phil-wifi on 2019-06-10 21:41:07 +0000
 1.4 20-Jan-2012  joerg Change CMSG_SPACE and CMSG_LEN to provide Integer Constant Expressions
again. This was changed in sys/socket.h r1.51 to work around fallout
from the IPv6 aux data migration. It broke the historic ABI on some
platforms. This commit restores compatibility for netbsd32 code on such
platforms and provides a template for future changes to the CMSG_*
alignment. Revert PCC/Clang workarounds in postfix and tmux.
 1.3 16-Mar-2009  cegger branches: 1.3.6;
ansify function definitions
 1.2 28-Apr-2008  martin branches: 1.2.8;
Remove clause 3 and 4 from TNF licenses
 1.1 20-Dec-2005  christos branches: 1.1.26;
Merge libkern + libc common files. As requested by core.
 1.1.26.1 18-May-2008  yamt sync with head.
 1.2.8.1 13-May-2009  jym Sync with HEAD.

Commit is split, to avoid a "too many arguments" protocol error.
 1.3.6.1 17-Apr-2012  yamt sync with head
 1.3 21-Mar-2012  he branches: 1.3.34;
Lint seems to prefer that we cast via (void*) and not directly to
(u_char*), so follow suit.
 1.2 04-Jul-2011  joerg branches: 1.2.2;
ANSIfy
 1.1 20-Dec-2005  christos Merge libkern + libc common files. As requested by core.
 1.2.2.1 17-Apr-2012  yamt sync with head
 1.3.34.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.3.34.1 21-Apr-2020  martin Sync with HEAD
 1.3 21-Mar-2012  he branches: 1.3.34;
Lint seems to prefer that we cast via (void*) and not directly to
(u_char*), so follow suit.
 1.2 04-Jul-2011  joerg branches: 1.2.2;
ANSIfy
 1.1 20-Dec-2005  christos Merge libkern + libc common files. As requested by core.
 1.2.2.1 17-Apr-2012  yamt sync with head
 1.3.34.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.3.34.1 21-Apr-2020  martin Sync with HEAD
 1.3 21-Mar-2012  he branches: 1.3.34;
Lint seems to prefer that we cast via (void*) and not directly to
(u_char*), so follow suit.
 1.2 04-Jul-2011  joerg branches: 1.2.2;
ANSIfy
 1.1 20-Dec-2005  christos Merge libkern + libc common files. As requested by core.
 1.2.2.1 17-Apr-2012  yamt sync with head
 1.3.34.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.3.34.1 21-Apr-2020  martin Sync with HEAD
 1.3 21-Mar-2012  he branches: 1.3.34;
Lint seems to prefer that we cast via (void*) and not directly to
(u_char*), so follow suit.
 1.2 04-Jul-2011  joerg branches: 1.2.2;
ANSIfy
 1.1 20-Dec-2005  christos Merge libkern + libc common files. As requested by core.
 1.2.2.1 17-Apr-2012  yamt sync with head
 1.3.34.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.3.34.1 21-Apr-2020  martin Sync with HEAD
 1.2 15-Mar-2009  cegger branches: 1.2.48;
ansify function definitions
 1.1 20-Dec-2005  christos branches: 1.1.36;
Merge libkern + libc common files. As requested by core.
 1.1.36.1 13-May-2009  jym Sync with HEAD.

Commit is split, to avoid a "too many arguments" protocol error.
 1.2.48.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.2.48.1 21-Apr-2020  martin Sync with HEAD
 1.2 15-Mar-2009  cegger branches: 1.2.48;
ansify function definitions
 1.1 20-Dec-2005  christos branches: 1.1.36;
Merge libkern + libc common files. As requested by core.
 1.1.36.1 13-May-2009  jym Sync with HEAD.

Commit is split, to avoid a "too many arguments" protocol error.
 1.2.48.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.2.48.1 21-Apr-2020  martin Sync with HEAD
 1.3 09-Mar-2012  christos branches: 1.3.34;
Casts and type changes to fix portability issues.
- int -> size_t
- adjust width of RHS of shift
- adjust widths of types
 1.2 15-Mar-2009  cegger branches: 1.2.6;
ansify function definitions
 1.1 20-Dec-2005  christos branches: 1.1.36;
Merge libkern + libc common files. As requested by core.
 1.1.36.1 13-May-2009  jym Sync with HEAD.

Commit is split, to avoid a "too many arguments" protocol error.
 1.2.6.1 17-Apr-2012  yamt sync with head
 1.3.34.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.3.34.1 21-Apr-2020  martin Sync with HEAD
 1.4 06-Aug-2012  matt branches: 1.4.34;
Add ARM EABI aliases to a few functions.
 1.3 09-Mar-2012  christos Casts and type changes to fix portability issues.
- int -> size_t
- adjust width of RHS of shift
- adjust widths of types
 1.2 15-Mar-2009  cegger branches: 1.2.6;
ansify function definitions
 1.1 20-Dec-2005  christos branches: 1.1.36;
Merge libkern + libc common files. As requested by core.
 1.1.36.1 13-May-2009  jym Sync with HEAD.

Commit is split, to avoid a "too many arguments" protocol error.
 1.2.6.2 30-Oct-2012  yamt sync with head
 1.2.6.1 17-Apr-2012  yamt sync with head
 1.4.34.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.4.34.1 21-Apr-2020  martin Sync with HEAD
 1.2 15-Mar-2009  cegger branches: 1.2.48;
ansify function definitions
 1.1 20-Dec-2005  christos branches: 1.1.36;
Merge libkern + libc common files. As requested by core.
 1.1.36.1 13-May-2009  jym Sync with HEAD.

Commit is split, to avoid a "too many arguments" protocol error.
 1.2.48.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.2.48.1 21-Apr-2020  martin Sync with HEAD
 1.2 15-Mar-2009  cegger branches: 1.2.48;
ansify function definitions
 1.1 20-Dec-2005  christos branches: 1.1.36;
Merge libkern + libc common files. As requested by core.
 1.1.36.1 13-May-2009  jym Sync with HEAD.

Commit is split, to avoid a "too many arguments" protocol error.
 1.2.48.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.2.48.1 21-Apr-2020  martin Sync with HEAD
 1.2 15-Mar-2009  cegger branches: 1.2.48;
ansify function definitions
 1.1 20-Dec-2005  christos branches: 1.1.36;
Merge libkern + libc common files. As requested by core.
 1.1.36.1 13-May-2009  jym Sync with HEAD.

Commit is split, to avoid a "too many arguments" protocol error.
 1.2.48.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.2.48.1 21-Apr-2020  martin Sync with HEAD
 1.4 07-Dec-2012  christos branches: 1.4.32;
use explicit cast to pacify lint.
 1.3 06-Aug-2012  matt branches: 1.3.2;
Add ARM EABI aliases to a few functions.
 1.2 15-Mar-2009  cegger branches: 1.2.6;
ansify function definitions
 1.1 20-Dec-2005  christos branches: 1.1.36;
Merge libkern + libc common files. As requested by core.
 1.1.36.1 13-May-2009  jym Sync with HEAD.

Commit is split, to avoid a "too many arguments" protocol error.
 1.2.6.2 16-Jan-2013  yamt sync with (a bit old) head
 1.2.6.1 30-Oct-2012  yamt sync with head
 1.3.2.1 25-Feb-2013  tls resync with head
 1.4.32.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.4.32.1 21-Apr-2020  martin Sync with HEAD
 1.4 06-Aug-2012  matt branches: 1.4.34;
Add ARM EABI aliases to a few functions.
 1.3 09-Mar-2012  christos Casts and type changes to fix portability issues.
- int -> size_t
- adjust width of RHS of shift
- adjust widths of types
 1.2 15-Mar-2009  cegger branches: 1.2.6;
ansify function definitions
 1.1 20-Dec-2005  christos branches: 1.1.36;
Merge libkern + libc common files. As requested by core.
 1.1.36.1 13-May-2009  jym Sync with HEAD.

Commit is split, to avoid a "too many arguments" protocol error.
 1.2.6.2 30-Oct-2012  yamt sync with head
 1.2.6.1 17-Apr-2012  yamt sync with head
 1.4.34.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.4.34.1 21-Apr-2020  martin Sync with HEAD
 1.2 15-Mar-2009  cegger branches: 1.2.48;
ansify function definitions
 1.1 20-Dec-2005  christos branches: 1.1.36;
Merge libkern + libc common files. As requested by core.
 1.1.36.1 13-May-2009  jym Sync with HEAD.

Commit is split, to avoid a "too many arguments" protocol error.
 1.2.48.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.2.48.1 21-Apr-2020  martin Sync with HEAD
 1.3 06-Aug-2012  matt branches: 1.3.34;
Add ARM EABI aliases to a few functions.
 1.2 15-Mar-2009  cegger branches: 1.2.6;
ansify function definitions
 1.1 20-Dec-2005  christos branches: 1.1.36;
Merge libkern + libc common files. As requested by core.
 1.1.36.1 13-May-2009  jym Sync with HEAD.

Commit is split, to avoid a "too many arguments" protocol error.
 1.2.6.1 30-Oct-2012  yamt sync with head
 1.3.34.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.3.34.1 21-Apr-2020  martin Sync with HEAD
 1.2 15-Mar-2009  cegger branches: 1.2.48;
ansify function definitions
 1.1 20-Dec-2005  christos branches: 1.1.36;
Merge libkern + libc common files. As requested by core.
 1.1.36.1 13-May-2009  jym Sync with HEAD.

Commit is split, to avoid a "too many arguments" protocol error.
 1.2.48.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.2.48.1 21-Apr-2020  martin Sync with HEAD
 1.2 15-Mar-2009  cegger branches: 1.2.48;
ansify function definitions
 1.1 20-Dec-2005  christos branches: 1.1.36;
Merge libkern + libc common files. As requested by core.
 1.1.36.1 13-May-2009  jym Sync with HEAD.

Commit is split, to avoid a "too many arguments" protocol error.
 1.2.48.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.2.48.1 21-Apr-2020  martin Sync with HEAD
 1.4 20-Mar-2012  matt branches: 1.4.34;
Remove __P usage.
Convert to C89 prototypes.
 1.3 09-Mar-2012  christos Casts and type changes to fix portability issues.
- int -> size_t
- adjust width of RHS of shift
- adjust widths of types
 1.2 15-Mar-2009  cegger branches: 1.2.6;
ansify function definitions
 1.1 20-Dec-2005  christos branches: 1.1.36;
Merge libkern + libc common files. As requested by core.
 1.1.36.1 13-May-2009  jym Sync with HEAD.

Commit is split, to avoid a "too many arguments" protocol error.
 1.2.6.1 17-Apr-2012  yamt sync with head
 1.4.34.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.4.34.1 21-Apr-2020  martin Sync with HEAD
 1.7 06-Aug-2012  matt branches: 1.7.34;
Add ARM EABI aliases to a few functions.
 1.6 20-Mar-2012  matt Remove __P usage.
Convert to C89 prototypes.
 1.5 09-Mar-2012  christos Casts and type changes to fix portability issues.
- int -> size_t
- adjust width of RHS of shift
- adjust widths of types
 1.4 09-Jul-2011  matt branches: 1.4.2;
Add long double prototypes.
 1.3 04-Jul-2011  mrg add support for __floatunsisf().
 1.2 04-Jul-2011  matt Update to C89 style.
Fix name of floatundidf (uns -> un).
 1.1 20-Dec-2005  christos add files I forgot to commit.
 1.4.2.2 30-Oct-2012  yamt sync with head
 1.4.2.1 17-Apr-2012  yamt sync with head
 1.7.34.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.7.34.1 21-Apr-2020  martin Sync with HEAD
 1.2 15-Mar-2009  cegger branches: 1.2.48;
ansify function definitions
 1.1 20-Dec-2005  christos branches: 1.1.36;
Merge libkern + libc common files. As requested by core.
 1.1.36.1 13-May-2009  jym Sync with HEAD.

Commit is split, to avoid a "too many arguments" protocol error.
 1.2.48.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.2.48.1 21-Apr-2020  martin Sync with HEAD
 1.2 15-Mar-2009  cegger branches: 1.2.48;
ansify function definitions
 1.1 20-Dec-2005  christos branches: 1.1.36;
Merge libkern + libc common files. As requested by core.
 1.1.36.1 13-May-2009  jym Sync with HEAD.

Commit is split, to avoid a "too many arguments" protocol error.
 1.2.48.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.2.48.1 21-Apr-2020  martin Sync with HEAD
 1.2 15-Mar-2009  cegger branches: 1.2.48;
ansify function definitions
 1.1 20-Dec-2005  christos branches: 1.1.36;
Merge libkern + libc common files. As requested by core.
 1.1.36.1 13-May-2009  jym Sync with HEAD.

Commit is split, to avoid a "too many arguments" protocol error.
 1.2.48.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.2.48.1 21-Apr-2020  martin Sync with HEAD
 1.2 15-Mar-2009  cegger branches: 1.2.48;
ansify function definitions
 1.1 20-Dec-2005  christos branches: 1.1.36;
Merge libkern + libc common files. As requested by core.
 1.1.36.1 13-May-2009  jym Sync with HEAD.

Commit is split, to avoid a "too many arguments" protocol error.
 1.2.48.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.2.48.1 21-Apr-2020  martin Sync with HEAD
 1.2 15-Mar-2009  cegger branches: 1.2.48;
ansify function definitions
 1.1 20-Dec-2005  christos branches: 1.1.36;
Merge libkern + libc common files. As requested by core.
 1.1.36.1 13-May-2009  jym Sync with HEAD.

Commit is split, to avoid a "too many arguments" protocol error.
 1.2.48.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.2.48.1 21-Apr-2020  martin Sync with HEAD
 1.4 12-May-2024  msaitoh s/descriminated/discriminated/ in comment.
 1.3 16-Jun-2019  christos get rid of caddr_t
 1.2 05-Jun-2019  hannken branches: 1.2.2;
Make XDR usable from kernel or module.

No user visible changes intended.
 1.1 04-Jun-2019  hannken Move the basic part of XDR to common/include/rpc and common/lib/libc/rpc.

No functional change intended.
 1.2.2.5 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.2.2.4 21-Apr-2020  martin Sync with HEAD
 1.2.2.3 13-Apr-2020  martin Mostly merge changes from HEAD upto 20200411
 1.2.2.2 10-Jun-2019  christos Sync with HEAD
 1.2.2.1 05-Jun-2019  christos file xdr.c was added on branch phil-wifi on 2019-06-10 21:41:07 +0000
 1.3 16-Jun-2019  christos get rid of caddr_t
 1.2 05-Jun-2019  hannken branches: 1.2.2;
Make XDR usable from kernel or module.

No user visible changes intended.
 1.1 04-Jun-2019  hannken Move the basic part of XDR to common/include/rpc and common/lib/libc/rpc.

No functional change intended.
 1.2.2.5 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.2.2.4 21-Apr-2020  martin Sync with HEAD
 1.2.2.3 13-Apr-2020  martin Mostly merge changes from HEAD upto 20200411
 1.2.2.2 10-Jun-2019  christos Sync with HEAD
 1.2.2.1 05-Jun-2019  christos file xdr_array.c was added on branch phil-wifi on 2019-06-10 21:41:07 +0000
 1.2 05-Jun-2019  hannken branches: 1.2.2;
Make XDR usable from kernel or module.

No user visible changes intended.
 1.1 04-Jun-2019  hannken Move the basic part of XDR to common/include/rpc and common/lib/libc/rpc.

No functional change intended.
 1.2.2.4 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.2.2.3 21-Apr-2020  martin Sync with HEAD
 1.2.2.2 10-Jun-2019  christos Sync with HEAD
 1.2.2.1 05-Jun-2019  christos file xdr_mem.c was added on branch phil-wifi on 2019-06-10 21:41:07 +0000
 1.5 24-Jul-2024  kre PR lib/58461 PR lib/58453 portability fixes

Revert previous (1.4) and make the changes properly.

If base is invalid, what gets left in *endptr by strtoimax()
is unspecified (it is not guaranteed to be either nptr or unaltered)
and so cannot (in that case) be used in any way at all.

Since it is hard to determine from some implementations of
strtoimax() whether this happened or not, simply duplicate
the validity test for base here, so we know that error (EINVAL
because base is invalid) cannot occur from strtoimax(). In that
case, if we get an EINVAL from strtoimax we can simply ignore it,
as all it can mean is the (optional in POSIX) case where no
conversion occurred (where strtoi() sets the status to ECANCELED).

Since NetBSD never did that, this all changes nothing here, but
makes strtoi() portable to other environments using a different
version of strtoimax().

NFCI.

No pullups required, nothing has really changed, there never was
a NetBSD bug to fix.
 1.4 21-Jul-2024  christos POSIX allows systems that report EINVAL when no digits are found. On
such systems the only way to differentiate EINVAL and ECANCELED is to
initialized the end pointer to NULL before the call. On EINVAL cases,
strto*max(3) will leave the pointer unmodified, so we'll read back the
original NULL. On ECANCELED cases, strto*max(3) will set it to nptr.
This also prevents UB read of endptr on an unsupported base argument.
(Alejandro Colomar)
 1.3 20-Jan-2024  christos branches: 1.3.2;
PR/57828: Alejandro Colomar: Prioritize test for ERANGE before testing for
fully consuming the string. Adjust strtonum(3) to behave as before. Document
the order of the tests and sync the man pages (I should really autogenerate
one of the two man pages...)
 1.2 18-Jan-2015  christos branches: 1.2.2; 1.2.18; 1.2.20; 1.2.32;
match variable names with man page (Kamil Rytarowski)
 1.1 16-Jan-2015  christos add strto{i,u} from Kamil Rytarowski as discussed in tech-userlevel.
 1.2.32.1 23-Aug-2024  martin Pull up following revision(s) (requested by riastradh in ticket #792):

common/lib/libc/stdlib/_strtoi.h: revision 1.3
lib/libc/stdlib/strtou.3: revision 1.8
lib/libc/stdlib/strtonum.c: revision 1.7
lib/libc/stdlib/Makefile.inc: revision 1.97
lib/libc/stdlib/strtoi.3: revision 1.8
lib/libc/stdlib/strtoi.3: revision 1.9
lib/libc/stdlib/strtou.3: file removal
tests/lib/libc/stdlib/t_strtoi.c: revision 1.3

PR/57828: Alejandro Colomar: Prioritize test for ERANGE before testing for
fully consuming the string. Adjust strtonum(3) to behave as before. Document
the order of the tests and sync the man pages (I should really autogenerate
one of the two man pages...)

generate strtou.3 from strtoi.3, grammar police

Add range tests with trailing characters.
 1.2.20.1 23-Aug-2024  martin Pull up following revision(s) (requested by riastradh in ticket #1870):

common/lib/libc/stdlib/_strtoi.h: revision 1.3
lib/libc/stdlib/strtou.3: revision 1.8
lib/libc/stdlib/strtonum.c: revision 1.7
lib/libc/stdlib/Makefile.inc: revision 1.97
lib/libc/stdlib/strtoi.3: revision 1.8
lib/libc/stdlib/strtoi.3: revision 1.9
lib/libc/stdlib/strtou.3: file removal
tests/lib/libc/stdlib/t_strtoi.c: revision 1.3

PR/57828: Alejandro Colomar: Prioritize test for ERANGE before testing for
fully consuming the string. Adjust strtonum(3) to behave as before. Document
the order of the tests and sync the man pages (I should really autogenerate
one of the two man pages...)
generate strtou.3 from strtoi.3, grammar police

Add range tests with trailing characters.
 1.2.18.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.2.18.1 21-Apr-2020  martin Sync with HEAD
 1.2.2.2 22-Apr-2015  snj Pull up following revision(s) (requested by roy in ticket #648):
common/lib/libc/stdlib/_strtoi.h: revisions 1.1, 1.2
common/lib/libc/stdlib/strtoi.c: revision 1.1
common/lib/libc/stdlib/strtou.c: revision 1.1
distrib/sets/lists/base/ad.aarch64: patch
distrib/sets/lists/base/ad.arm: patch
distrib/sets/lists/base/ad.mips: patch
distrib/sets/lists/base/ad.powerpc: patch
distrib/sets/lists/base/md.amd64: patch
distrib/sets/lists/base/md.sparc64: patch
distrib/sets/lists/base/shl.mi: patch
distrib/sets/lists/comp/mi: revision 1.1939
distrib/sets/lists/debug/ad.aarch64: patch
distrib/sets/lists/debug/ad.arm: patch
distrib/sets/lists/debug/ad.mips: patch
distrib/sets/lists/debug/ad.powerpc: patch
distrib/sets/lists/debug/md.amd64: patch
distrib/sets/lists/debug/md.sparc64: patch
distrib/sets/lists/debug/shl.mi: patch
include/inttypes.h: revision 1.11
lib/libc/shlib_version: patch
lib/libc/stdlib/Makefile.inc: revision 1.84
lib/libc/stdlib/strtol.3: revisions 1.27-1.31
lib/libc/stdlib/strtoul.3: revisions 1.26-1.29
sys/lib/libkern/Makefile.libkern: revision 1.37
sys/lib/libkern/libkern.h: revision 1.117
tools/compat/Makefile: revision 1.73
tools/compat/compat_defs.h: revision 1.101
tools/compat/configure.ac: revision 1.83
tools/compat/configure: revision 1.82
tools/compat/nbtool_config.h.in: revision 1.36
add strto{i,u} from Kamil Rytarowski as discussed in tech-userlevel.
--
strtoi and strtou additions
--
strtoi and strtou for the kernel
--
strtoi and strtou additions
--
strtoi and strtou man pages
--
strto{i,u}
--
regen
--
Remove trailing whitespace.
--
match variable names with man page (Kamil Rytarowski)
--
cleanups from (Kamil Rytarowski)
--
add strtoi strtou (Kamil Rytarowski)
--
PR/49640: Kamil Rytarowski: Improve error printing
--
Use existing month for Dd.
 1.2.2.1 18-Jan-2015  snj file _strtoi.h was added on branch netbsd-7 on 2015-04-22 07:18:57 +0000
 1.3.2.1 02-Aug-2025  perseant Sync with HEAD
 1.11 06-Jul-2017  joerg branches: 1.11.6;
Fix ISO C compliance: strtol of "0xX" should give the largest valid
numeric prefix, which is 0.
 1.10 13-Nov-2015  christos branches: 1.10.8;
comment out 0{b,B} handling.
 1.9 12-Nov-2015  christos Fix capitalization
 1.8 12-Nov-2015  christos Recognize 0[bB] as binary (base 2)
 1.7 17-May-2013  joerg branches: 1.7.6;
Provide explicit LC_C_LOCALE accessor and drop the various NULL checks.
Provide LC_GLOBAL_LOCALE in a way that works with all locale functions.
Merge constant data used by the initial global locale and the C locale.
Drop function call layer for _current_locale() and directly return the
locale reference, not a pointer to it. Use protected access for global
variables, so that libc references can avoid the GOT overhead.
 1.6 26-Apr-2013  joerg Restore standalone strtol/strtoul for citrus for now.
 1.5 16-Apr-2013  joerg Add strtol_l and friends. Switch _citrus_bcs_strtol to use plain
strtol_l unless in tools mode. Add note to retire the BCS code on the
next libc major bump.
 1.4 16-Apr-2013  joerg Do not use isalpha here, since we explicitly only support the Portable
Character Set as base and in theory a locale could define a ASCII
control character as letter, resulting in negations. Also avoid isdigit
here to give the compiler a better chance of deciding whether an
unsigned compare or a jump table is a better option, both are very
likely better choices than the memory indirection.
 1.3 09-Mar-2012  christos branches: 1.3.2;
Casts and type changes to fix portability issues.
- int -> size_t
- adjust width of RHS of shift
- adjust widths of types
 1.2 20-May-2009  christos branches: 1.2.6; 1.2.8;
Set endptr if the base is not supported. The opengroup spec does not special
case this condition. Note: glibc has the same problem.
 1.1 20-Aug-2008  joerg branches: 1.1.2; 1.1.8;
Unify the implementation of strto{l,ul,ll,ull,imax,umax,q,uq} into one
version for signed and one version for unsigned data types.

Add a check for supported bases and set errno (userland) or panic
(kernel, libsa) otherwise.

Make strto{ll,ull,imax,umax} normal symbols and just keep the underscore
versions as strong alias.

Obtained from DragonFly, based on the wide char version from Citrus.
Reviewed by christos@
 1.1.8.1 23-Jul-2009  jym Sync with HEAD.
 1.1.2.2 18-Sep-2008  wrstuden Sync with wrstuden-revivesa-base-2.
 1.1.2.1 20-Aug-2008  wrstuden file _strtol.h was added on branch wrstuden-revivesa on 2008-09-18 04:54:18 +0000
 1.2.8.1 26-Aug-2017  snj Pull up following revision(s) (requested by joerg in ticket #1460):
common/lib/libc/stdlib/_strtol.h: 1.11 via patch
common/lib/libc/stdlib/_strtoul.h: 1.11 via patch
tests/lib/libc/stdlib/t_strtol.c: 1.6-1.7
Fix testing of returned entptr, and fix three affected tests.
From kamil@ via PR lib/49632
--
Fix ISO C compliance: strtol of "0xX" should give the largest valid
numeric prefix, which is 0.
 1.2.6.2 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.2.6.1 17-Apr-2012  yamt sync with head
 1.3.2.1 23-Jun-2013  tls resync from head
 1.7.6.1 17-Nov-2017  snj Pull up following revision(s) (requested by joerg in ticket #1448):
common/lib/libc/stdlib/_strtol.h: revision 1.11
common/lib/libc/stdlib/_strtoul.h: revision 1.11
tests/lib/libc/stdlib/t_strtol.c: revision 1.7
Fix ISO C compliance: strtol of "0xX" should give the largest valid
numeric prefix, which is 0.
 1.10.8.1 10-Jul-2017  martin Pull up following revision(s) (requested by joerg in ticket #112):
common/lib/libc/stdlib/_strtol.h: revision 1.11
common/lib/libc/stdlib/_strtoul.h: revision 1.11
tests/lib/libc/stdlib/t_strtol.c: revision 1.7
Fix ISO C compliance: strtol of "0xX" should give the largest valid
numeric prefix, which is 0.
 1.11.6.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.11.6.1 21-Apr-2020  martin Sync with HEAD
 1.11 06-Jul-2017  joerg branches: 1.11.6;
Fix ISO C compliance: strtol of "0xX" should give the largest valid
numeric prefix, which is 0.
 1.10 05-Nov-2016  riastradh branches: 1.10.6;
Set *endptr in all paths out of strtoul and family.

XXX pullup-6
XXX pullup-7
 1.9 13-Nov-2015  christos branches: 1.9.2;
comment out 0{b,B} handling.
 1.8 12-Nov-2015  christos Recognize 0[bB] as binary (base 2)
 1.7 17-May-2013  joerg branches: 1.7.6; 1.7.8; 1.7.10;
Provide explicit LC_C_LOCALE accessor and drop the various NULL checks.
Provide LC_GLOBAL_LOCALE in a way that works with all locale functions.
Merge constant data used by the initial global locale and the C locale.
Drop function call layer for _current_locale() and directly return the
locale reference, not a pointer to it. Use protected access for global
variables, so that libc references can avoid the GOT overhead.
 1.6 26-Apr-2013  joerg Restore standalone strtol/strtoul for citrus for now.
 1.5 16-Apr-2013  joerg Add strtol_l and friends. Switch _citrus_bcs_strtol to use plain
strtol_l unless in tools mode. Add note to retire the BCS code on the
next libc major bump.
 1.4 16-Apr-2013  joerg Do not use isalpha here, since we explicitly only support the Portable
Character Set as base and in theory a locale could define a ASCII
control character as letter, resulting in negations. Also avoid isdigit
here to give the compiler a better chance of deciding whether an
unsigned compare or a jump table is a better option, both are very
likely better choices than the memory indirection.
 1.3 22-Mar-2012  christos branches: 1.3.2;
bring the casts to the operands, not the operation results.
 1.2 09-Mar-2012  christos Casts and type changes to fix portability issues.
- int -> size_t
- adjust width of RHS of shift
- adjust widths of types
 1.1 20-Aug-2008  joerg branches: 1.1.2; 1.1.4; 1.1.22; 1.1.26; 1.1.30;
Unify the implementation of strto{l,ul,ll,ull,imax,umax,q,uq} into one
version for signed and one version for unsigned data types.

Add a check for supported bases and set errno (userland) or panic
(kernel, libsa) otherwise.

Make strto{ll,ull,imax,umax} normal symbols and just keep the underscore
versions as strong alias.

Obtained from DragonFly, based on the wide char version from Citrus.
Reviewed by christos@
 1.1.30.1 11-Jul-2017  snj Pull up following revision(s) (requested by riastradh in ticket #1421):
common/lib/libc/stdlib/_strtoul.h: revision 1.10
Set *endptr in all paths out of strtoul and family.
 1.1.26.1 11-Jul-2017  snj Pull up following revision(s) (requested by riastradh in ticket #1421):
common/lib/libc/stdlib/_strtoul.h: revision 1.10
Set *endptr in all paths out of strtoul and family.
 1.1.22.2 26-Aug-2017  snj Pull up following revision(s) (requested by joerg in ticket #1460):
common/lib/libc/stdlib/_strtol.h: 1.11 via patch
common/lib/libc/stdlib/_strtoul.h: 1.11 via patch
tests/lib/libc/stdlib/t_strtol.c: 1.6-1.7
Fix testing of returned entptr, and fix three affected tests.
From kamil@ via PR lib/49632
--
Fix ISO C compliance: strtol of "0xX" should give the largest valid
numeric prefix, which is 0.
 1.1.22.1 11-Jul-2017  snj Pull up following revision(s) (requested by riastradh in ticket #1421):
common/lib/libc/stdlib/_strtoul.h: revision 1.10
Set *endptr in all paths out of strtoul and family.
 1.1.4.2 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.1.4.1 17-Apr-2012  yamt sync with head
 1.1.2.2 18-Sep-2008  wrstuden Sync with wrstuden-revivesa-base-2.
 1.1.2.1 20-Aug-2008  wrstuden file _strtoul.h was added on branch wrstuden-revivesa on 2008-09-18 04:54:18 +0000
 1.3.2.1 23-Jun-2013  tls resync from head
 1.7.10.1 18-Jan-2017  skrll Sync with netbsd-5
 1.7.8.1 20-Dec-2016  snj Pull up following revision(s) (requested by riastradh in ticket #1310):
common/lib/libc/stdlib/_strtoul.h: revision 1.10
Set *endptr in all paths out of strtoul and family.
 1.7.6.2 17-Nov-2017  snj Pull up following revision(s) (requested by joerg in ticket #1448):
common/lib/libc/stdlib/_strtol.h: revision 1.11
common/lib/libc/stdlib/_strtoul.h: revision 1.11
tests/lib/libc/stdlib/t_strtol.c: revision 1.7
Fix ISO C compliance: strtol of "0xX" should give the largest valid
numeric prefix, which is 0.
 1.7.6.1 18-Dec-2016  snj Pull up following revision(s) (requested by riastradh in ticket #1310):
common/lib/libc/stdlib/_strtoul.h: revision 1.10
Set *endptr in all paths out of strtoul and family.
XXX pullup-6
XXX pullup-7
 1.9.2.1 07-Jan-2017  pgoyette Sync with HEAD. (Note that most of these changes are simply $NetBSD$
tag issues.)
 1.10.6.1 10-Jul-2017  martin Pull up following revision(s) (requested by joerg in ticket #112):
common/lib/libc/stdlib/_strtol.h: revision 1.11
common/lib/libc/stdlib/_strtoul.h: revision 1.11
tests/lib/libc/stdlib/t_strtol.c: revision 1.7
Fix ISO C compliance: strtol of "0xX" should give the largest valid
numeric prefix, which is 0.
 1.11.6.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.11.6.1 21-Apr-2020  martin Sync with HEAD
 1.2 29-Jun-2024  rin getopt(): Make this built for _KERNEL || _STANDALONE

NFC as a libc routine.
 1.1 29-Jun-2024  rin getopt.c: Migrate from lib/libc/stdlib to common/lib/libc/stdlib
 1.4 02-Mar-2025  riastradh libc: New _r variants of heapsort, mergesort, qsort.

Also kheapsort_r for kernel/standalone use.

These variants allow the caller to pass a cookie through to the
comparison function, e.g. if you want to sort an array of indices
into a buffer.

qsort_r is new in POSIX.1-2024; the others are obvious analogues of
our nonstandard extensions for heapsort and mergesort.

PR lib/58931: qsort_r() missing
 1.3 17-Nov-2008  jnemeth branches: 1.3.50; 1.3.70;
oops, converted some #ifndef into #if defined...
 1.2 17-Nov-2008  jnemeth Convert all #ifdef _KERNEL to #if defined (_KERNEL) || defined(_STANDALONE)
from Bernd Ernesti on source-changes@. This fixes a build breakage
with i386/amd64 bootblocks.
 1.1 16-Nov-2008  ad Our qsort() is inappropriate for kernel use because it makes recursive
calls. Replace it with a kheapsort() function in kernel. Pointed out
by tron@.
 1.3.70.1 02-Aug-2025  perseant Sync with HEAD
 1.3.50.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.3.50.1 21-Apr-2020  martin Sync with HEAD
 1.1 11-Dec-2013  joerg branches: 1.1.4; 1.1.8; 1.1.30;
Allow kernel code to access constant databases by moving cdbr(3) and the
required mi_vector_hash(3) into src/common.
 1.1.30.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.1.30.1 21-Apr-2020  martin Sync with HEAD
 1.1.8.2 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.1.8.1 11-Dec-2013  tls file mi_vector_hash.c was added on branch tls-maxphys on 2014-08-19 23:45:14 +0000
 1.1.4.2 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.1.4.1 11-Dec-2013  yamt file mi_vector_hash.c was added on branch yamt-pagecache on 2014-05-22 11:26:30 +0000
 1.2 16-Nov-2008  ad Our qsort() is inappropriate for kernel use because it makes recursive
calls. Replace it with a kheapsort() function in kernel. Pointed out
by tron@.
 1.1 16-Nov-2008  ad Make qsort() available in libkern.
 1.7 12-Dec-2021  andvar fix various typos in comments.
 1.6 22-Feb-2020  fox common/lib/libc/stdlib: Fix possible signed integer overflow.

common/lib/libc/stdlib/random.c:482:6 can result in signed integer overflow.

This bug was reported by UBSan runs.

The change has been tested using the following program to generate random numbers
in both the old and the new library and can be used to verify the correctness of the
library after the change.

#include <stdio.h>
#include <stdlib.h>

#define COUNT 1000 * 1000

int
main(void)
{
int i;
FILE *fp = fopen("numbers.txt", "w");

srandom(0xdeadbeef);

for(i = 0; i < COUNT; i++) {
fprintf(fp, "%ld\n", random());
}

fclose(fp);

return 0;
}

Reviewed by: riastradh@ , kamil@
 1.5 08-Feb-2016  dholland branches: 1.5.16; 1.5.20;
whitespace
 1.4 12-Jun-2014  christos update for the new function signatures of initstate/srandom
 1.3 21-Dec-2005  christos branches: 1.3.56; 1.3.70;
Add the old small random from the libkern and use it for now.
 1.2 21-Dec-2005  christos Make this compile for the kernel again.
 1.1 20-Dec-2005  christos Merge libkern + libc common files. As requested by core.
 1.3.70.1 10-Aug-2014  tls Rebase.
 1.3.56.1 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.5.20.1 29-Feb-2020  ad Sync with head.
 1.5.16.3 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.5.16.2 21-Apr-2020  martin Sync with HEAD
 1.5.16.1 08-Apr-2020  martin Merge changes from current as of 20200406
 1.3 28-Nov-2019  roy Make it easier to use strtoi and strtou in downsteam applications
without the need to define HAVE_NBTOOL_CONFIG_H and yet allow -Wundef
not to log any warnings.
 1.2 01-May-2015  christos branches: 1.2.16;
- new test for strtoi
- namespace protection for strto{i,u}
- separate manpages for strto{i,u} from the ones for strto{u,}l
From: Kamil Rytarowski
 1.1 16-Jan-2015  christos branches: 1.1.2;
add strto{i,u} from Kamil Rytarowski as discussed in tech-userlevel.
 1.1.2.3 16-May-2015  snj Pull up following revision(s) (requested by christos in ticket #781):
common/lib/libc/stdlib/strtoi.c: revision 1.2
common/lib/libc/stdlib/strtou.c: revision 1.2
distrib/sets/lists/debug/mi: revision 1.113
distrib/sets/lists/tests/mi: revision 1.617
lib/libc/include/namespace.h: revision 1.178
lib/libc/stdlib/Makefile.inc: revision 1.90 via patch
lib/libc/stdlib/strtoi.3: revisions 1.1, 1.2
lib/libc/stdlib/strtol.3: revision 1.32
lib/libc/stdlib/strtou.3: revisions 1.1, 1.2
lib/libc/stdlib/strtoul.3: revisions 1.30, 1.31
tests/lib/libc/stdlib/Makefile: revision 1.25
tests/lib/libc/stdlib/t_strtoi.c: revision 1.1
- new test for strtoi
- namespace protection for strto{i,u}
- separate manpages for strto{i,u} from the ones for strto{u,}l
From: Kamil Rytarowski
--
Sort ERRORS and SEE ALSO.
--
Sort ERRORS and SEE ALSO.
--
Sort SEE ALSO.
 1.1.2.2 22-Apr-2015  snj Pull up following revision(s) (requested by roy in ticket #648):
common/lib/libc/stdlib/_strtoi.h: revisions 1.1, 1.2
common/lib/libc/stdlib/strtoi.c: revision 1.1
common/lib/libc/stdlib/strtou.c: revision 1.1
distrib/sets/lists/base/ad.aarch64: patch
distrib/sets/lists/base/ad.arm: patch
distrib/sets/lists/base/ad.mips: patch
distrib/sets/lists/base/ad.powerpc: patch
distrib/sets/lists/base/md.amd64: patch
distrib/sets/lists/base/md.sparc64: patch
distrib/sets/lists/base/shl.mi: patch
distrib/sets/lists/comp/mi: revision 1.1939
distrib/sets/lists/debug/ad.aarch64: patch
distrib/sets/lists/debug/ad.arm: patch
distrib/sets/lists/debug/ad.mips: patch
distrib/sets/lists/debug/ad.powerpc: patch
distrib/sets/lists/debug/md.amd64: patch
distrib/sets/lists/debug/md.sparc64: patch
distrib/sets/lists/debug/shl.mi: patch
include/inttypes.h: revision 1.11
lib/libc/shlib_version: patch
lib/libc/stdlib/Makefile.inc: revision 1.84
lib/libc/stdlib/strtol.3: revisions 1.27-1.31
lib/libc/stdlib/strtoul.3: revisions 1.26-1.29
sys/lib/libkern/Makefile.libkern: revision 1.37
sys/lib/libkern/libkern.h: revision 1.117
tools/compat/Makefile: revision 1.73
tools/compat/compat_defs.h: revision 1.101
tools/compat/configure.ac: revision 1.83
tools/compat/configure: revision 1.82
tools/compat/nbtool_config.h.in: revision 1.36
add strto{i,u} from Kamil Rytarowski as discussed in tech-userlevel.
--
strtoi and strtou additions
--
strtoi and strtou for the kernel
--
strtoi and strtou additions
--
strtoi and strtou man pages
--
strto{i,u}
--
regen
--
Remove trailing whitespace.
--
match variable names with man page (Kamil Rytarowski)
--
cleanups from (Kamil Rytarowski)
--
add strtoi strtou (Kamil Rytarowski)
--
PR/49640: Kamil Rytarowski: Improve error printing
--
Use existing month for Dd.
 1.1.2.1 16-Jan-2015  snj file strtoi.c was added on branch netbsd-7 on 2015-04-22 07:18:57 +0000
 1.2.16.3 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.2.16.2 21-Apr-2020  martin Sync with HEAD
 1.2.16.1 08-Apr-2020  martin Merge changes from current as of 20200406
 1.2 02-Dec-2013  joerg branches: 1.2.4; 1.2.8; 1.2.30;
Fix aliases.
 1.1 02-Dec-2013  lneto changed lua_Number to int64_t
 1.2.30.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.2.30.1 21-Apr-2020  martin Sync with HEAD
 1.2.8.2 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.2.8.1 02-Dec-2013  tls file strtoimax.c was added on branch tls-maxphys on 2014-08-19 23:45:14 +0000
 1.2.4.2 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.2.4.1 02-Dec-2013  yamt file strtoimax.c was added on branch yamt-pagecache on 2014-05-22 11:26:30 +0000
 1.7 16-Apr-2013  joerg branches: 1.7.30;
Add strtol_l and friends. Switch _citrus_bcs_strtol to use plain
strtol_l unless in tools mode. Add note to retire the BCS code on the
next libc major bump.
 1.6 22-Aug-2008  matt branches: 1.6.2; 1.6.24;
Properly deal with LIBC namespace issues.
 1.5 22-Aug-2008  dogcow Unbork cross-builds in tools/compat/
 1.4 20-Aug-2008  oster Fix build breakage by adding appropriate #includes.
Patch from joerg@
 1.3 20-Aug-2008  joerg Unify the implementation of strto{l,ul,ll,ull,imax,umax,q,uq} into one
version for signed and one version for unsigned data types.

Add a check for supported bases and set errno (userland) or panic
(kernel, libsa) otherwise.

Make strto{ll,ull,imax,umax} normal symbols and just keep the underscore
versions as strong alias.

Obtained from DragonFly, based on the wide char version from Citrus.
Reviewed by christos@
 1.2 18-Dec-2006  christos branches: 1.2.18;
remove bogus (void)&var; from Anon Ymous
 1.1 08-Oct-2006  thorpej branches: 1.1.2;
- Move strtoll.c and strtoull.c from libc/stdlib to common/libc/stdlib.
- Add strtoll.c and strtoull.c to libkern.
 1.1.2.2 08-Oct-2006  thorpej - Move strtoll.c and strtoull.c from libc/stdlib to common/libc/stdlib.
- Add strtoll.c and strtoull.c to libkern.
 1.1.2.1 08-Oct-2006  thorpej file strtoll.c was added on branch newlock2 on 2006-10-08 03:14:56 +0000
 1.2.18.1 18-Sep-2008  wrstuden Sync with wrstuden-revivesa-base-2.
 1.6.24.1 23-Jun-2013  tls resync from head
 1.6.2.1 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.7.30.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.7.30.1 21-Apr-2020  martin Sync with HEAD
 1.3 28-Nov-2019  roy Make it easier to use strtoi and strtou in downsteam applications
without the need to define HAVE_NBTOOL_CONFIG_H and yet allow -Wundef
not to log any warnings.
 1.2 01-May-2015  christos branches: 1.2.16;
- new test for strtoi
- namespace protection for strto{i,u}
- separate manpages for strto{i,u} from the ones for strto{u,}l
From: Kamil Rytarowski
 1.1 16-Jan-2015  christos branches: 1.1.2;
add strto{i,u} from Kamil Rytarowski as discussed in tech-userlevel.
 1.1.2.3 16-May-2015  snj Pull up following revision(s) (requested by christos in ticket #781):
common/lib/libc/stdlib/strtoi.c: revision 1.2
common/lib/libc/stdlib/strtou.c: revision 1.2
distrib/sets/lists/debug/mi: revision 1.113
distrib/sets/lists/tests/mi: revision 1.617
lib/libc/include/namespace.h: revision 1.178
lib/libc/stdlib/Makefile.inc: revision 1.90 via patch
lib/libc/stdlib/strtoi.3: revisions 1.1, 1.2
lib/libc/stdlib/strtol.3: revision 1.32
lib/libc/stdlib/strtou.3: revisions 1.1, 1.2
lib/libc/stdlib/strtoul.3: revisions 1.30, 1.31
tests/lib/libc/stdlib/Makefile: revision 1.25
tests/lib/libc/stdlib/t_strtoi.c: revision 1.1
- new test for strtoi
- namespace protection for strto{i,u}
- separate manpages for strto{i,u} from the ones for strto{u,}l
From: Kamil Rytarowski
--
Sort ERRORS and SEE ALSO.
--
Sort ERRORS and SEE ALSO.
--
Sort SEE ALSO.
 1.1.2.2 22-Apr-2015  snj Pull up following revision(s) (requested by roy in ticket #648):
common/lib/libc/stdlib/_strtoi.h: revisions 1.1, 1.2
common/lib/libc/stdlib/strtoi.c: revision 1.1
common/lib/libc/stdlib/strtou.c: revision 1.1
distrib/sets/lists/base/ad.aarch64: patch
distrib/sets/lists/base/ad.arm: patch
distrib/sets/lists/base/ad.mips: patch
distrib/sets/lists/base/ad.powerpc: patch
distrib/sets/lists/base/md.amd64: patch
distrib/sets/lists/base/md.sparc64: patch
distrib/sets/lists/base/shl.mi: patch
distrib/sets/lists/comp/mi: revision 1.1939
distrib/sets/lists/debug/ad.aarch64: patch
distrib/sets/lists/debug/ad.arm: patch
distrib/sets/lists/debug/ad.mips: patch
distrib/sets/lists/debug/ad.powerpc: patch
distrib/sets/lists/debug/md.amd64: patch
distrib/sets/lists/debug/md.sparc64: patch
distrib/sets/lists/debug/shl.mi: patch
include/inttypes.h: revision 1.11
lib/libc/shlib_version: patch
lib/libc/stdlib/Makefile.inc: revision 1.84
lib/libc/stdlib/strtol.3: revisions 1.27-1.31
lib/libc/stdlib/strtoul.3: revisions 1.26-1.29
sys/lib/libkern/Makefile.libkern: revision 1.37
sys/lib/libkern/libkern.h: revision 1.117
tools/compat/Makefile: revision 1.73
tools/compat/compat_defs.h: revision 1.101
tools/compat/configure.ac: revision 1.83
tools/compat/configure: revision 1.82
tools/compat/nbtool_config.h.in: revision 1.36
add strto{i,u} from Kamil Rytarowski as discussed in tech-userlevel.
--
strtoi and strtou additions
--
strtoi and strtou for the kernel
--
strtoi and strtou additions
--
strtoi and strtou man pages
--
strto{i,u}
--
regen
--
Remove trailing whitespace.
--
match variable names with man page (Kamil Rytarowski)
--
cleanups from (Kamil Rytarowski)
--
add strtoi strtou (Kamil Rytarowski)
--
PR/49640: Kamil Rytarowski: Improve error printing
--
Use existing month for Dd.
 1.1.2.1 16-Jan-2015  snj file strtou.c was added on branch netbsd-7 on 2015-04-22 07:18:57 +0000
 1.2.16.3 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.2.16.2 21-Apr-2020  martin Sync with HEAD
 1.2.16.1 08-Apr-2020  martin Merge changes from current as of 20200406
 1.3 20-Aug-2008  oster branches: 1.3.62;
Fix build breakage by adding appropriate #includes.
Patch from joerg@
 1.2 20-Aug-2008  joerg Unify the implementation of strto{l,ul,ll,ull,imax,umax,q,uq} into one
version for signed and one version for unsigned data types.

Add a check for supported bases and set errno (userland) or panic
(kernel, libsa) otherwise.

Make strto{ll,ull,imax,umax} normal symbols and just keep the underscore
versions as strong alias.

Obtained from DragonFly, based on the wide char version from Citrus.
Reviewed by christos@
 1.1 20-Dec-2005  christos branches: 1.1.28;
Merge libkern + libc common files. As requested by core.
 1.1.28.1 18-Sep-2008  wrstuden Sync with wrstuden-revivesa-base-2.
 1.3.62.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.3.62.1 21-Apr-2020  martin Sync with HEAD
 1.6 16-Apr-2013  joerg branches: 1.6.30;
Add strtol_l and friends. Switch _citrus_bcs_strtol to use plain
strtol_l unless in tools mode. Add note to retire the BCS code on the
next libc major bump.
 1.5 10-Sep-2008  joerg branches: 1.5.2; 1.5.24;
Prepare use of strtoull, strtoumax and strtoimax in tools/compat by
including nbtool_config.h.
 1.4 22-Aug-2008  matt Properly deal with LIBC namespace issues.
 1.3 20-Aug-2008  oster Fix build breakage by adding appropriate #includes.
Patch from joerg@
 1.2 20-Aug-2008  joerg Unify the implementation of strto{l,ul,ll,ull,imax,umax,q,uq} into one
version for signed and one version for unsigned data types.

Add a check for supported bases and set errno (userland) or panic
(kernel, libsa) otherwise.

Make strto{ll,ull,imax,umax} normal symbols and just keep the underscore
versions as strong alias.

Obtained from DragonFly, based on the wide char version from Citrus.
Reviewed by christos@
 1.1 08-Oct-2006  thorpej branches: 1.1.2; 1.1.26;
- Move strtoll.c and strtoull.c from libc/stdlib to common/libc/stdlib.
- Add strtoll.c and strtoull.c to libkern.
 1.1.26.2 24-Sep-2008  wrstuden Merge in changes between wrstuden-revivesa-base-2 and
wrstuden-revivesa-base-3.
 1.1.26.1 18-Sep-2008  wrstuden Sync with wrstuden-revivesa-base-2.
 1.1.2.2 08-Oct-2006  thorpej - Move strtoll.c and strtoull.c from libc/stdlib to common/libc/stdlib.
- Add strtoll.c and strtoull.c to libkern.
 1.1.2.1 08-Oct-2006  thorpej file strtoull.c was added on branch newlock2 on 2006-10-08 03:14:56 +0000
 1.5.24.1 23-Jun-2013  tls resync from head
 1.5.2.1 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.6.30.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.6.30.1 21-Apr-2020  martin Sync with HEAD
 1.6 16-Apr-2013  joerg branches: 1.6.30;
Add strtol_l and friends. Switch _citrus_bcs_strtol to use plain
strtol_l unless in tools mode. Add note to retire the BCS code on the
next libc major bump.
 1.5 10-Sep-2008  joerg branches: 1.5.2; 1.5.24;
Prepare use of strtoull, strtoumax and strtoimax in tools/compat by
including nbtool_config.h.
 1.4 22-Aug-2008  matt Properly deal with LIBC namespace issues.
 1.3 20-Aug-2008  oster Fix build breakage by adding appropriate #includes.
Patch from joerg@
 1.2 20-Aug-2008  joerg Unify the implementation of strto{l,ul,ll,ull,imax,umax,q,uq} into one
version for signed and one version for unsigned data types.

Add a check for supported bases and set errno (userland) or panic
(kernel, libsa) otherwise.

Make strto{ll,ull,imax,umax} normal symbols and just keep the underscore
versions as strong alias.

Obtained from DragonFly, based on the wide char version from Citrus.
Reviewed by christos@
 1.1 22-Apr-2006  thorpej branches: 1.1.28;
Move strtoumax.c from libc/stdlib to common/libc/stdlib and include it
in libkern. Required for new code coming soon.
 1.1.28.2 24-Sep-2008  wrstuden Merge in changes between wrstuden-revivesa-base-2 and
wrstuden-revivesa-base-3.
 1.1.28.1 18-Sep-2008  wrstuden Sync with wrstuden-revivesa-base-2.
 1.5.24.1 23-Jun-2013  tls resync from head
 1.5.2.1 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.6.30.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.6.30.1 21-Apr-2020  martin Sync with HEAD
 1.10 29-Jan-2020  ad Some boot blocks too big now, only compare in big chunks if !_STANDALONE.
 1.9 27-Jan-2020  ad Drop the alignment check if __NO_STRICT_ALIGNMENT (x86, m68k, vax).
 1.8 27-Jan-2020  ad bcmp() / memcmp(): compare in uintptr_t sized chunks when it's easy to.
 1.7 09-Mar-2012  christos branches: 1.7.34; 1.7.38;
Casts and type changes to fix portability issues.
- int -> size_t
- adjust width of RHS of shift
- adjust widths of types
 1.6 25-Apr-2009  skrll branches: 1.6.6;
Only include <assert.h> and <string.h> if !_KERNEL && !_STANDALONE
 1.5 19-Mar-2009  he Actually... We need libkern.h also under _STANDALONE, so that
_DIAGASSERT gets defined. Fixes the mvmeppc libsa build.
 1.4 18-Mar-2009  tsutsui - remove bcmp(9), bcopy(9), and bzero(9) from libkern since <sys/systm.h> has
macro which replace them with mem*() functions in #ifdef _KERNEL as noted
in man pages
- move declarations of bcopy(3) and bzero(3) into <lib/libsa/stand.h>
since they are still in libsa for some MD standalone sources
(I guess all bcmp(3) in standalone sources have been replaced with memcmp(3)
but they should be replaced with memcmp() anyway)
 1.3 17-Mar-2009  he Add local declarations of bcmp() and bzero() because libkern.h no longer
provides those declarations.

It's possible that these can be removed later when we find out whether
these functions are actually needed in the kernel, but meanwhile, this
allows alpha, hp700 and the powerpc ports (who all build these as part
of libkern) to complete their builds.
 1.2 04-Jun-2007  christos branches: 1.2.20;
handle fortify, ansify.
 1.1 20-Dec-2005  christos Merge libkern + libc common files. As requested by core.
 1.2.20.1 13-May-2009  jym Sync with HEAD.

Commit is split, to avoid a "too many arguments" protocol error.
 1.6.6.1 17-Apr-2012  yamt sync with head
 1.7.38.1 29-Feb-2020  ad Sync with head.
 1.7.34.3 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.7.34.2 21-Apr-2020  martin Sync with HEAD
 1.7.34.1 08-Apr-2020  martin Merge changes from current as of 20200406
 1.13 12-Feb-2018  martin branches: 1.13.4;
Complete previous by complteley removing the _DIAGASSERT from memmove -
the accidental left over from previous fired on all legitimate calls
and caused PR bin/52986 and PR lib/52987.
 1.12 04-Feb-2018  mrg updates for GCC 6.4:

- remove many _DIAGASSERT() checks against not NULL for functions
with arguments with nonnull attributes. (probably more to come,
the set between x86 and sparc us disjoint.)

- port libsanitizer's GetPcSpBp() to sparc, sparc64 and amd64.
 1.11 16-Apr-2014  joerg Undef possible macro versions of memcpy and friends for kernel builds
too. libkern sometimes defines them to the builtins.
 1.10 14-Apr-2014  joerg Using bcopy/memcpy with NULL arguments is valid as long as the size is
also 0.
 1.9 18-Mar-2009  tsutsui branches: 1.9.6; 1.9.12; 1.9.24;
- remove bcmp(9), bcopy(9), and bzero(9) from libkern since <sys/systm.h> has
macro which replace them with mem*() functions in #ifdef _KERNEL as noted
in man pages
- move declarations of bcopy(3) and bzero(3) into <lib/libsa/stand.h>
since they are still in libsa for some MD standalone sources
(I guess all bcmp(3) in standalone sources have been replaced with memcmp(3)
but they should be replaced with memcmp() anyway)
 1.8 26-Mar-2008  apb branches: 1.8.10;
Add /* __OPTIMIZE_SIZE__ */ comments on relevant #else and #endif lines.
Also change some #ifdef to #if defined() for consistency.
 1.7 25-Mar-2008  christos bcopy does not return ptr.
 1.6 25-Mar-2008  christos fix #ifdefs for BCOPY
 1.5 25-Mar-2008  christos add bcopy and bzero for the benefit of loadfile.
 1.4 25-Mar-2008  christos add __OPTIMIZE_SIZE__ versions.
 1.3 04-Jun-2007  christos handle fortify, ansify.
 1.2 05-Feb-2006  ross Don't do DIAGASSERT against NULL if _KERNEL This puts us back to
kernel behavior before src/common where 0 was a valid dest addr.

This should fix the macppc (and other OEA ppc) silent-death-on-DIAGNOSTIC
issue, and possibly other DIAGNOSTIC failures.

XXX it might be worth thinking about an API where 0 is allowed in
documented cases and not allowed otherwise.
 1.1 20-Dec-2005  christos Merge libkern + libc common files. As requested by core.
 1.8.10.1 13-May-2009  jym Sync with HEAD.

Commit is split, to avoid a "too many arguments" protocol error.
 1.9.24.1 10-Aug-2014  tls Rebase.
 1.9.12.1 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.9.6.1 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.13.4.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.13.4.1 21-Apr-2020  martin Sync with HEAD
 1.1 20-Dec-2005  christos branches: 1.1.94;
Merge libkern + libc common files. As requested by core.
 1.1.94.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.1.94.1 21-Apr-2020  martin Sync with HEAD
 1.2 14-Dec-2009  matt branches: 1.2.48;
Merge from matt-nb5-mips64
 1.1 17-Aug-2009  matt branches: 1.1.2;
file bzero2.c was initially added on branch matt-nb5-mips64.
 1.1.2.1 17-Aug-2009  matt Add word-load-store optimized version of memset/bzero. On platforms with
64bit registers they can be as efficient as assembly coded version.
 1.2.48.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.2.48.1 21-Apr-2020  martin Sync with HEAD
 1.2 24-Jun-2013  riastradh Replace consttime_bcmp/explicit_bzero by consttime_memequal/explicit_memset.

consttime_memequal is the same as the old consttime_bcmp.
explicit_memset is to memset as explicit_bzero was to bcmp.

Passes amd64 release and i386/ALL, but I'm sure I missed some spots,
so please let me know.
 1.1 30-Aug-2012  drochner branches: 1.1.2; 1.1.4;
Add "consttime_bcmp" and "explicit_bzero" functions for both kernel
abd userland, as proposed on tech-security, with explicit_bzero using
a volatile function pointer as suggested by Alan Barrett.
Both do what the name says. For userland, both are prefixed by "__"
to keep them out of the user namespace.
Change some memset/memcmp uses to the new functions where it makes
sense -- these are just some examples, more to come.
 1.1.4.3 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.1.4.2 30-Oct-2012  yamt sync with head
 1.1.4.1 30-Aug-2012  yamt file consttime_bcmp.c was added on branch yamt-pagecache on 2012-10-30 18:46:14 +0000
 1.1.2.1 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.6 18-Mar-2015  riastradh branches: 1.6.16;
Switch to the suggested constant-time result conversion.

Not hard to find CPU/compiler combinations with branches for `!res'.

While here, make everything unsigned for good measure.
 1.5 24-Jun-2014  drochner branches: 1.5.4;
add a public domain notice
 1.4 28-Aug-2013  riastradh branches: 1.4.2; 1.4.4;
Comment on possible data-dependent branch in `!res'.
 1.3 28-Aug-2013  riastradh Publish explicit_memset and consttime_memequal in userland libc.

Remove the double-underscore from the userland versions, and do the
weak alias dance instead, now that these are public parts of libc.

As discussed on tech-userlevel:

https://mail-index.netbsd.org/tech-userlevel/2013/06/24/msg007843.html
(option 3)
 1.2 28-Aug-2013  riastradh Fix sense of consttime_memequal and update all callers.

Now it returns true (nonzero) to mean equal and false (zero) to mean
inequal, as the name suggests.

As promised on tech-userlevel back in June:

https://mail-index.netbsd.org/tech-userlevel/2013/06/24/msg007843.html
 1.1 24-Jun-2013  riastradh Replace consttime_bcmp/explicit_bzero by consttime_memequal/explicit_memset.

consttime_memequal is the same as the old consttime_bcmp.
explicit_memset is to memset as explicit_bzero was to bcmp.

Passes amd64 release and i386/ALL, but I'm sure I missed some spots,
so please let me know.
 1.4.4.2 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.4.4.1 28-Aug-2013  yamt file consttime_memequal.c was added on branch yamt-pagecache on 2014-05-22 11:26:30 +0000
 1.4.2.1 10-Aug-2014  tls Rebase.
 1.5.4.2 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.5.4.1 24-Jun-2014  tls file consttime_memequal.c was added on branch tls-maxphys on 2014-08-19 23:45:14 +0000
 1.6.16.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.6.16.1 21-Apr-2020  martin Sync with HEAD
 1.2 24-Jun-2013  riastradh Replace consttime_bcmp/explicit_bzero by consttime_memequal/explicit_memset.

consttime_memequal is the same as the old consttime_bcmp.
explicit_memset is to memset as explicit_bzero was to bcmp.

Passes amd64 release and i386/ALL, but I'm sure I missed some spots,
so please let me know.
 1.1 30-Aug-2012  drochner branches: 1.1.2; 1.1.4;
Add "consttime_bcmp" and "explicit_bzero" functions for both kernel
abd userland, as proposed on tech-security, with explicit_bzero using
a volatile function pointer as suggested by Alan Barrett.
Both do what the name says. For userland, both are prefixed by "__"
to keep them out of the user namespace.
Change some memset/memcmp uses to the new functions where it makes
sense -- these are just some examples, more to come.
 1.1.4.3 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.1.4.2 30-Oct-2012  yamt sync with head
 1.1.4.1 30-Aug-2012  yamt file explicit_bzero.c was added on branch yamt-pagecache on 2012-10-30 18:46:14 +0000
 1.1.2.1 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.5 02-Nov-2024  riastradh memset_explicit(3): Add C23 alias for explicit_memset.

PR standards/58607: C23: <string.h> memset_explicit
 1.4 24-Jun-2014  drochner branches: 1.4.4; 1.4.26; 1.4.46;
add a public domain notice
 1.3 28-Aug-2013  riastradh branches: 1.3.2; 1.3.4;
Publish explicit_memset and consttime_memequal in userland libc.

Remove the double-underscore from the userland versions, and do the
weak alias dance instead, now that these are public parts of libc.

As discussed on tech-userlevel:

https://mail-index.netbsd.org/tech-userlevel/2013/06/24/msg007843.html
(option 3)
 1.2 28-Aug-2013  riastradh Make explicit_memset match memset's return value.
 1.1 24-Jun-2013  riastradh Replace consttime_bcmp/explicit_bzero by consttime_memequal/explicit_memset.

consttime_memequal is the same as the old consttime_bcmp.
explicit_memset is to memset as explicit_bzero was to bcmp.

Passes amd64 release and i386/ALL, but I'm sure I missed some spots,
so please let me know.
 1.3.4.2 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.3.4.1 28-Aug-2013  yamt file explicit_memset.c was added on branch yamt-pagecache on 2014-05-22 11:26:30 +0000
 1.3.2.1 10-Aug-2014  tls Rebase.
 1.4.46.1 02-Aug-2025  perseant Sync with HEAD
 1.4.26.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.4.26.1 21-Apr-2020  martin Sync with HEAD
 1.4.4.2 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.4.4.1 24-Jun-2014  tls file explicit_memset.c was added on branch tls-maxphys on 2014-08-19 23:45:14 +0000
 1.4 07-Aug-2011  mrg branches: 1.4.44;
add a weak alias to __ffssi2, needed for hppa gcc 4.5.
 1.3 02-Nov-2007  christos use the ffs32 from bitops.h which is faster
 1.2 04-Jun-2007  christos branches: 1.2.4;
handle fortify, ansify.
 1.1 20-Dec-2005  christos Merge libkern + libc common files. As requested by core.
 1.2.4.1 06-Nov-2007  matt sync with HEAD
 1.4.44.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.4.44.1 21-Apr-2020  martin Sync with HEAD
 1.2 07-Nov-2024  christos branches: 1.2.4;
pass lint
 1.1 01-Nov-2024  riastradh strings.h: Spruce up for POSIX 2024.

1. Add ffsl, ffsll.
2. Hide bcmp, bcopy, bzero, index, rindex for POSIX>=2024.
3. Expose ffs only for NetBSD or POSIX>=2008 with XSI option.
4. Hide popcount* NetBSD extensions for any POSIX.
5. Sprinkle __constfunc on ffs*.

Add tests for ffs/ffsl/ffsll in tests/include/sys/t_bitops next to
ffs32/ffs64 for convenience.

XXX Still missing strcasecmp_l, strncasecmp_l, and locale_t.

PR lib/58802: missing ffsl(), ffsll() functions from POSIX 2024
 1.2.4.2 02-Aug-2025  perseant Sync with HEAD
 1.2.4.1 07-Nov-2024  perseant file ffsl.c was added on branch perseant-exfatfs on 2025-08-02 05:18:34 +0000
 1.1 01-Nov-2024  riastradh branches: 1.1.4;
strings.h: Spruce up for POSIX 2024.

1. Add ffsl, ffsll.
2. Hide bcmp, bcopy, bzero, index, rindex for POSIX>=2024.
3. Expose ffs only for NetBSD or POSIX>=2008 with XSI option.
4. Hide popcount* NetBSD extensions for any POSIX.
5. Sprinkle __constfunc on ffs*.

Add tests for ffs/ffsl/ffsll in tests/include/sys/t_bitops next to
ffs32/ffs64 for convenience.

XXX Still missing strcasecmp_l, strncasecmp_l, and locale_t.

PR lib/58802: missing ffsl(), ffsll() functions from POSIX 2024
 1.1.4.2 02-Aug-2025  perseant Sync with HEAD
 1.1.4.1 01-Nov-2024  perseant file ffsll.c was added on branch perseant-exfatfs on 2025-08-02 05:18:34 +0000
 1.4 04-Feb-2018  mrg branches: 1.4.4;
updates for GCC 6.4:

- remove many _DIAGASSERT() checks against not NULL for functions
with arguments with nonnull attributes. (probably more to come,
the set between x86 and sparc us disjoint.)

- port libsanitizer's GetPcSpBp() to sparc, sparc64 and amd64.
 1.3 08-Jan-2008  martin When passing char (or unsigned char) arguments as int, assign them to a
temporary char (resp. unsigned char) before comparing, to force truncation
to the proper domain. This makes the coresponding regression tests succeed.
 1.2 04-Jun-2007  christos branches: 1.2.4;
handle fortify, ansify.
 1.1 20-Dec-2005  christos Merge libkern + libc common files. As requested by core.
 1.2.4.1 23-Mar-2008  matt sync with HEAD
 1.4.4.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.4.4.1 21-Apr-2020  martin Sync with HEAD
 1.8 29-Jan-2020  ad Some boot blocks too big now, only compare in big chunks if !_STANDALONE.
 1.7 27-Jan-2020  ad Drop the alignment check if __NO_STRICT_ALIGNMENT (x86, m68k, vax).
 1.6 27-Jan-2020  ad bcmp() / memcmp(): compare in uintptr_t sized chunks when it's easy to.
 1.5 04-Feb-2018  mrg branches: 1.5.4; 1.5.8;
updates for GCC 6.4:

- remove many _DIAGASSERT() checks against not NULL for functions
with arguments with nonnull attributes. (probably more to come,
the set between x86 and sparc us disjoint.)

- port libsanitizer's GetPcSpBp() to sparc, sparc64 and amd64.
 1.4 02-Dec-2013  joerg Improve EABI handling of string functions. Most importantly, fix
__aeabi_memset, which has the arguments in the wrong order.
 1.3 08-Nov-2011  joerg branches: 1.3.6;
libkern.h maps memcmp, memset and strcpy to the compiler builtins on
some platforms, so make sure to #undef them first.
 1.2 04-Jun-2007  christos branches: 1.2.34;
handle fortify, ansify.
 1.1 20-Dec-2005  christos Merge libkern + libc common files. As requested by core.
 1.2.34.2 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.2.34.1 10-Nov-2011  yamt sync with head
 1.3.6.1 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.5.8.1 29-Feb-2020  ad Sync with head.
 1.5.4.3 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.5.4.2 21-Apr-2020  martin Sync with HEAD
 1.5.4.1 08-Apr-2020  martin Merge changes from current as of 20200406
 1.2 02-Dec-2013  joerg branches: 1.2.26;
Improve EABI handling of string functions. Most importantly, fix
__aeabi_memset, which has the arguments in the wrong order.
 1.1 20-Dec-2005  christos branches: 1.1.50; 1.1.56;
Merge libkern + libc common files. As requested by core.
 1.1.56.1 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.1.50.1 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.2.26.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.2.26.1 21-Apr-2020  martin Sync with HEAD
 1.6 14-Dec-2024  riastradh memmem(3): Avoid undefined behaviour in shift.

PR lib/58906: memmem.c: sync with musl upstream that fixes UB on
signed overflow
 1.5 14-Dec-2024  riastradh memmem(3): Nix trailing whitespace.

No functional change intended.

Preparation for:

PR lib/58906: memmem.c: sync with musl upstream that fixes UB on signed overflow
 1.4 16-May-2021  rillig branches: 1.4.10;
memmem: remove unreachable return statement
 1.3 15-Oct-2018  christos branches: 1.3.2; 1.3.12;
use postincrement, like the patch
XXX: pullup-8
 1.2 15-Oct-2018  christos Avoid out-of-bounds reads
https://www.openwall.com/lists/musl/2017/06/29/6
XXX: pullup-8
 1.1 08-Jul-2018  christos branches: 1.1.2;
switch to FreeBSD's memmem (faster)
 1.1.2.3 20-Oct-2018  pgoyette Sync with head
 1.1.2.2 28-Jul-2018  pgoyette Sync with HEAD
 1.1.2.1 08-Jul-2018  pgoyette file memmem.c was added on branch pgoyette-compat on 2018-07-28 04:32:57 +0000
 1.3.12.1 31-May-2021  cjep sync with head
 1.3.2.4 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.3.2.3 21-Apr-2020  martin Sync with HEAD
 1.3.2.2 10-Jun-2019  christos Sync with HEAD
 1.3.2.1 15-Oct-2018  christos file memmem.c was added on branch phil-wifi on 2019-06-10 21:41:07 +0000
 1.4.10.1 02-Aug-2025  perseant Sync with HEAD
 1.2 02-Dec-2013  joerg branches: 1.2.26;
Improve EABI handling of string functions. Most importantly, fix
__aeabi_memset, which has the arguments in the wrong order.
 1.1 20-Dec-2005  christos branches: 1.1.50; 1.1.56;
Merge libkern + libc common files. As requested by core.
 1.1.56.1 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.1.50.1 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.2.26.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.2.26.1 21-Apr-2020  martin Sync with HEAD
 1.12 30-Mar-2019  jmcneill Fix typo: __aebi_memset -> __aeabi_memset
 1.11 06-Feb-2018  mrg branches: 1.11.4;
- remove two more _DIAGASSERT() checks against not NULL for functions
with arguments with nonnull attributes. in two cases, leave
code behind that should set defaults to "(null)".
 1.10 02-Dec-2013  joerg Improve EABI handling of string functions. Most importantly, fix
__aeabi_memset, which has the arguments in the wrong order.
 1.9 08-Nov-2011  joerg branches: 1.9.6;
libkern.h maps memcmp, memset and strcpy to the compiler builtins on
some platforms, so make sure to #undef them first.
 1.8 18-Mar-2009  tsutsui branches: 1.8.6;
- remove bcmp(9), bcopy(9), and bzero(9) from libkern since <sys/systm.h> has
macro which replace them with mem*() functions in #ifdef _KERNEL as noted
in man pages
- move declarations of bcopy(3) and bzero(3) into <lib/libsa/stand.h>
since they are still in libsa for some MD standalone sources
(I guess all bcmp(3) in standalone sources have been replaced with memcmp(3)
but they should be replaced with memcmp() anyway)
 1.7 17-Mar-2009  he Add local declarations of bcmp() and bzero() because libkern.h no longer
provides those declarations.

It's possible that these can be removed later when we find out whether
these functions are actually needed in the kernel, but meanwhile, this
allows alpha, hp700 and the powerpc ports (who all build these as part
of libkern) to complete their builds.
 1.6 29-Mar-2008  he branches: 1.6.10;
Do like the non-__OPTIMIZE_SIZE__ variant, and compile memset() unless
BZERO is defined, instead of requiring MEMSET to be defined.

Avoids violating the principle of least astonishment, and also fixes
the build of stand/ and kernels for (at least) alpha.
 1.5 29-Mar-2008  he Make the size-optimized memset() actually conform to the man page
by returning the first arg.
 1.4 26-Mar-2008  apb Add /* __OPTIMIZE_SIZE__ */ comments on relevant #else and #endif lines.
Also change some #ifdef to #if defined() for consistency.
 1.3 25-Mar-2008  christos add bcopy and bzero for the benefit of loadfile.
 1.2 04-Jun-2007  christos handle fortify, ansify.
 1.1 20-Dec-2005  christos Merge libkern + libc common files. As requested by core.
 1.6.10.1 13-May-2009  jym Sync with HEAD.

Commit is split, to avoid a "too many arguments" protocol error.
 1.8.6.2 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.8.6.1 10-Nov-2011  yamt sync with head
 1.9.6.1 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.11.4.3 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.11.4.2 21-Apr-2020  martin Sync with HEAD
 1.11.4.1 10-Jun-2019  christos Sync with HEAD
 1.11 15-Jan-2022  andvar fix typos in comments.
 1.10 19-Apr-2021  simonb Add CVS ID line.
 1.9 17-Apr-2021  mrg avoid redefinition warning for __OPTIMIZE_SIZE__.
 1.8 17-Apr-2021  simonb Use __register_t instead of uregister_t - this is available to all ports
and both userland and kernel.
 1.7 17-Apr-2021  simonb Cast the fill value to unsigned char so that the "fill" value used for
full-word fills isn't garbage.
 1.6 17-Apr-2021  simonb Disable the larger/faster code path. While the optimised code path was
indeed quicker, it nonetheless failed to actually fill all the requested
memory with the specified value much of the time if a non-aligned start
address was used.
 1.5 02-Mar-2012  apb branches: 1.5.34;
remove trailing white space
 1.4 02-Mar-2012  apb Change CTASSERT to __CTASSERT (the spelling used in <sys/cdefs.h>),
include <sys/cdefs.h> so that it will be defined,
and move it to a better place.

The previous CTASSERT would never have been used because
of the ifdef. If it had been used, it would have had
unwanted effects from being just after the "static inline"
that appears when BZERO is defined.

Also move the __RCSID to a more conventional location.
 1.3 08-Nov-2011  joerg libkern.h maps memcmp, memset and strcpy to the compiler builtins on
some platforms, so make sure to #undef them first.
 1.2 14-Dec-2009  matt branches: 1.2.6;
Merge from matt-nb5-mips64
 1.1 17-Aug-2009  matt branches: 1.1.2;
file memset2.c was initially added on branch matt-nb5-mips64.
 1.1.2.3 23-Aug-2009  matt CTASSERT only defined for kernel enviornments.
 1.1.2.2 23-Aug-2009  matt Add a unit test to this file. Fix two bugs.
 1.1.2.1 17-Aug-2009  matt Add word-load-store optimized version of memset/bzero. On platforms with
64bit registers they can be as efficient as assembly coded version.
 1.2.6.2 17-Apr-2012  yamt sync with head
 1.2.6.1 10-Nov-2011  yamt sync with head
 1.5.34.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.5.34.1 21-Apr-2020  martin Sync with HEAD
 1.5 29-May-2015  matt branches: 1.5.16;
Don't compile if there is a macro of the same name.
 1.4 21-Aug-2011  dholland branches: 1.4.4;
Requires stdint.h.
 1.3 21-Jul-2009  drochner branches: 1.3.2;
remove stray semicolons, makes lint happy and lets the libc build proceed
 1.2 21-Jul-2009  joerg Include limits.h.
 1.1 21-Jul-2009  joerg Move popcount et al to src/common and add popcount32/popcount64.
Requested by rmind@. MD should now override popcount32/popcount64 and
provide the aliases as fitting.
 1.3.2.2 23-Jul-2009  jym Sync with HEAD.
 1.3.2.1 21-Jul-2009  jym file popcount32.c was added on branch jym-xensuspend on 2009-07-23 23:31:35 +0000
 1.4.4.2 27-Dec-2011  matt Add popcount32 and popcount64 for flash/nand
 1.4.4.1 21-Aug-2011  matt file popcount32.c was added on branch matt-nb5-mips64 on 2011-12-27 19:20:59 +0000
 1.5.16.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.5.16.1 21-Apr-2020  martin Sync with HEAD
 1.8 29-May-2015  matt branches: 1.8.16;
Don't compile if there is a macro of the same name.
 1.7 09-Mar-2012  christos Casts and type changes to fix portability issues.
- int -> size_t
- adjust width of RHS of shift
- adjust widths of types
 1.6 21-Aug-2011  dholland branches: 1.6.2; 1.6.4;
Requires stdint.h.
 1.5 05-Aug-2009  joerg Fix a comment to not be self-referencing.
 1.4 21-Jul-2009  drochner branches: 1.4.2;
remove stray semicolons, makes lint happy and lets the libc build proceed
 1.3 21-Jul-2009  joerg Make the constant u_longlong too, lint is just too stupid...
 1.2 21-Jul-2009  joerg Add explicit casts to make lint happy.
 1.1 21-Jul-2009  joerg Move popcount et al to src/common and add popcount32/popcount64.
Requested by rmind@. MD should now override popcount32/popcount64 and
provide the aliases as fitting.
 1.4.2.2 23-Jul-2009  jym Sync with HEAD.
 1.4.2.1 21-Jul-2009  jym file popcount64.c was added on branch jym-xensuspend on 2009-07-23 23:31:35 +0000
 1.6.4.2 27-Dec-2011  matt Add popcount32 and popcount64 for flash/nand
 1.6.4.1 21-Aug-2011  matt file popcount64.c was added on branch matt-nb5-mips64 on 2011-12-27 19:20:59 +0000
 1.6.2.1 17-Apr-2012  yamt sync with head
 1.8.16.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.8.16.1 21-Apr-2020  martin Sync with HEAD
 1.3 11-Aug-2018  christos Add nbotool stuff.
 1.2 04-Jun-2007  christos branches: 1.2.76; 1.2.78;
handle fortify, ansify.
 1.1 20-Dec-2005  christos Merge libkern + libc common files. As requested by core.
 1.2.78.3 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.2.78.2 21-Apr-2020  martin Sync with HEAD
 1.2.78.1 10-Jun-2019  christos Sync with HEAD
 1.2.76.1 06-Sep-2018  pgoyette Sync with HEAD

Resolve a couple of conflicts (result of the uimin/uimax changes)
 1.4 07-Apr-2020  skrll Fix KASAN build on aarch64
 1.3 04-Feb-2018  mrg branches: 1.3.4;
updates for GCC 6.4:

- remove many _DIAGASSERT() checks against not NULL for functions
with arguments with nonnull attributes. (probably more to come,
the set between x86 and sparc us disjoint.)

- port libsanitizer's GetPcSpBp() to sparc, sparc64 and amd64.
 1.2 04-Jun-2007  christos handle fortify, ansify.
 1.1 20-Dec-2005  christos Merge libkern + libc common files. As requested by core.
 1.3.4.3 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.3.4.2 21-Apr-2020  martin Sync with HEAD
 1.3.4.1 13-Apr-2020  martin Mostly merge changes from HEAD upto 20200411
 1.7 07-Apr-2020  skrll Fix KASAN build on aarch64
 1.6 04-Feb-2018  mrg branches: 1.6.4;
updates for GCC 6.4:

- remove many _DIAGASSERT() checks against not NULL for functions
with arguments with nonnull attributes. (probably more to come,
the set between x86 and sparc us disjoint.)

- port libsanitizer's GetPcSpBp() to sparc, sparc64 and amd64.
 1.5 31-Aug-2011  plunky NULL does not need a cast
 1.4 17-Jul-2009  dsl Change all archs so that strchr.[cS] and strrchr.[cS] exist and generate
duplicate symbols for index() and rindex().
 1.3 08-Jan-2008  martin branches: 1.3.12;
When passing char (or unsigned char) arguments as int, assign them to a
temporary char (resp. unsigned char) before comparing, to force truncation
to the proper domain. This makes the coresponding regression tests succeed.
 1.2 04-Jun-2007  christos branches: 1.2.4;
handle fortify, ansify.
 1.1 20-Dec-2005  christos Merge libkern + libc common files. As requested by core.
 1.2.4.1 23-Mar-2008  matt sync with HEAD
 1.3.12.1 23-Jul-2009  jym Sync with HEAD.
 1.6.4.3 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.6.4.2 21-Apr-2020  martin Sync with HEAD
 1.6.4.1 13-Apr-2020  martin Mostly merge changes from HEAD upto 20200411
 1.1 12-Oct-2016  christos branches: 1.1.2; 1.1.16;
Add strchrnul
 1.1.16.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.1.16.1 21-Apr-2020  martin Sync with HEAD
 1.1.2.2 04-Nov-2016  pgoyette Sync with HEAD
 1.1.2.1 12-Oct-2016  pgoyette file strchrnul.c was added on branch pgoyette-localcount on 2016-11-04 14:42:22 +0000
 1.4 04-Feb-2018  mrg branches: 1.4.4;
updates for GCC 6.4:

- remove many _DIAGASSERT() checks against not NULL for functions
with arguments with nonnull attributes. (probably more to come,
the set between x86 and sparc us disjoint.)

- port libsanitizer's GetPcSpBp() to sparc, sparc64 and amd64.
 1.3 01-Jul-2013  joerg Don't pick up strcmp as macro from libkern.h.
 1.2 04-Jun-2007  christos branches: 1.2.34; 1.2.40;
handle fortify, ansify.
 1.1 20-Dec-2005  christos Merge libkern + libc common files. As requested by core.
 1.2.40.1 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.2.34.1 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.4.4.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.4.4.1 21-Apr-2020  martin Sync with HEAD
 1.4 04-Feb-2018  mrg branches: 1.4.4;
updates for GCC 6.4:

- remove many _DIAGASSERT() checks against not NULL for functions
with arguments with nonnull attributes. (probably more to come,
the set between x86 and sparc us disjoint.)

- port libsanitizer's GetPcSpBp() to sparc, sparc64 and amd64.
 1.3 08-Nov-2011  joerg libkern.h maps memcmp, memset and strcpy to the compiler builtins on
some platforms, so make sure to #undef them first.
 1.2 04-Jun-2007  christos branches: 1.2.34;
handle fortify, ansify.
 1.1 20-Dec-2005  christos Merge libkern + libc common files. As requested by core.
 1.2.34.1 10-Nov-2011  yamt sync with head
 1.4.4.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.4.4.1 21-Apr-2020  martin Sync with HEAD
 1.2 04-Feb-2018  mrg branches: 1.2.4;
fixes for GCC 6:

- -Wstrict-prototypes is not available for C++, so don't try to
ignore it for C++.
- remove many _DIAGASSERT() checks against not NULL for functions
with arguments with nonnull attributes. in two cases, leave
code behind that should set defaults to "(null)".
- use -Wno-error=frame-address for i386 mcount, as it seems valid
to assume the caller will have a frame.fair
 1.1 19-Jul-2014  lneto branches: 1.1.2; 1.1.6;
lua: updated from 5.1 to 5.3 work3

* lua(1):
- changed lua_Integer to intmax_t
- updated distrib/sets/lists and etc/mtree
- updated bsd.lua.mk
- fixed bozohttpd (lua-bozo.c)
- compatibilized bindings: gpio, sqlite
* lua(4):
- removed floating-point and libc dependencies using '#ifndef _KERNEL'
- fixed division by zero and exponentiation
- libkern: added isalnum(), iscntrl(), isgraph(), isprint() and ispunct()
- acpica: removed isprint() from acnetbsd.h
- libc: moved strcspn.c, strpbrk.c and strspn.c to common
- removed stub headers
- compatibilized bindings: luapmf, luasystm
* reorganized luaconf.h
* updated doc/CHANGES and doc/RESPONSIBLE
 1.1.6.2 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.1.6.1 19-Jul-2014  tls file strcspn.c was added on branch tls-maxphys on 2014-08-19 23:45:14 +0000
 1.1.2.2 10-Aug-2014  tls Rebase.
 1.1.2.1 19-Jul-2014  tls file strcspn.c was added on branch tls-earlyentropy on 2014-08-10 06:47:06 +0000
 1.2.4.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.2.4.1 21-Apr-2020  martin Sync with HEAD
 1.5 01-Nov-2024  riastradh string.h: Fix various symbol visibility issues.

1. Order declarations according to POSIX 2024 to make this easier to
review side-by-side with the spec.
2. Fix visibility of memccpy: XSI-only, not POSIX in general; require
_XOPEN_SOURCE, not just _POSIX_C_SOURCE.
3. Omit redundant _XOPEN_SOURCE test around stpcpy/stpncpy.
4. Hide strdup in POSIX 2001. Not POSIX (without XSI) until 2008.
5. Hide strerror_r until POSIX 2001. Can't find evidence of it in
any earlier POSIX or X/Open. (Not 100% sure on this one, maybe
someone can double-check my research.)
6. Add restrict to strlcat/strlcpy.
7. Omit redundant _XOPEN_SOURCE test around strndup and strnlen.
8. Hide strtok_r until POSIX 2001. Can't find evidence of it in
any earlier POSIX or X/Open. (Not 100% sure on this one, maybe
someone can double-check my research.)

Carry the restrict qualifiers on strlcat/strlcpy to libkern too.

Main reference:
https://pubs.opengroup.org/onlinepubs/9799919799/basedefs/string.h.html

PR standards/58804: string.h: wrong visibility for memccpy
 1.4 23-Jan-2013  matt branches: 1.4.32; 1.4.52;
Add a (unused) variant of strlcat that uses strnlen and strlcpy to do the work.
 1.3 04-Jun-2007  christos branches: 1.3.34; 1.3.40;
handle fortify, ansify.
 1.2 30-Mar-2006  christos remove bogus _LIBC defines. broke lint.
 1.1 20-Dec-2005  christos Merge libkern + libc common files. As requested by core.
 1.3.40.1 25-Feb-2013  tls resync with head
 1.3.34.1 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.4.52.1 02-Aug-2025  perseant Sync with HEAD
 1.4.32.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.4.32.1 21-Apr-2020  martin Sync with HEAD
 1.4 01-Nov-2024  riastradh string.h: Fix various symbol visibility issues.

1. Order declarations according to POSIX 2024 to make this easier to
review side-by-side with the spec.
2. Fix visibility of memccpy: XSI-only, not POSIX in general; require
_XOPEN_SOURCE, not just _POSIX_C_SOURCE.
3. Omit redundant _XOPEN_SOURCE test around stpcpy/stpncpy.
4. Hide strdup in POSIX 2001. Not POSIX (without XSI) until 2008.
5. Hide strerror_r until POSIX 2001. Can't find evidence of it in
any earlier POSIX or X/Open. (Not 100% sure on this one, maybe
someone can double-check my research.)
6. Add restrict to strlcat/strlcpy.
7. Omit redundant _XOPEN_SOURCE test around strndup and strnlen.
8. Hide strtok_r until POSIX 2001. Can't find evidence of it in
any earlier POSIX or X/Open. (Not 100% sure on this one, maybe
someone can double-check my research.)

Carry the restrict qualifiers on strlcat/strlcpy to libkern too.

Main reference:
https://pubs.opengroup.org/onlinepubs/9799919799/basedefs/string.h.html

PR standards/58804: string.h: wrong visibility for memccpy
 1.3 04-Jun-2007  christos branches: 1.3.78; 1.3.98;
handle fortify, ansify.
 1.2 30-Mar-2006  christos remove bogus _LIBC defines. broke lint.
 1.1 20-Dec-2005  christos Merge libkern + libc common files. As requested by core.
 1.3.98.1 02-Aug-2025  perseant Sync with HEAD
 1.3.78.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.3.78.1 21-Apr-2020  martin Sync with HEAD
 1.3 06-Feb-2018  mrg branches: 1.3.4;
- remove two more _DIAGASSERT() checks against not NULL for functions
with arguments with nonnull attributes. in two cases, leave
code behind that should set defaults to "(null)".
 1.2 04-Jun-2007  christos handle fortify, ansify.
 1.1 20-Dec-2005  christos Merge libkern + libc common files. As requested by core.
 1.3.4.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.3.4.1 21-Apr-2020  martin Sync with HEAD
 1.3 16-Aug-2018  christos toolify.
 1.2 04-Jun-2007  christos branches: 1.2.76; 1.2.78;
handle fortify, ansify.
 1.1 20-Dec-2005  christos Merge libkern + libc common files. As requested by core.
 1.2.78.3 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.2.78.2 21-Apr-2020  martin Sync with HEAD
 1.2.78.1 10-Jun-2019  christos Sync with HEAD
 1.2.76.1 06-Sep-2018  pgoyette Sync with HEAD

Resolve a couple of conflicts (result of the uimin/uimax changes)
 1.3 04-Feb-2018  mrg branches: 1.3.4;
fixes for GCC 6:

- -Wstrict-prototypes is not available for C++, so don't try to
ignore it for C++.
- remove many _DIAGASSERT() checks against not NULL for functions
with arguments with nonnull attributes. in two cases, leave
code behind that should set defaults to "(null)".
- use -Wno-error=frame-address for i386 mcount, as it seems valid
to assume the caller will have a frame.fair
 1.2 27-Dec-2013  christos branches: 1.2.4; 1.2.8;
make it kernel friendly.
 1.1 27-Dec-2013  christos kernel needs it now too.
 1.2.8.2 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.2.8.1 27-Dec-2013  tls file strncat.c was added on branch tls-maxphys on 2014-08-19 23:45:14 +0000
 1.2.4.2 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.2.4.1 27-Dec-2013  yamt file strncat.c was added on branch yamt-pagecache on 2014-05-22 11:26:30 +0000
 1.3.4.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.3.4.1 21-Apr-2020  martin Sync with HEAD
 1.3 04-Feb-2018  mrg branches: 1.3.4;
updates for GCC 6.4:

- remove many _DIAGASSERT() checks against not NULL for functions
with arguments with nonnull attributes. (probably more to come,
the set between x86 and sparc us disjoint.)

- port libsanitizer's GetPcSpBp() to sparc, sparc64 and amd64.
 1.2 04-Jun-2007  christos handle fortify, ansify.
 1.1 20-Dec-2005  christos Merge libkern + libc common files. As requested by core.
 1.3.4.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.3.4.1 21-Apr-2020  martin Sync with HEAD
 1.4 04-Feb-2018  mrg branches: 1.4.4;
fixes for GCC 6:

- -Wstrict-prototypes is not available for C++, so don't try to
ignore it for C++.
- remove many _DIAGASSERT() checks against not NULL for functions
with arguments with nonnull attributes. in two cases, leave
code behind that should set defaults to "(null)".
- use -Wno-error=frame-address for i386 mcount, as it seems valid
to assume the caller will have a frame.fair
 1.3 04-Jun-2007  christos handle fortify, ansify.
 1.2 03-Jun-2007  christos Make these work with _FORTIFY_SOURCE, by overriding the fortified definitions
 1.1 20-Dec-2005  christos Merge libkern + libc common files. As requested by core.
 1.4.4.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.4.4.1 21-Apr-2020  martin Sync with HEAD
 1.2 09-Jan-2014  apb branches: 1.2.26;
Make strnlen implementation usable from src/tools/compat.
 1.1 01-Sep-2011  jym branches: 1.1.2; 1.1.8;
Make strnlen(3) accessible in kernel. ok christos@.
 1.1.8.1 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.1.2.1 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.2.26.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.2.26.1 21-Apr-2020  martin Sync with HEAD
 1.3 20-Jan-2024  christos Add __UNCONST
 1.2 04-Feb-2018  mrg branches: 1.2.4;
fixes for GCC 6:

- -Wstrict-prototypes is not available for C++, so don't try to
ignore it for C++.
- remove many _DIAGASSERT() checks against not NULL for functions
with arguments with nonnull attributes. in two cases, leave
code behind that should set defaults to "(null)".
- use -Wno-error=frame-address for i386 mcount, as it seems valid
to assume the caller will have a frame.fair
 1.1 19-Jul-2014  lneto branches: 1.1.2; 1.1.6;
lua: updated from 5.1 to 5.3 work3

* lua(1):
- changed lua_Integer to intmax_t
- updated distrib/sets/lists and etc/mtree
- updated bsd.lua.mk
- fixed bozohttpd (lua-bozo.c)
- compatibilized bindings: gpio, sqlite
* lua(4):
- removed floating-point and libc dependencies using '#ifndef _KERNEL'
- fixed division by zero and exponentiation
- libkern: added isalnum(), iscntrl(), isgraph(), isprint() and ispunct()
- acpica: removed isprint() from acnetbsd.h
- libc: moved strcspn.c, strpbrk.c and strspn.c to common
- removed stub headers
- compatibilized bindings: luapmf, luasystm
* reorganized luaconf.h
* updated doc/CHANGES and doc/RESPONSIBLE
 1.1.6.2 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.1.6.1 19-Jul-2014  tls file strpbrk.c was added on branch tls-maxphys on 2014-08-19 23:45:14 +0000
 1.1.2.2 10-Aug-2014  tls Rebase.
 1.1.2.1 19-Jul-2014  tls file strpbrk.c was added on branch tls-earlyentropy on 2014-08-10 06:47:06 +0000
 1.2.4.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.2.4.1 21-Apr-2020  martin Sync with HEAD
 1.7 07-Apr-2020  skrll Fix KASAN build on aarch64
 1.6 04-Feb-2018  mrg branches: 1.6.4;
updates for GCC 6.4:

- remove many _DIAGASSERT() checks against not NULL for functions
with arguments with nonnull attributes. (probably more to come,
the set between x86 and sparc us disjoint.)

- port libsanitizer's GetPcSpBp() to sparc, sparc64 and amd64.
 1.5 17-Jul-2009  dsl Change all archs so that strchr.[cS] and strrchr.[cS] exist and generate
duplicate symbols for index() and rindex().
 1.4 08-Jan-2008  martin branches: 1.4.12;
When passing char (or unsigned char) arguments as int, assign them to a
temporary char (resp. unsigned char) before comparing, to force truncation
to the proper domain. This makes the coresponding regression tests succeed.
 1.3 04-Jun-2007  christos branches: 1.3.4;
handle fortify, ansify.
 1.2 13-Mar-2006  martin Enforce argument domain by assigning to a char variable and searching
for that. This way we pass the lib/libc/string/strrchr regression tests.
 1.1 20-Dec-2005  christos Merge libkern + libc common files. As requested by core.
 1.3.4.1 23-Mar-2008  matt sync with HEAD
 1.4.12.1 23-Jul-2009  jym Sync with HEAD.
 1.6.4.3 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.6.4.2 21-Apr-2020  martin Sync with HEAD
 1.6.4.1 13-Apr-2020  martin Mostly merge changes from HEAD upto 20200411
 1.3 04-Jun-2007  christos branches: 1.3.78;
handle fortify, ansify.
 1.2 19-Feb-2007  chs don't do the weak symbol thing in kernel context.
 1.1 13-Nov-2006  dyoung branches: 1.1.2; 1.1.4;
Add strsep(3) to libkern.

To avoid code duplication, move strsep.c to the kernel/userland
common files.

Soon I will commit source-address selection (options IPSELSRC).
It will use strsep(3).
 1.1.4.1 10-Mar-2007  bouyer Pull up following revision(s) (requested by chs in ticket #504):
common/lib/libc/string/strsep.c: revision 1.2
don't do the weak symbol thing in kernel context.
 1.1.2.2 13-Nov-2006  dyoung Add strsep(3) to libkern.

To avoid code duplication, move strsep.c to the kernel/userland
common files.

Soon I will commit source-address selection (options IPSELSRC).
It will use strsep(3).
 1.1.2.1 13-Nov-2006  dyoung file strsep.c was added on branch newlock2 on 2006-11-13 03:26:44 +0000
 1.3.78.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.3.78.1 21-Apr-2020  martin Sync with HEAD
 1.3 18-Jun-2023  rillig strspn: fix typo in comment
 1.2 04-Feb-2018  mrg branches: 1.2.4;
fixes for GCC 6:

- -Wstrict-prototypes is not available for C++, so don't try to
ignore it for C++.
- remove many _DIAGASSERT() checks against not NULL for functions
with arguments with nonnull attributes. in two cases, leave
code behind that should set defaults to "(null)".
- use -Wno-error=frame-address for i386 mcount, as it seems valid
to assume the caller will have a frame.fair
 1.1 19-Jul-2014  lneto branches: 1.1.2; 1.1.6;
lua: updated from 5.1 to 5.3 work3

* lua(1):
- changed lua_Integer to intmax_t
- updated distrib/sets/lists and etc/mtree
- updated bsd.lua.mk
- fixed bozohttpd (lua-bozo.c)
- compatibilized bindings: gpio, sqlite
* lua(4):
- removed floating-point and libc dependencies using '#ifndef _KERNEL'
- fixed division by zero and exponentiation
- libkern: added isalnum(), iscntrl(), isgraph(), isprint() and ispunct()
- acpica: removed isprint() from acnetbsd.h
- libc: moved strcspn.c, strpbrk.c and strspn.c to common
- removed stub headers
- compatibilized bindings: luapmf, luasystm
* reorganized luaconf.h
* updated doc/CHANGES and doc/RESPONSIBLE
 1.1.6.2 19-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.1.6.1 19-Jul-2014  tls file strspn.c was added on branch tls-maxphys on 2014-08-19 23:45:14 +0000
 1.1.2.2 10-Aug-2014  tls Rebase.
 1.1.2.1 19-Jul-2014  tls file strspn.c was added on branch tls-earlyentropy on 2014-08-10 06:47:06 +0000
 1.2.4.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.2.4.1 21-Apr-2020  martin Sync with HEAD
 1.3 04-Feb-2018  mrg branches: 1.3.4;
fixes for GCC 6:

- -Wstrict-prototypes is not available for C++, so don't try to
ignore it for C++.
- remove many _DIAGASSERT() checks against not NULL for functions
with arguments with nonnull attributes. in two cases, leave
code behind that should set defaults to "(null)".
- use -Wno-error=frame-address for i386 mcount, as it seems valid
to assume the caller will have a frame.fair
 1.2 04-Jun-2007  christos handle fortify, ansify.
 1.1 20-Dec-2005  christos Merge libkern + libc common files. As requested by core.
 1.3.4.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.3.4.1 21-Apr-2020  martin Sync with HEAD
 1.21 11-May-2019  maxv Fix bug, the computation of cpuset_nentries was incorrect, we must do +1
to be able to address the last 32 bits.

On a machine with 80 CPUs, this caused "cpuctl identify >64" to return
garbage.
 1.20 26-Jul-2018  kamil Avoid undefined behavior in an cpuset.c

Do not change the signedness bit with a left shift operation.
Switch to unsigned integer to prevent this.

cpuset.c:112:18, left shift of 1 by 31 places cannot be represented in type 'int'

Detected with micro-UBSan in the user mode.
 1.19 04-Jan-2018  kamil branches: 1.19.2; 1.19.4;
Add bunch of missing includes of namespace.h in libc

The NetBSD Standard C Library uses internally some of its functions with
a mangled symbol name, usually "_symbol". The internal functions shall not
use the global (public) symbols.

This change eliminates usage of the global changes of the following symbols:
- strlcat -> _strlcat
- sysconf -> __sysconf
- closedir -> _closedir
- fparseln -> _fparseln
- kill -> _kill
- mkstemp -> _mkstemp
- reallocarr -> _reallocarr
- strcasecmp -> _strcasecmp
- strncasecmp -> _strncasecmp
- strptime -> _strptime
- strtok_r -> _strtok_r
- sysctl -> _sysctl
- dlopen -> __dlopen
- dlclose -> __dlclose
- dlsym -> __dlsym

Sponsored by <The NetBSD Foundation>
 1.18 09-Mar-2012  christos branches: 1.18.26;
Casts and type changes to fix portability issues.
- int -> size_t
- adjust width of RHS of shift
- adjust widths of types
 1.17 07-Aug-2011  rmind branches: 1.17.2;
Add kcpuset(9) - a reworked dynamic CPU set implementation for kernel.
Suitable for use during the early boot. MD and other implementations
should be replaced with this interface.

Discussed on: tech-kern@
 1.16 21-Sep-2010  rmind _cpuset_create: initialize size argument for sysctl call.
From PR/43837 by Sandy Snaman.
 1.15 25-Apr-2009  rmind - Add kcpuset_iszero().
- Fix kcpuset_match().
 1.14 15-Mar-2009  rmind - Add kcpuset_fill(), kcpuset_set() and kcpuset_match().
- KNF while here.
 1.13 14-Feb-2009  christos make this work with pcc. The previous construct is not valid c99.
 1.12 27-Jul-2008  dholland branches: 1.12.4; 1.12.6;
Don't convert cpuid_t through int while testing it for being in range.
(Might truncate it, for one thing. Also, if the shift result is < 0 the
demons have already flown out of our nose.)

Fixes some -Wsign-compare warnings.
 1.11 14-Jul-2008  rmind Add few KASSERTs.
 1.10 22-Jun-2008  he branches: 1.10.2;
Be consistent in how cpuset_nentries is computed in the kernel and
in user-land. This fixes the user-land part. Now "cpuctl identify 0"
gives a result instead of getting EINVAL from the set-affinity syscall.
 1.9 22-Jun-2008  rmind kcpuset_isset: remove if-check. It is not reasonable for kernel,
saves few cycles, and it is better for CPU cache.
 1.8 22-Jun-2008  christos Separate cpuset and kcpuset, and only use the bits in userland. Requested
by yamt
 1.7 16-Jun-2008  christos branches: 1.7.2;
We don't need any of this in standalone mode.
 1.6 16-Jun-2008  christos little more lint
 1.5 16-Jun-2008  christos make lint happy
 1.4 16-Jun-2008  rmind - Add general cpuset macros.
- Use kcpuset name for kernel-only functions.
- Use cpuid_t to specify CPU ID.
- Unify all cpuset users.

API is expected to be stable now.
 1.3 15-Jun-2008  rmind cpuset_create: check the return value of calloc().
 1.2 15-Jun-2008  rmind - cpuset_create: pass correct argument to sysctl(3).
- Constify few variables.
 1.1 15-Jun-2008  christos cpuset related functions, shared between the kernel and userland.
 1.7.2.2 17-Jun-2008  yamt sync with head.
 1.7.2.1 16-Jun-2008  yamt file cpuset.c was added on branch yamt-pf42 on 2008-06-17 09:13:31 +0000
 1.10.2.4 18-Sep-2008  wrstuden Sync with wrstuden-revivesa-base-2.
 1.10.2.3 24-Jun-2008  wrstuden Actually use the version in wrstuden-revivesa-base-1. Compiles better.
 1.10.2.2 24-Jun-2008  wrstuden Add file missed in merge. Patch will add files, but won't make
new directories.
 1.10.2.1 22-Jun-2008  wrstuden file cpuset.c was added on branch wrstuden-revivesa on 2008-06-24 04:47:49 +0000
 1.12.6.1 13-May-2009  jym Sync with HEAD.

Commit is split, to avoid a "too many arguments" protocol error.
 1.12.4.1 20-Oct-2013  bouyer Pull up following revision(s) (requested by dholland in ticket #1883):
common/lib/libc/sys/cpuset.c: revision 1.16
_cpuset_create: initialize size argument for sysctl call.
From PR/43837 by Sandy Snaman.
 1.17.2.1 17-Apr-2012  yamt sync with head
 1.18.26.1 12-May-2019  martin Pull up following revision(s) (requested by maxv in ticket #1260):

common/lib/libc/sys/cpuset.c: revision 1.21
usr.sbin/cpuctl/cpuctl.c: revision 1.30

Fix bug, the computation of cpuset_nentries was incorrect, we must do +1
to be able to address the last 32 bits.

On a machine with 80 CPUs, this caused "cpuctl identify >64" to return
garbage.

Check the return value of cpuset_set(), to prevent future surprises.
 1.19.4.3 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.19.4.2 21-Apr-2020  martin Sync with HEAD
 1.19.4.1 10-Jun-2019  christos Sync with HEAD
 1.19.2.1 28-Jul-2018  pgoyette Sync with HEAD

RSS XML Feed