Home | History | Annotate | only in /src/common/lib/libc/arch/sparc64/atomic
History log of /src/common/lib/libc/arch/sparc64/atomic
RevisionDateAuthorComments
 1.6 04-Jan-2009  pooka branches: 1.6.50;
allow inclusion of atomic ops in librump
 1.5 29-Sep-2008  ad Allow atomic ops to be built as part of libpthread.
 1.4 11-Feb-2008  ad Only build atomic ops for libkern/libc.
 1.3 10-Feb-2008  ad Enable the atomic ops in userspace.
 1.2 28-Nov-2007  ad branches: 1.2.4;
Pull in sparc64 atomic ops from the thorpej-atomic branch.
 1.1 27-Apr-2007  thorpej branches: 1.1.2;
file Makefile.inc was initially added on branch thorpej-atomic.
 1.1.2.1 27-Apr-2007  thorpej First cut at atomic ops for sparc64.
 1.2.4.3 23-Mar-2008  matt sync with HEAD
 1.2.4.2 09-Jan-2008  matt sync with HEAD
 1.2.4.1 28-Nov-2007  matt file Makefile.inc was added on branch matt-armv6 on 2008-01-09 01:21:09 +0000
 1.6.50.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.6.50.1 21-Apr-2020  martin Sync with HEAD
 1.8 06-Apr-2022  riastradh Nix trailing whitespace in files of membars, atomics, and lock stubs.

Will be touching many of these files soon for functional changes.

No functional change intended.
 1.7 18-Oct-2015  nakayama branches: 1.7.16;
Fix 64-bit atomic operations on 32-bit kernel and userland, and add
support for missing __sync_*_8 builtins.
 1.6 25-May-2008  chs enable profiling of assembly functions.
 1.5 28-Apr-2008  martin branches: 1.5.2;
Remove clause 3 and 4 from TNF licenses
 1.4 09-Apr-2008  nakayama branches: 1.4.2;
Use %xcc for 64-bit comparisons.
 1.3 22-Dec-2007  mrg branches: 1.3.2;
- provide 32 bit versions of the sparc v9 atomic ops.
- use them on sparc64 32 bit kernels.

fixes issues on 32 bit sparc64 kernels that were using sparc v8 ops.
tested by rjs@.
 1.2 28-Nov-2007  ad Pull in sparc64 atomic ops from the thorpej-atomic branch.
 1.1 27-Apr-2007  thorpej branches: 1.1.2;
file atomic_add.S was initially added on branch thorpej-atomic.
 1.1.2.1 27-Apr-2007  thorpej First cut at atomic ops for sparc64.
 1.3.2.2 09-Jan-2008  matt sync with HEAD
 1.3.2.1 22-Dec-2007  matt file atomic_add.S was added on branch matt-armv6 on 2008-01-09 01:21:09 +0000
 1.4.2.2 04-Jun-2008  yamt sync with head
 1.4.2.1 18-May-2008  yamt sync with head.
 1.5.2.1 23-Jun-2008  wrstuden Sync w/ -current. 34 merge conflicts to follow.
 1.7.16.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.7.16.1 21-Apr-2020  martin Sync with HEAD
 1.8 06-Apr-2022  riastradh Nix trailing whitespace in files of membars, atomics, and lock stubs.

Will be touching many of these files soon for functional changes.

No functional change intended.
 1.7 18-Oct-2015  nakayama branches: 1.7.16;
Fix 64-bit atomic operations on 32-bit kernel and userland, and add
support for missing __sync_*_8 builtins.
 1.6 25-May-2008  chs enable profiling of assembly functions.
 1.5 28-Apr-2008  martin branches: 1.5.2;
Remove clause 3 and 4 from TNF licenses
 1.4 09-Apr-2008  nakayama branches: 1.4.2;
Use %xcc for 64-bit comparisons.
 1.3 22-Dec-2007  mrg branches: 1.3.2;
- provide 32 bit versions of the sparc v9 atomic ops.
- use them on sparc64 32 bit kernels.

fixes issues on 32 bit sparc64 kernels that were using sparc v8 ops.
tested by rjs@.
 1.2 28-Nov-2007  ad Pull in sparc64 atomic ops from the thorpej-atomic branch.
 1.1 27-Apr-2007  thorpej branches: 1.1.2;
file atomic_and.S was initially added on branch thorpej-atomic.
 1.1.2.1 27-Apr-2007  thorpej First cut at atomic ops for sparc64.
 1.3.2.2 09-Jan-2008  matt sync with HEAD
 1.3.2.1 22-Dec-2007  matt file atomic_and.S was added on branch matt-armv6 on 2008-01-09 01:21:10 +0000
 1.4.2.2 04-Jun-2008  yamt sync with head
 1.4.2.1 18-May-2008  yamt sync with head.
 1.5.2.1 23-Jun-2008  wrstuden Sync w/ -current. 34 merge conflicts to follow.
 1.7.16.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.7.16.1 21-Apr-2020  martin Sync with HEAD
 1.8 06-Apr-2022  riastradh Nix trailing whitespace in files of membars, atomics, and lock stubs.

Will be touching many of these files soon for functional changes.

No functional change intended.
 1.7 18-Oct-2015  nakayama branches: 1.7.16;
Fix 64-bit atomic operations on 32-bit kernel and userland, and add
support for missing __sync_*_8 builtins.
 1.6 25-May-2008  chs enable profiling of assembly functions.
 1.5 28-Apr-2008  martin branches: 1.5.2;
Remove clause 3 and 4 from TNF licenses
 1.4 10-Feb-2008  ad branches: 1.4.4;
Add atomic_cas_foo_ni().
 1.3 22-Dec-2007  mrg branches: 1.3.2;
- provide 32 bit versions of the sparc v9 atomic ops.
- use them on sparc64 32 bit kernels.

fixes issues on 32 bit sparc64 kernels that were using sparc v8 ops.
tested by rjs@.
 1.2 28-Nov-2007  ad Pull in sparc64 atomic ops from the thorpej-atomic branch.
 1.1 27-Apr-2007  thorpej branches: 1.1.2;
file atomic_cas.S was initially added on branch thorpej-atomic.
 1.1.2.1 27-Apr-2007  thorpej First cut at atomic ops for sparc64.
 1.3.2.3 23-Mar-2008  matt sync with HEAD
 1.3.2.2 09-Jan-2008  matt sync with HEAD
 1.3.2.1 22-Dec-2007  matt file atomic_cas.S was added on branch matt-armv6 on 2008-01-09 01:21:10 +0000
 1.4.4.2 04-Jun-2008  yamt sync with head
 1.4.4.1 18-May-2008  yamt sync with head.
 1.5.2.1 23-Jun-2008  wrstuden Sync w/ -current. 34 merge conflicts to follow.
 1.7.16.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.7.16.1 21-Apr-2020  martin Sync with HEAD
 1.9 06-Apr-2022  riastradh Nix trailing whitespace in files of membars, atomics, and lock stubs.

Will be touching many of these files soon for functional changes.

No functional change intended.
 1.8 18-Oct-2015  nakayama branches: 1.8.16;
Fix 64-bit atomic operations on 32-bit kernel and userland, and add
support for missing __sync_*_8 builtins.
 1.7 25-May-2008  chs enable profiling of assembly functions.
 1.6 28-Apr-2008  martin branches: 1.6.2;
Remove clause 3 and 4 from TNF licenses
 1.5 09-Apr-2008  nakayama branches: 1.5.2;
Use %xcc for 64-bit comparisons.
 1.4 22-Dec-2007  mrg branches: 1.4.2;
- provide 32 bit versions of the sparc v9 atomic ops.
- use them on sparc64 32 bit kernels.

fixes issues on 32 bit sparc64 kernels that were using sparc v8 ops.
tested by rjs@.
 1.3 30-Nov-2007  ad This is atomic_dec, not atomic_inc!
 1.2 28-Nov-2007  ad Pull in sparc64 atomic ops from the thorpej-atomic branch.
 1.1 27-Apr-2007  thorpej branches: 1.1.2;
file atomic_dec.S was initially added on branch thorpej-atomic.
 1.1.2.1 27-Apr-2007  thorpej First cut at atomic ops for sparc64.
 1.4.2.2 09-Jan-2008  matt sync with HEAD
 1.4.2.1 22-Dec-2007  matt file atomic_dec.S was added on branch matt-armv6 on 2008-01-09 01:21:11 +0000
 1.5.2.2 04-Jun-2008  yamt sync with head
 1.5.2.1 18-May-2008  yamt sync with head.
 1.6.2.1 23-Jun-2008  wrstuden Sync w/ -current. 34 merge conflicts to follow.
 1.8.16.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.8.16.1 21-Apr-2020  martin Sync with HEAD
 1.8 06-Apr-2022  riastradh Nix trailing whitespace in files of membars, atomics, and lock stubs.

Will be touching many of these files soon for functional changes.

No functional change intended.
 1.7 18-Oct-2015  nakayama branches: 1.7.16;
Fix 64-bit atomic operations on 32-bit kernel and userland, and add
support for missing __sync_*_8 builtins.
 1.6 25-May-2008  chs enable profiling of assembly functions.
 1.5 28-Apr-2008  martin branches: 1.5.2;
Remove clause 3 and 4 from TNF licenses
 1.4 09-Apr-2008  nakayama branches: 1.4.2;
Use %xcc for 64-bit comparisons.
 1.3 22-Dec-2007  mrg branches: 1.3.2;
- provide 32 bit versions of the sparc v9 atomic ops.
- use them on sparc64 32 bit kernels.

fixes issues on 32 bit sparc64 kernels that were using sparc v8 ops.
tested by rjs@.
 1.2 28-Nov-2007  ad Pull in sparc64 atomic ops from the thorpej-atomic branch.
 1.1 27-Apr-2007  thorpej branches: 1.1.2;
file atomic_inc.S was initially added on branch thorpej-atomic.
 1.1.2.1 27-Apr-2007  thorpej First cut at atomic ops for sparc64.
 1.3.2.2 09-Jan-2008  matt sync with HEAD
 1.3.2.1 22-Dec-2007  matt file atomic_inc.S was added on branch matt-armv6 on 2008-01-09 01:21:12 +0000
 1.4.2.2 04-Jun-2008  yamt sync with head
 1.4.2.1 18-May-2008  yamt sync with head.
 1.5.2.1 23-Jun-2008  wrstuden Sync w/ -current. 34 merge conflicts to follow.
 1.7.16.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.7.16.1 21-Apr-2020  martin Sync with HEAD
 1.8 06-Apr-2022  riastradh Nix trailing whitespace in files of membars, atomics, and lock stubs.

Will be touching many of these files soon for functional changes.

No functional change intended.
 1.7 18-Oct-2015  nakayama branches: 1.7.16;
Fix 64-bit atomic operations on 32-bit kernel and userland, and add
support for missing __sync_*_8 builtins.
 1.6 17-Jan-2011  joerg Move SPARC and SPARC64 to modern CPP. Update UPDATING note for this and
Alpha.
 1.5 28-Apr-2008  martin Remove clause 3 and 4 from TNF licenses
 1.4 22-Jan-2008  nakayama branches: 1.4.4;
Use __arch64__ instead of __sparc64__, since -D__sparc64__ is appended
to build option of sparc64 32-bit kernel.
 1.3 22-Dec-2007  mrg branches: 1.3.2;
- provide 32 bit versions of the sparc v9 atomic ops.
- use them on sparc64 32 bit kernels.

fixes issues on 32 bit sparc64 kernels that were using sparc v8 ops.
tested by rjs@.
 1.2 28-Nov-2007  ad Pull in sparc64 atomic ops from the thorpej-atomic branch.
 1.1 27-Apr-2007  thorpej branches: 1.1.2;
file atomic_op_asm.h was initially added on branch thorpej-atomic.
 1.1.2.1 27-Apr-2007  thorpej First cut at atomic ops for sparc64.
 1.3.2.3 23-Mar-2008  matt sync with HEAD
 1.3.2.2 09-Jan-2008  matt sync with HEAD
 1.3.2.1 22-Dec-2007  matt file atomic_op_asm.h was added on branch matt-armv6 on 2008-01-09 01:21:12 +0000
 1.4.4.1 18-May-2008  yamt sync with head.
 1.7.16.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.7.16.1 21-Apr-2020  martin Sync with HEAD
 1.9 06-Apr-2022  riastradh Nix trailing whitespace in files of membars, atomics, and lock stubs.

Will be touching many of these files soon for functional changes.

No functional change intended.
 1.8 18-Oct-2015  nakayama branches: 1.8.16;
Fix 64-bit atomic operations on 32-bit kernel and userland, and add
support for missing __sync_*_8 builtins.
 1.7 25-May-2008  chs enable profiling of assembly functions.
 1.6 28-Apr-2008  martin branches: 1.6.2;
Remove clause 3 and 4 from TNF licenses
 1.5 09-Apr-2008  nakayama branches: 1.5.2;
Use %xcc for 64-bit comparisons.
 1.4 12-Feb-2008  ad Correct a dodgy alias.
 1.3 22-Dec-2007  mrg branches: 1.3.2;
- provide 32 bit versions of the sparc v9 atomic ops.
- use them on sparc64 32 bit kernels.

fixes issues on 32 bit sparc64 kernels that were using sparc v8 ops.
tested by rjs@.
 1.2 28-Nov-2007  ad Pull in sparc64 atomic ops from the thorpej-atomic branch.
 1.1 27-Apr-2007  thorpej branches: 1.1.2;
file atomic_or.S was initially added on branch thorpej-atomic.
 1.1.2.1 27-Apr-2007  thorpej First cut at atomic ops for sparc64.
 1.3.2.3 23-Mar-2008  matt sync with HEAD
 1.3.2.2 09-Jan-2008  matt sync with HEAD
 1.3.2.1 22-Dec-2007  matt file atomic_or.S was added on branch matt-armv6 on 2008-01-09 01:21:12 +0000
 1.5.2.2 04-Jun-2008  yamt sync with head
 1.5.2.1 18-May-2008  yamt sync with head.
 1.6.2.1 23-Jun-2008  wrstuden Sync w/ -current. 34 merge conflicts to follow.
 1.8.16.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.8.16.1 21-Apr-2020  martin Sync with HEAD
 1.8 06-Apr-2022  riastradh Nix trailing whitespace in files of membars, atomics, and lock stubs.

Will be touching many of these files soon for functional changes.

No functional change intended.
 1.7 18-Oct-2015  nakayama branches: 1.7.16;
Fix 64-bit atomic operations on 32-bit kernel and userland, and add
support for missing __sync_*_8 builtins.
 1.6 25-May-2008  chs enable profiling of assembly functions.
 1.5 28-Apr-2008  martin branches: 1.5.2;
Remove clause 3 and 4 from TNF licenses
 1.4 09-Apr-2008  nakayama branches: 1.4.2;
Use %xcc for 64-bit comparisons.
 1.3 22-Dec-2007  mrg branches: 1.3.2;
- provide 32 bit versions of the sparc v9 atomic ops.
- use them on sparc64 32 bit kernels.

fixes issues on 32 bit sparc64 kernels that were using sparc v8 ops.
tested by rjs@.
 1.2 28-Nov-2007  ad Pull in sparc64 atomic ops from the thorpej-atomic branch.
 1.1 27-Apr-2007  thorpej branches: 1.1.2;
file atomic_swap.S was initially added on branch thorpej-atomic.
 1.1.2.1 27-Apr-2007  thorpej First cut at atomic ops for sparc64.
 1.3.2.2 09-Jan-2008  matt sync with HEAD
 1.3.2.1 22-Dec-2007  matt file atomic_swap.S was added on branch matt-armv6 on 2008-01-09 01:21:13 +0000
 1.4.2.2 04-Jun-2008  yamt sync with head
 1.4.2.1 18-May-2008  yamt sync with head.
 1.5.2.1 23-Jun-2008  wrstuden Sync w/ -current. 34 merge conflicts to follow.
 1.7.16.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.7.16.1 21-Apr-2020  martin Sync with HEAD
 1.10 06-Sep-2025  riastradh paravirt_membar_sync(9): New memory barrier.

For use in paravirtualized drivers which require store-before-load
ordering -- irrespective of whether the kernel is built for a single
processor, or whether the (virtual) machine is booted with a single
processor.

This is even required on architectures that don't even have a
store-before-load ordering barrier, like m68k; adding, e.g., a virtio
bus is _as if_ the architecture has been extended with relaxed memory
ordering when talking with that new bus. Such architectures need
some way to request the hypervisor enforce that ordering -- on m68k,
that's done by issuing a CASL instruction, which qemu maps to an
atomic r/m/w with sequential consistency ordering in the host.

PR kern/59618: occasional virtio block device lock ups/hangs
 1.9 09-Apr-2022  riastradh Introduce membar_acquire/release. Deprecate membar_enter/exit.

The names membar_enter/exit were unclear, and the documentation of
membar_enter has disagreed with the implementations on sparc,
powerpc, and even x86(!) for the entire time it has been in NetBSD.

The terms `acquire' and `release' are ubiquitous in the literature
today, and have been adopted in the C and C++ standards to mean
load-before-load/store and load/store-before-store, respectively,
which are exactly the orderings required by acquiring and releasing a
mutex, as well as other useful applications like decrementing a
reference count and then freeing the underlying object if it went to
zero.

Originally I proposed changing one word in the documentation for
membar_enter to make it load-before-load/store instead of
store-before-load/store, i.e., to make it an acquire barrier. I
proposed this on the grounds that

(a) all implementations guarantee load-before-load/store,
(b) some implementations fail to guarantee store-before-load/store,
and
(c) all uses in-tree assume load-before-load/store.

I verified parts (a) and (b) (except, for (a), powerpc didn't even
guarantee load-before-load/store -- isync isn't necessarily enough;
need lwsync in general -- but it _almost_ did, and it certainly didn't
guarantee store-before-load/store).

Part (c) might not be correct, however: under the mistaken assumption
that atomic-r/m/w then membar-w/rw is equivalent to atomic-r/m/w then
membar-r/rw, I only audited the cases of membar_enter that _aren't_
immediately after an atomic-r/m/w. All of those cases assume
load-before-load/store. But my assumption was wrong -- there are
cases of atomic-r/m/w then membar-w/rw that would be broken by
changing to atomic-r/m/w then membar-r/rw:

https://mail-index.netbsd.org/tech-kern/2022/03/29/msg028044.html

Furthermore, the name membar_enter has been adopted in other places
like OpenBSD where it actually does follow the documentation and
guarantee store-before-load/store, even if that order is not useful.
So the name membar_enter currently lives in a bad place where it
means either of two things -- r/rw or w/rw.

With this change, we deprecate membar_enter/exit, introduce
membar_acquire/release as better names for the useful pair (r/rw and
rw/w), and make sure the implementation of membar_enter guarantees
both what was documented _and_ what was implemented, making it an
alias for membar_sync.

While here, rework all of the membar_* definitions and aliases. The
new logic follows a rule to make it easier to audit:

membar_X is defined as an alias for membar_Y iff membar_X is
guaranteed by membar_Y.

The `no stronger than' relation is (the transitive closure of):

- membar_consumer (r/r) is guaranteed by membar_acquire (r/rw)
- membar_producer (w/w) is guaranteed by membar_release (rw/w)
- membar_acquire (r/rw) is guaranteed by membar_sync (rw/rw)
- membar_release (rw/w) is guaranteed by membar_sync (rw/rw)

And, for the deprecated membars:

- membar_enter (whether r/rw, w/rw, or rw/rw) is guaranteed by
membar_sync (rw/rw)
- membar_exit (rw/w) is guaranteed by membar_release (rw/w)

(membar_exit is identical to membar_release, but the name is
deprecated.)

Finally, while here, annotate some of the instructions with their
semantics. For powerpc, leave an essay with citations on the
unfortunate but -- as far as I can tell -- necessary decision to use
lwsync, not isync, for membar_acquire and membar_consumer.

Also add membar(3) and atomic(3) man page links.
 1.8 09-Apr-2022  riastradh sparc64/membar_ops: Upgrade membar_enter from R/RW to RW/RW.

This will be deprecated soon but let's avoid leaving rakes to trip on
with it arising from disagreement over the documentation (W/RW) and
implementation and usage (R/RW).
 1.7 09-Apr-2022  riastradh sparc64: Fix membar_sync by issuing membar #StoreLoad.

In TSO this is the only memory barrier ever needed, and somehow we
got this wrong and instead issued an unnecessary membar #LoadLoad --
not needed even in PSO let alone in TSO.

XXX Apparently we may run userland programs with PSO or RMO, in which
case all of these membars need fixing:

PSO RMO
membar_consumer nop membar #LoadLoad
membar_producer membar #StoreStore membar #StoreStore
membar_enter nop membar #LoadLoad|LoadStore
membar_exit membar #StoreStore membar #LoadStore|StoreStore
membar_sync membar #StoreLoad|StoreStore
membar #...everything...

But at least this fixes the TSO case in which we run the kernel.
Also I'm not sure there's any non-TSO hardware out there in practice.
 1.6 06-Apr-2022  riastradh Nix trailing whitespace in files of membars, atomics, and lock stubs.

Will be touching many of these files soon for functional changes.

No functional change intended.
 1.5 25-May-2008  chs branches: 1.5.62;
enable profiling of assembly functions.
 1.4 28-Apr-2008  martin branches: 1.4.2;
Remove clause 3 and 4 from TNF licenses
 1.3 12-Jan-2008  ad branches: 1.3.4;
Don't put membar in the delay slot. thorpej@ says it may be problematic.
 1.2 29-Nov-2007  ad branches: 1.2.4;
Fix the aliases.
 1.1 28-Nov-2007  ad Memory barriers for sparc64. XXX Can the 'membar' go in retl delay slot?
 1.2.4.3 23-Mar-2008  matt sync with HEAD
 1.2.4.2 09-Jan-2008  matt sync with HEAD
 1.2.4.1 29-Nov-2007  matt file membar_ops.S was added on branch matt-armv6 on 2008-01-09 01:21:13 +0000
 1.3.4.2 04-Jun-2008  yamt sync with head
 1.3.4.1 18-May-2008  yamt sync with head.
 1.4.2.1 23-Jun-2008  wrstuden Sync w/ -current. 34 merge conflicts to follow.
 1.5.62.2 21-Apr-2020  martin Ooops, restore accidently removed files from merge mishap
 1.5.62.1 21-Apr-2020  martin Sync with HEAD

RSS XML Feed