Home | History | Annotate | only in /src/lib/libc/atomic
History log of /src/lib/libc/atomic
RevisionDateAuthorComments
 1.5 09-Apr-2022  riastradh Introduce membar_acquire/release. Deprecate membar_enter/exit.

The names membar_enter/exit were unclear, and the documentation of
membar_enter has disagreed with the implementations on sparc,
powerpc, and even x86(!) for the entire time it has been in NetBSD.

The terms `acquire' and `release' are ubiquitous in the literature
today, and have been adopted in the C and C++ standards to mean
load-before-load/store and load/store-before-store, respectively,
which are exactly the orderings required by acquiring and releasing a
mutex, as well as other useful applications like decrementing a
reference count and then freeing the underlying object if it went to
zero.

Originally I proposed changing one word in the documentation for
membar_enter to make it load-before-load/store instead of
store-before-load/store, i.e., to make it an acquire barrier. I
proposed this on the grounds that

(a) all implementations guarantee load-before-load/store,
(b) some implementations fail to guarantee store-before-load/store,
and
(c) all uses in-tree assume load-before-load/store.

I verified parts (a) and (b) (except, for (a), powerpc didn't even
guarantee load-before-load/store -- isync isn't necessarily enough;
need lwsync in general -- but it _almost_ did, and it certainly didn't
guarantee store-before-load/store).

Part (c) might not be correct, however: under the mistaken assumption
that atomic-r/m/w then membar-w/rw is equivalent to atomic-r/m/w then
membar-r/rw, I only audited the cases of membar_enter that _aren't_
immediately after an atomic-r/m/w. All of those cases assume
load-before-load/store. But my assumption was wrong -- there are
cases of atomic-r/m/w then membar-w/rw that would be broken by
changing to atomic-r/m/w then membar-r/rw:

https://mail-index.netbsd.org/tech-kern/2022/03/29/msg028044.html

Furthermore, the name membar_enter has been adopted in other places
like OpenBSD where it actually does follow the documentation and
guarantee store-before-load/store, even if that order is not useful.
So the name membar_enter currently lives in a bad place where it
means either of two things -- r/rw or w/rw.

With this change, we deprecate membar_enter/exit, introduce
membar_acquire/release as better names for the useful pair (r/rw and
rw/w), and make sure the implementation of membar_enter guarantees
both what was documented _and_ what was implemented, making it an
alias for membar_sync.

While here, rework all of the membar_* definitions and aliases. The
new logic follows a rule to make it easier to audit:

membar_X is defined as an alias for membar_Y iff membar_X is
guaranteed by membar_Y.

The `no stronger than' relation is (the transitive closure of):

- membar_consumer (r/r) is guaranteed by membar_acquire (r/rw)
- membar_producer (w/w) is guaranteed by membar_release (rw/w)
- membar_acquire (r/rw) is guaranteed by membar_sync (rw/rw)
- membar_release (rw/w) is guaranteed by membar_sync (rw/rw)

And, for the deprecated membars:

- membar_enter (whether r/rw, w/rw, or rw/rw) is guaranteed by
membar_sync (rw/rw)
- membar_exit (rw/w) is guaranteed by membar_release (rw/w)

(membar_exit is identical to membar_release, but the name is
deprecated.)

Finally, while here, annotate some of the instructions with their
semantics. For powerpc, leave an essay with citations on the
unfortunate but -- as far as I can tell -- necessary decision to use
lwsync, not isync, for membar_acquire and membar_consumer.

Also add membar(3) and atomic(3) man page links.
 1.4 07-Dec-2019  riastradh Add missing MLINK for membar_datadep_consumer(3).

Detected by the indefatigable wizd.
 1.3 27-Aug-2016  dholland branches: 1.3.14;
Add MLINKS for atomic_add_nv, atomic_cas_ni, and so forth - otherwise
you have to remember that they're the same page as plain atomic_add and
atomic_cas, or remember one of the types that they're defined on.
PR 51033.
 1.2 12-Feb-2010  dyoung Link atomic_cas.3 to atomic_cas_{32,64,ptr,uint,ulong}_ni.3.
 1.1 23-Jun-2008  ad branches: 1.1.2; 1.1.8;
Install manual pages for the atomic ops.
 1.1.8.2 23-Jun-2008  ad Install manual pages for the atomic ops.
 1.1.8.1 23-Jun-2008  ad file Makefile.inc was added on branch christos-time_t on 2008-06-23 10:22:41 +0000
 1.1.2.2 18-Sep-2008  wrstuden Sync with wrstuden-revivesa-base-2.
 1.1.2.1 23-Jun-2008  wrstuden file Makefile.inc was added on branch wrstuden-revivesa on 2008-09-18 04:39:20 +0000
 1.3.14.1 08-Apr-2020  martin Merge changes from current as of 20200406
 1.1 23-Jun-2008  ad branches: 1.1.2; 1.1.8;
Install manual pages for the atomic ops.
 1.1.8.2 23-Jun-2008  ad Install manual pages for the atomic ops.
 1.1.8.1 23-Jun-2008  ad file atomic_add.3 was added on branch christos-time_t on 2008-06-23 10:22:41 +0000
 1.1.2.2 18-Sep-2008  wrstuden Sync with wrstuden-revivesa-base-2.
 1.1.2.1 23-Jun-2008  wrstuden file atomic_add.3 was added on branch wrstuden-revivesa on 2008-09-18 04:39:20 +0000
 1.2 16-Jun-2018  dholland atomic_and/atomic_or do bitwise ops, not logical ops.
 1.1 23-Jun-2008  ad branches: 1.1.2; 1.1.8; 1.1.62;
Install manual pages for the atomic ops.
 1.1.62.1 25-Jun-2018  pgoyette Sync with HEAD
 1.1.8.2 23-Jun-2008  ad Install manual pages for the atomic ops.
 1.1.8.1 23-Jun-2008  ad file atomic_and.3 was added on branch christos-time_t on 2008-06-23 10:22:41 +0000
 1.1.2.2 18-Sep-2008  wrstuden Sync with wrstuden-revivesa-base-2.
 1.1.2.1 23-Jun-2008  wrstuden file atomic_and.3 was added on branch wrstuden-revivesa on 2008-09-18 04:39:20 +0000
 1.5 02-Feb-2014  dholland bump date for previous
 1.4 02-Feb-2014  dholland Reorg second paragraph too, to group related info together.
 1.3 02-Feb-2014  dholland Rework description for clarity; prompted by chat comments from bad@.
 1.2 12-Feb-2010  dyoung branches: 1.2.6; 1.2.12;
With help from rmind@, describe the non-interlocked (*_ni) variants of
the standard atomic compare-and-swap operations. Tell some caveats.

Manual page links, *_ni.3 -> atomic_cas.3 are coming up after a
successful 'build.sh distribution'.
 1.1 23-Jun-2008  ad branches: 1.1.2; 1.1.8;
Install manual pages for the atomic ops.
 1.1.8.2 23-Jun-2008  ad Install manual pages for the atomic ops.
 1.1.8.1 23-Jun-2008  ad file atomic_cas.3 was added on branch christos-time_t on 2008-06-23 10:22:41 +0000
 1.1.2.2 18-Sep-2008  wrstuden Sync with wrstuden-revivesa-base-2.
 1.1.2.1 23-Jun-2008  wrstuden file atomic_cas.3 was added on branch wrstuden-revivesa on 2008-09-18 04:39:20 +0000
 1.2.12.1 20-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.2.6.1 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.1 23-Jun-2008  ad branches: 1.1.2; 1.1.8;
Install manual pages for the atomic ops.
 1.1.8.2 23-Jun-2008  ad Install manual pages for the atomic ops.
 1.1.8.1 23-Jun-2008  ad file atomic_dec.3 was added on branch christos-time_t on 2008-06-23 10:22:41 +0000
 1.1.2.2 18-Sep-2008  wrstuden Sync with wrstuden-revivesa-base-2.
 1.1.2.1 23-Jun-2008  wrstuden file atomic_dec.3 was added on branch wrstuden-revivesa on 2008-09-18 04:39:20 +0000
 1.1 23-Jun-2008  ad branches: 1.1.2; 1.1.8;
Install manual pages for the atomic ops.
 1.1.8.2 23-Jun-2008  ad Install manual pages for the atomic ops.
 1.1.8.1 23-Jun-2008  ad file atomic_inc.3 was added on branch christos-time_t on 2008-06-23 10:22:41 +0000
 1.1.2.2 18-Sep-2008  wrstuden Sync with wrstuden-revivesa-base-2.
 1.1.2.1 23-Jun-2008  wrstuden file atomic_inc.3 was added on branch wrstuden-revivesa on 2008-09-18 04:39:20 +0000
 1.9 06-Sep-2023  ad Node that aarch64 implements CAS.
 1.8 16-Jun-2018  dholland branches: 1.8.12;
atomic_and/atomic_or do bitwise ops, not logical ops.
 1.7 07-Feb-2018  martin branches: 1.7.2;
Bump date for previous
 1.6 07-Feb-2018  martin Fix typo, from Eitan Adler.
 1.5 14-Apr-2010  jruoho Bump date for previous. (How hard can it be to remember this.)
 1.4 14-Apr-2010  jruoho Use subtitles instead of a list.
 1.3 18-May-2009  wiz Punctuation nit. Remove superfluous .Pp.
 1.2 18-May-2009  wiz Fix typo in Dd argument.
 1.1 23-Jun-2008  ad branches: 1.1.2; 1.1.8;
Install manual pages for the atomic ops.
 1.1.8.2 23-Jun-2008  ad Install manual pages for the atomic ops.
 1.1.8.1 23-Jun-2008  ad file atomic_ops.3 was added on branch christos-time_t on 2008-06-23 10:22:41 +0000
 1.1.2.2 18-Sep-2008  wrstuden Sync with wrstuden-revivesa-base-2.
 1.1.2.1 23-Jun-2008  wrstuden file atomic_ops.3 was added on branch wrstuden-revivesa on 2008-09-18 04:39:20 +0000
 1.7.2.1 25-Jun-2018  pgoyette Sync with HEAD
 1.8.12.1 20-Jul-2024  martin Pull up following revision(s) (requested by rin in ticket #748):

lib/libc/atomic/atomic_ops.3: revision 1.9

Note that aarch64 implements CAS.
 1.2 16-Jun-2018  dholland atomic_and/atomic_or do bitwise ops, not logical ops.
 1.1 23-Jun-2008  ad branches: 1.1.2; 1.1.8; 1.1.62;
Install manual pages for the atomic ops.
 1.1.62.1 25-Jun-2018  pgoyette Sync with HEAD
 1.1.8.2 23-Jun-2008  ad Install manual pages for the atomic ops.
 1.1.8.1 23-Jun-2008  ad file atomic_or.3 was added on branch christos-time_t on 2008-06-23 10:22:41 +0000
 1.1.2.2 18-Sep-2008  wrstuden Sync with wrstuden-revivesa-base-2.
 1.1.2.1 23-Jun-2008  wrstuden file atomic_or.3 was added on branch wrstuden-revivesa on 2008-09-18 04:39:20 +0000
 1.1 23-Jun-2008  ad branches: 1.1.2; 1.1.8;
Install manual pages for the atomic ops.
 1.1.8.2 23-Jun-2008  ad Install manual pages for the atomic ops.
 1.1.8.1 23-Jun-2008  ad file atomic_swap.3 was added on branch christos-time_t on 2008-06-23 10:22:41 +0000
 1.1.2.2 18-Sep-2008  wrstuden Sync with wrstuden-revivesa-base-2.
 1.1.2.1 23-Jun-2008  wrstuden file atomic_swap.3 was added on branch wrstuden-revivesa on 2008-09-18 04:39:20 +0000
 1.10 09-Apr-2022  riastradh Introduce membar_acquire/release. Deprecate membar_enter/exit.

The names membar_enter/exit were unclear, and the documentation of
membar_enter has disagreed with the implementations on sparc,
powerpc, and even x86(!) for the entire time it has been in NetBSD.

The terms `acquire' and `release' are ubiquitous in the literature
today, and have been adopted in the C and C++ standards to mean
load-before-load/store and load/store-before-store, respectively,
which are exactly the orderings required by acquiring and releasing a
mutex, as well as other useful applications like decrementing a
reference count and then freeing the underlying object if it went to
zero.

Originally I proposed changing one word in the documentation for
membar_enter to make it load-before-load/store instead of
store-before-load/store, i.e., to make it an acquire barrier. I
proposed this on the grounds that

(a) all implementations guarantee load-before-load/store,
(b) some implementations fail to guarantee store-before-load/store,
and
(c) all uses in-tree assume load-before-load/store.

I verified parts (a) and (b) (except, for (a), powerpc didn't even
guarantee load-before-load/store -- isync isn't necessarily enough;
need lwsync in general -- but it _almost_ did, and it certainly didn't
guarantee store-before-load/store).

Part (c) might not be correct, however: under the mistaken assumption
that atomic-r/m/w then membar-w/rw is equivalent to atomic-r/m/w then
membar-r/rw, I only audited the cases of membar_enter that _aren't_
immediately after an atomic-r/m/w. All of those cases assume
load-before-load/store. But my assumption was wrong -- there are
cases of atomic-r/m/w then membar-w/rw that would be broken by
changing to atomic-r/m/w then membar-r/rw:

https://mail-index.netbsd.org/tech-kern/2022/03/29/msg028044.html

Furthermore, the name membar_enter has been adopted in other places
like OpenBSD where it actually does follow the documentation and
guarantee store-before-load/store, even if that order is not useful.
So the name membar_enter currently lives in a bad place where it
means either of two things -- r/rw or w/rw.

With this change, we deprecate membar_enter/exit, introduce
membar_acquire/release as better names for the useful pair (r/rw and
rw/w), and make sure the implementation of membar_enter guarantees
both what was documented _and_ what was implemented, making it an
alias for membar_sync.

While here, rework all of the membar_* definitions and aliases. The
new logic follows a rule to make it easier to audit:

membar_X is defined as an alias for membar_Y iff membar_X is
guaranteed by membar_Y.

The `no stronger than' relation is (the transitive closure of):

- membar_consumer (r/r) is guaranteed by membar_acquire (r/rw)
- membar_producer (w/w) is guaranteed by membar_release (rw/w)
- membar_acquire (r/rw) is guaranteed by membar_sync (rw/rw)
- membar_release (rw/w) is guaranteed by membar_sync (rw/rw)

And, for the deprecated membars:

- membar_enter (whether r/rw, w/rw, or rw/rw) is guaranteed by
membar_sync (rw/rw)
- membar_exit (rw/w) is guaranteed by membar_release (rw/w)

(membar_exit is identical to membar_release, but the name is
deprecated.)

Finally, while here, annotate some of the instructions with their
semantics. For powerpc, leave an essay with citations on the
unfortunate but -- as far as I can tell -- necessary decision to use
lwsync, not isync, for membar_acquire and membar_consumer.

Also add membar(3) and atomic(3) man page links.
 1.9 15-Feb-2022  riastradh membar_ops(3): xref bus_dma, bus_space in SEE ALSO section too
 1.8 09-Oct-2020  uwe Add italic correction to _Atomic-qualified.
 1.7 09-Oct-2020  riastradh Reword advice about when not to use membar_exit.

With help from skrll and pgoyette.

While here, change the example so it doesn't violate the advice just
given. Still not a great example -- abusing UINT_MAX for a reference
count is kinda sketchy, but it'll serve for now.
 1.6 03-Sep-2020  riastradh Update membar_ops(3) man page with examples and relation to C11.

Add exhortation to always always always document how membars come in
pairs for synchronization between two CPUs when you use them.
 1.5 24-Oct-2017  abhinav branches: 1.5.6;
Add membar_datadep_consumer to the NAME section
Remove Pp before Bl
 1.4 08-Jan-2015  riastradh Introduce membar_datadep_consumer.

Discussed briefly on tech-kern without objection:

https://mail-index.netbsd.org/tech-kern/2014/11/20/msg018054.html
https://mail-index.netbsd.org/tech-kern/2015/01/07/msg018326.html
 1.3 28-Apr-2011  wiz branches: 1.3.22;
Fix a typo.
 1.2 18-May-2009  wiz Fix typo in Dd argument. Remove superfluous .Pp.
 1.1 23-Jun-2008  ad branches: 1.1.2; 1.1.8;
Install manual pages for the atomic ops.
 1.1.8.2 23-Jun-2008  ad Install manual pages for the atomic ops.
 1.1.8.1 23-Jun-2008  ad file membar_ops.3 was added on branch christos-time_t on 2008-06-23 10:22:41 +0000
 1.1.2.2 18-Sep-2008  wrstuden Sync with wrstuden-revivesa-base-2.
 1.1.2.1 23-Jun-2008  wrstuden file membar_ops.3 was added on branch wrstuden-revivesa on 2008-09-18 04:39:20 +0000
 1.3.22.1 18-Mar-2015  snj Pull up following revision(s) (requested by riastradh in ticket #600):
common/lib/libc/arch/alpha/atomic/membar_ops.S: revision 1.7
lib/libc/atomic/membar_ops.3: revision 1.4
sys/arch/alpha/include/types.h: revision 1.50
sys/sys/atomic.h: revision 1.13
Introduce membar_datadep_consumer.
Discussed briefly on tech-kern without objection:
https://mail-index.netbsd.org/tech-kern/2014/11/20/msg018054.html
https://mail-index.netbsd.org/tech-kern/2015/01/07/msg018326.html
 1.5.6.1 13-Sep-2020  martin Pull up following revision(s) (requested by riastradh in ticket #1082):

share/man/man9/atomic_loadstore.9: revision 1.6
share/man/man9/ipi.9: revision 1.5
lib/libc/atomic/membar_ops.3: revision 1.6

Spell out acronyms in title for clarity.

Update membar_ops(3) man page with examples and relation to C11.

Add exhortation to always always always document how membars come in
pairs for synchronization between two CPUs when you use them.

atomic_load/store_* appeared in NetBSD 9, not 10.

Pullup preceded release of 9.0.

RSS XML Feed