Home | History | Annotate | Download | only in kern
History log of /src/sys/kern/vfs_cwd.c
RevisionDateAuthorComments
 1.12  07-Dec-2024  riastradh vfs(9): Sprinkle KNF.

No functional change intended.
 1.11  23-Sep-2023  ad branches: 1.11.6;
Repply this change with a couple of bugs fixed:

- Do away with separate pool_cache for some kernel objects that have no special
requirements and use the general purpose allocator instead. On one of my
test systems this makes for a small (~1%) but repeatable reduction in system
time during builds presumably because it decreases the kernel's cache /
memory bandwidth footprint a little.
- vfs_lockf: cache a pointer to the uidinfo and put mutex in the data segment.
 1.10  12-Sep-2023  ad Back out recent change to replace pool_cache with then general allocator.
Will return to this when I have time again.
 1.9  10-Sep-2023  ad - Do away with separate pool_cache for some kernel objects that have no special
requirements and use the general purpose allocator instead. On one of my
test systems this makes for a small (~1%) but repeatable reduction in system
time during builds presumably because it decreases the kernel's cache /
memory bandwidth footprint a little.
- vfs_lockf: cache a pointer to the uidinfo and put mutex in the data segment.
 1.8  09-Apr-2022  riastradh sys: Use membar_release/acquire around reference drop.

This just goes through my recent reference count membar audit and
changes membar_exit to membar_release and membar_enter to
membar_acquire -- this should make everything cheaper on most CPUs
without hurting correctness, because membar_acquire is generally
cheaper than membar_enter.
 1.7  12-Mar-2022  riastradh sys: Membar audit around reference count releases.

If two threads are using an object that is freed when the reference
count goes to zero, we need to ensure that all memory operations
related to the object happen before freeing the object.

Using an atomic_dec_uint_nv(&refcnt) == 0 ensures that only one
thread takes responsibility for freeing, but it's not enough to
ensure that the other thread's memory operations happen before the
freeing.

Consider:

Thread A Thread B
obj->foo = 42; obj->baz = 73;
mumble(&obj->bar); grumble(&obj->quux);
/* membar_exit(); */ /* membar_exit(); */
atomic_dec -- not last atomic_dec -- last
/* membar_enter(); */
KASSERT(invariant(obj->foo,
obj->bar));
free_stuff(obj);

The memory barriers ensure that

obj->foo = 42;
mumble(&obj->bar);

in thread A happens before

KASSERT(invariant(obj->foo, obj->bar));
free_stuff(obj);

in thread B. Without them, this ordering is not guaranteed.

So in general it is necessary to do

membar_exit();
if (atomic_dec_uint_nv(&obj->refcnt) != 0)
return;
membar_enter();

to release a reference, for the `last one out hit the lights' style
of reference counting. (This is in contrast to the style where one
thread blocks new references and then waits under a lock for existing
ones to drain with a condvar -- no membar needed thanks to mutex(9).)

I searched for atomic_dec to find all these. Obviously we ought to
have a better abstraction for this because there's so much copypasta.
This is a stop-gap measure to fix actual bugs until we have that. It
would be nice if an abstraction could gracefully handle the different
styles of reference counting in use -- some years ago I drafted an
API for this, but making it cover everything got a little out of hand
(particularly with struct vnode::v_usecount) and I ended up setting
it aside to work on psref/localcount instead for better scalability.

I got bored of adding #ifdef __HAVE_ATOMIC_AS_MEMBAR everywhere, so I
only put it on things that look performance-critical on 5sec review.
We should really adopt membar_enter_preatomic/membar_exit_postatomic
or something (except they are applicable only to atomic r/m/w, not to
atomic_load/store_*, making the naming annoying) and get rid of all
the ifdefs.
 1.6  21-Apr-2020  ad Revert the changes made in February to make cwdinfo use mostly lockless,
which relied on taking extra vnode refs.

Having benchmarked various experimental changes over the past few months it
seems that it's better to avoid vnode refs as much as possible. cwdi_lock
as a RW lock already did that to some extent for getcwd() and will permit
the same for namei() too.
 1.5  23-Feb-2020  ad branches: 1.5.4;
Merge from ad-namecache:

- Have a stab at clustering the members of vnode_t and vnode_impl_t in a
more cache-conscious way. With that done, go back to adjusting v_usecount
with atomics and keep vi_lock directly in vnode_impl_t (saves KVA).

- Allow VOP_LOCK(LK_NONE) for the benefit of VFS_VGET() and VFS_ROOT().
Make sure LK_UPGRADE always comes with LK_NOWAIT.

- Make cwdinfo use mostly lockless.
 1.4  15-Feb-2011  pooka branches: 1.4.56; 1.4.62;
Support FD_CLOEXEC in rump kernels.
 1.3  08-Jan-2010  pooka branches: 1.3.4; 1.3.6; 1.3.8;
The VATTR_NULL/VREF/VHOLD/HOLDRELE() macros lost their will to live
years ago when the kernel was modified to not alter ABI based on
DIAGNOSTIC, and now just call the respective function interfaces
(in lowercase). Plenty of mix'n match upper/lowercase has creeped
into the tree since then. Nuke the macros and convert all callsites
to lowercase.

no functional change
 1.2  24-Sep-2009  yamt cwdinit: whitespace fix. no functional changes.
 1.1  18-Nov-2008  pooka branches: 1.1.4; 1.1.6; 1.1.8; 1.1.12;
cwd is logically a vfs concept, so take it out from the bosom of
kern_descrip and into vfs_cwd. No functional change.
 1.1.12.3  11-Mar-2010  yamt sync with head
 1.1.12.2  04-May-2009  yamt sync with head.
 1.1.12.1  18-Nov-2008  yamt file vfs_cwd.c was added on branch yamt-nfs-mp on 2009-05-04 08:13:49 +0000
 1.1.8.2  19-Jan-2009  skrll Sync with HEAD.
 1.1.8.1  18-Nov-2008  skrll file vfs_cwd.c was added on branch nick-hppapmap on 2009-01-19 13:19:40 +0000
 1.1.6.2  17-Jan-2009  mjf Sync with HEAD.
 1.1.6.1  18-Nov-2008  mjf file vfs_cwd.c was added on branch mjf-devfs2 on 2009-01-17 13:29:20 +0000
 1.1.4.2  13-Dec-2008  haad Update haad-dm branch to haad-dm-base2.
 1.1.4.1  18-Nov-2008  haad file vfs_cwd.c was added on branch haad-dm on 2008-12-13 01:15:09 +0000
 1.3.8.1  17-Feb-2011  bouyer Sync with HEAD
 1.3.6.1  06-Jun-2011  jruoho Sync with HEAD.
 1.3.4.1  05-Mar-2011  rmind sync with head
 1.4.62.3  29-Feb-2020  ad Sync with head.
 1.4.62.2  25-Jan-2020  ad Add a missing membar to previous.
 1.4.62.1  25-Jan-2020  ad Make cwdinfo use mostly lockless, and largely hide the details in vfs_cwd.c.
 1.4.56.1  08-Apr-2020  martin Merge changes from current as of 20200406
 1.5.4.1  25-Apr-2020  bouyer Sync with bouyer-xenpvh-base2 (HEAD)
 1.11.6.1  02-Aug-2025  perseant Sync with HEAD

RSS XML Feed