Home | History | Annotate | Download | only in lfs
History log of /src/sys/ufs/lfs/lfs_vnops.c
RevisionDateAuthorComments
 1.344  01-Oct-2025  perseant Align case labels with 8-character tab stops. No functional change.
 1.343  17-Sep-2025  perseant Add working in-kernel roll forward.
 1.342  06-Sep-2025  perseant Lock the vnode before calling lfs_set_dirop, to meet the conditions of
the assertion. Fixes a regression introduced in rev 1.341.
 1.341  05-Sep-2025  perseant Protect the changed link count of the linked vnode with {,UN}MARK_DIROP
in lfs_link(). Necessary for roll-forward.
 1.340  20-Oct-2021  thorpej Overhaul of the EVFILT_VNODE kevent(2) filter:

- Centralize vnode kevent handling in the VOP_*() wrappers, rather than
forcing each individual file system to deal with it (except VOP_RENAME(),
because VOP_RENAME() is a mess and we currently have 2 different ways
of handling it; at least it's reasonably well-centralized in the "new"
way).
- Add support for NOTE_OPEN, NOTE_CLOSE, NOTE_CLOSE_WRITE, and NOTE_READ,
compatible with the same events in FreeBSD.
- Track which kevent notifications clients are interested in receiving
to avoid doing work for events no one cares about (avoiding, e.g.
taking locks and traversing the klist to send a NOTE_WRITE when
someone is merely watching for a file to be deleted, for example).

In support of the above:

- Add support in vnode_if.sh for specifying PRE- and POST-op handlers,
to be invoked before and after vop_pre() and vop_post(), respectively.
Basic idea from FreeBSD, but implemented differently.
- Add support in vnode_if.sh for specifying CONTEXT fields in the
vop_*_args structures. These context fields are used to convey information
between the file system VOP function and the VOP wrapper, but do not
occupy an argument slot in the VOP_*() call itself. These context fields
are initialized and subsequently interpreted by PRE- and POST-op handlers.
- Version VOP_REMOVE(), uses the a context field for the file system to report
back the resulting link count of the target vnode. Return this in tmpfs,
udf, nfs, chfs, ext2fs, lfs, and ufs.

NetBSD 9.99.92.
 1.339  18-Jul-2021  dholland Abolish all the silly indirection macros for initializing vnode ops tables.

These are things of the form #define foofs_op genfs_op, or #define
foofs_op genfs_eopnotsupp, or similar. They serve no purpose besides
obfuscation, and have gotten cutpasted all over everywhere.
 1.338  18-Jul-2021  dholland Use macros for the canned parts of device and fifo vnode op tables.

Add GENFS_SPECOP_ENTRIES and GENFS_FIFOOP_ENTRIES macros that contain
the portion of the vnode ops table declaration that is
(conservatively) the same in every fs. Use these in every fs that
supports devices and/or fifos with separate ops tables.

Note that ptyfs works differently (it has one type of vnode with
open-coded dispatch to the specfs code, which I haven't changed in
this commit) and rump/librump/rumpvfs/rumpfs.c has an indirect dynamic
dispatch that already does more or less the same thing, which I also
haven't changed.

Also note that this anticipates a few bits in the next changeset here
and there, and adds missing but unreachable calls in some cases (e.g.
most fses weren't defining whiteout on devices and fifos, but it isn't
reachable there), and it changes parsepath on devices and fifos to
genfs_badop from genfs_parsepath (but it's not reachable there
either).

It appears that devices in kernfs were missing kqfilter, so it's
possible that if you try to use kqueue on /kern/rootdev that it'll
explode.

And finally note that the ops declaration tables aren't
order-dependent. (Other than vop_default_desc has to come first.)
Otherwise this wouldn't work.
 1.337  29-Jun-2021  dholland - Add a new vnode op: VOP_PARSEPATH.
- Move namei_getcomponent to genfs_vnops.c and call it genfs_parsepath.
- Add a parsepath entry to every vnode ops table.

VOP_PARSEPATH takes a directory vnode to be searched and a complete
following path and chooses how much of that path to consume. To begin
with, all parsepath calls are genfs_parsepath, which locates the first
'/' as always.

Note that the call doesn't take the whole struct componentname, only
the string. The other bits of struct componentname should not be
needed and there's no reason to cause potential complications by
exposing them.
 1.336  05-Sep-2020  riastradh branches: 1.336.6;
Round of uvm.h cleanup.

The poorly named uvm.h is generally supposed to be for uvm-internal
users only.

- Narrow it to files that actually need it -- mostly files that need
to query whether curlwp is the pagedaemon, which should maybe be
exposed by an external header.

- Use uvm_extern.h where feasible and uvm_*.h for things not exposed
by it. We should split up uvm_extern.h but this will serve for now
to reduce the uvm.h dependencies.

- Use uvm_stat.h and #ifdef UVMHIST uvm.h for files that use
UVMHIST(ubchist), since ubchist is declared in uvm.h but the
reference evaporates if UVMHIST is not defined, so we reduce header
file dependencies.

- Make uvm_device.h and uvm_swap.h independently includable while
here.

ok chs@
 1.335  05-Sep-2020  riastradh Revert "ufs: Prevent mkdir from choking on deleted directories."

This change made no sense and should not have been committed.
 1.334  05-Sep-2020  riastradh ufs: Prevent mkdir from choking on deleted directories.

Fix some missing uvm_vnp_setsize in screw cases while here.
 1.333  16-May-2020  christos Add ACL support for FFS. From FreeBSD.
 1.332  13-Apr-2020  ad Replace most uses of vp->v_usecount with a call to vrefcnt(vp), a function
that hides the details and does atomic_load_relaxed(). Signature matches
FreeBSD.
 1.331  23-Feb-2020  ad branches: 1.331.4;
UVM locking changes, proposed on tech-kern:

- Change the lock on uvm_object, vm_amap and vm_anon to be a RW lock.
- Break v_interlock and vmobjlock apart. v_interlock remains a mutex.
- Do partial PV list locking in the x86 pmap. Others to follow later.
 1.330  23-Feb-2020  riastradh Fix missing <sys/evcnt.h> by removing the evcnts instead.

Just wanted to confirm that a race might happen, and indeed it did.
These serve little diagnostic value otherwise.
 1.329  23-Feb-2020  riastradh Take a reference and fix assertions in lfs_flush_dirops.

Fixes panic:

KASSERT((ip->i_state & IN_ADIROP) == 0) at lfs_vnops.c:1670
lfs_flush_dirops
lfs_check
lfs_setattr
VOP_SETATTR
change_mode
sys_fchmod
syscall

This assertion -- and the assertion that vp->v_uflag has VU_DIROP set
-- is valid only until we release lfs_lock, because we may race with
lfs_unmark_dirop which will remove the nodes and change the flags.

Further, vp itself is valid only as long as it is referenced, which it
is as long as it's on the dchain, but lfs_unmark_dirop drops the
dchain's reference.
 1.328  23-Feb-2020  riastradh Change some cheap KDASSERT into KASSERT.
 1.327  23-Feb-2020  riastradh Assert lfs_writer where I think we can now prove it.
 1.326  23-Feb-2020  riastradh Use a marker node to iterate lfs_dchainhd / i_lfs_dchain.

I believe elements can be removed while the lock is dropped,
including the next node we're hanging on to.
 1.325  18-Sep-2019  christos branches: 1.325.2;
Add newly created vnodes to the namei cache. The rest of the filesystems
already did that (or they don't support writing). Discussed in tech-kern.
 1.324  20-Jun-2019  christos branches: 1.324.2;
unifdef -DLFS_READWRITE ulfs_readwrite.c
 1.323  01-Jan-2019  hannken Add "void *extra" argument to vcache_new() so a file system may
pass more information about the file to create.

Welcome to 8.99.30
 1.322  11-Aug-2018  zafer In lfs_mkdir fix wrong return path in case of EMLINK which causes a panic. Also, check earlier before setting up dirop.
 1.321  20-Aug-2017  maya branches: 1.321.2; 1.321.4;
Fix typo in comment
 1.320  19-Aug-2017  maya Not much point doing anything after a panic call
 1.319  19-Aug-2017  maya Consistently use {,UN}MARK_VNODE macros rather than function calls.
 1.318  26-Jul-2017  maya change lfs_nextsegsleep and lfs_allclean_wakeup to use condvar

XXX had to use lfs_lock in lfs_segwait, removed kernel_lock, is this
appropriate?
 1.317  10-Jun-2017  maya Rename i_flag to i_state.

The similarity to i_flags has previously caused errors.
 1.316  05-Jun-2017  maya Correct confusion between i_flag and i_flags
These will have to be renamed.

Spotted by Riastradh, thanks!
 1.315  26-May-2017  riastradh branches: 1.315.2;
Make VOP_RECLAIM do the last unlock of the vnode.

VOP_RECLAIM naturally has exclusive access to the vnode, so having it
locked on entry is not strictly necessary -- but it means if there
are any final operations that must be done on the vnode, such as
ffs_update, requiring exclusive access to it, we can now kassert that
the vnode is locked in those operations.

We can't just have the caller release the last lock because some file
systems don't use genfs_lock, and require the vnode to remain valid
for VOP_UNLOCK to work, notably unionfs.
 1.314  26-Apr-2017  riastradh Change VOP_REMOVE and VOP_RMDIR to preserve lock/ref on dvp.

No change to vp -- the plan is to replace the node by the
componentname in the vop parameters, and let all directory vops do
lookups internally.

Proposed on tech-kern with no objections:
https://mail-index.netbsd.org/tech-kern/2017/04/17/msg021825.html
 1.313  11-Apr-2017  riastradh Make VOP_INACTIVE preserve vnode lock on return.

Discussed on tech-kern:
https://mail-index.netbsd.org/tech-kern/2017/04/01/msg021751.html

Ride 7.99.68, a bumpy bus of incremental vfs improvements!
 1.312  11-Apr-2017  riastradh Fix non-DIAGNOSTIC build by using vp outside KASSERT too.
 1.311  11-Apr-2017  riastradh Sprinkle lock ownership assertions.
 1.310  01-Apr-2017  maya Switch lfs_writer_daemon to use condvar instead of mtsleep.
track thread existence with struct lwp instead of pid + lid,
it's more useful from ddb.
 1.309  01-Apr-2017  maya switch lfs_dirops to condvar (from mtsleep)
 1.308  01-Apr-2017  maya switch lfs_sleepers to condvar (from mtsleep)
 1.307  30-Mar-2017  hannken Remove now redundant calls to fstrans_start()/fstrans_done().

Add fstrans_start()/fstrans_done() to lfs_putpages().
 1.306  16-Mar-2017  maya actually cast to unsigned long long and use %llu. certainly not use hex (oops)
suggested by dh
 1.305  15-Mar-2017  maya print inode number in an assert I keep hitting and the adjacent one.
use PRIx64 for printing inode number elsewhere.
 1.304  13-Jul-2016  maya branches: 1.304.2; 1.304.4;
Fix a deadlock

ok dholland@
 1.303  20-Jun-2016  dholland In lfs_mknod, don't release dvp until done with it. This was exposed a
while back when I removed a sketchy preprocessor macro scheme, but I'd
left it the way it was at the time and marked it for later. Now I
guess it's later.

Also don't randomly use both dvp and ap->a_dvp; they're the same, so
pick one and stick to it.
 1.302  20-Jun-2016  dholland One more batch of already-synced ufs changes:

ufs_extern.h 1.79 is equivalent to ulfs_extern.h 1.14
ufsmount.h 1.43 is (roughly) equivalent to lfs_extern.h 1.102
ufs_inode.c 1.94 does not apply to lfs
ufs_inode.c 1.95 does not apply to lfs either
ufs_readwrite.c 1.108 is equivalent to ulfs_readwrite.c 1.8
ufs_readwrite.c 1.109 is equivalent to ulfs_readwrite.c 1.9
ufs_readwrite.c 1.110 is equivalent to ulfs_readwrite.c 1.10
ufs_readwrite.c 1.111 does not apply to lfs
ufs_readwrite.c 1.112 is equivalent to ulfs_readwrite.c 1.11
ufs_readwrite.c 1.113 is equivalent to ulfs_readwrite.c 1.13
ufs_readwrite.c 1.114 is equivalent to ulfs_readwrite.c 1.14
ufs_readwrite.c 1.115 is equivalent to ulfs_readwrite.c 1.15
ufs_readwrite.c 1.116-1.118 does not apply to lfs
ufs_readwrite.c 1.119-1.120 are equivalent to ulfs_readwrite.c 1.16
ufs_rename.c 1.12 is equivalent to lfs_rename.c 1.8
ufs_vnops.c 1.226 is equivalent to ulfs_vnops.c 1.22 and lfs_vnops.c 1.270
ufs_vnops.c 1.227 is equivalent to ulfs_vnops.c 1.23
ufs_vnops.c 1.228-1.229 are equivalent to ulfs_vnops.c 1.24
ufs_vnops.c 1.230 is equivalent to ulfs_vnops.c 1.25 and lfs_vnops.c 1.271
ufs_vnops.c 1.231 originated in lfs
ufs_vnops.c 1.232 does not apply to lfs
 1.301  20-Jun-2016  dholland With the previous we seem to have the changes from -r1.225 of ufs_vnops.c.
(as that was stuff from moving ffs to the new vcache and lfs has also been
moved, this is not surprising)
 1.300  20-Jun-2016  dholland ulfs_makeinode -> lfs_makeinode
 1.299  20-Jun-2016  dholland Merge (effectively) -r1.78 of ufs_extern.h: shift ulfs_makeinode to
lfs_vnops.c and make it file-static there, as that's the only place
it's used.
 1.298  20-Jun-2016  dholland Note more already-merged versions:

inode.h 1.68 is subsumed by ulfs_inode.h 1.19
inode.h 1.69-1.72 do not apply to lfs
ufs_extern.h 1.74 was covered when lfs was moved to the new vnode cache
ufs_extern.h 1.75 is equivalent to ulfs_extern.h 1.13
ufs_extern.h 1.76-1.77 do not apply to lfs
ufsmount.h 1.42 does not apply to lfs
ufs_inode.c 1.90 is subsumed by ulfs_inode.c 1.10
ufs_inode.c 1.91-1.92 do not apply to lfs
ufs_lookup.c 1.130 is subsumed by ulfs_lookup.c 1.24
ufs_lookup.c 1.131 is equivalent to ulfs_lookup.c 1.20
ufs_lookup.c 1.132 is equivalent to ulfs_lookup.c 1.21
ufs_lookup.c 1.133 is equivalent to ulfs_lookup.c 1.22
ufs_lookup.c 1.134 is equivalent to ulfs_lookup.c 1.23
ufs_lookup.c 1.135 is equivalent to ulfs_lookup.c 1.25
ufs_quota2.c 1.38 is equivalent to ulfs_quota2.c 1.17
ufs_quota2.c 1.39 is equivalent to ulfs_quota2.c 1.16
ufs_quota2.c 1.40 is equivalent to ulfs_quota2.c 1.18
ufs_vfsops.c 1.53 is subsumed by lfs_vfsops.c 1.324
ufs_vfsops.c 1.54 is subsumed by lfs_vfsops.c 1.324
ufs_vnops.c 1.223-1.224 do not apply to lfs
 1.297  20-Jun-2016  dholland More already-merged or equivalent changes:

ufs_dirhash.c 1.36 corresponds to ulfs_dirhash.c 1.8
ufs_extattr.c 1.43 corresponds to ulfs_extattr.c 1.7
ufs_lookup.c 1.126 does not apply to lfs
ufs_lookup.c 1.127 we already have
ufs_lookup.c 1.128 does not apply to lfs
ufs_lookup.c 1.129 corresponds to ulfs_lookup.c 1.19
ufs_quota1.c 1.19 corresponds to ulfs_quota1.c 1.7
ufs_quota1.c 1.20 corresponds to ulfs_quota1.c 1.8
ufs_quota2.c 1.36 we have equivalent changes for
ufs_rename.c 1.9 corresponds to lfs_rename.c 1.5
ufs_rename.c 1.10 corresponds to lfs_rename.c 1.6
ufs_vnops.c 1.219 corresponds to lfs_vnops.c 1.260 and ulfs_vnops.c 1.19
ufs_vnops.c 1.220 corresponds to lfs_vnops.c 1.261 and ulfs_vnops.c 1.20
ufs_vnops.c 1.221 was superseded by later changes
ufs_vnops.c 1.222 got fixed independently in lfs
 1.296  19-Jun-2016  dholland we already have ufs_lookup.c 1.125 and ufs_vnops.c 1.218.
 1.295  19-Jun-2016  dholland missed one
(probably this should be tracked in some way other than pasting rcsid
comments, but it's what we've got)
 1.294  19-Jun-2016  dholland Merge -r1.216 of ufs_vnops.c: comments about maxsymlinklen handling
 1.293  21-Sep-2015  dholland Add 64-bit directory entry structures, and adjust accessors accordingly.

The LFS64 directory entry has a 64-bit inode number. This is stored as
two 32-bit values to avoid inducing 64-bit alignment requirements.

The exposed type for manipulating directory entries is now
LFS_DIRHEADER, following the same convention as e.g. IFILE and SEGUSE.
(But with LFS_ on it, because.)
 1.292  21-Sep-2015  dholland Oops; LFS_DIRECTSIZ() is going to need the fs as an argument.

Also, it turns out that dirhash needs a compile-time-constant version
of LFS_DIRECTSIZ(LFS_MAXNAMLEN+1), independent of 64-vs-32, so create
LFS_MAXDIRENTRYSIZE for this. Sigh.
 1.291  20-Sep-2015  dholland Clean up struct lfs_dirtemplate.
 1.290  15-Sep-2015  dholland Kill off ulfs_makedirentry; just pass the data to ulfs_direnter instead.
For now, move one copy of the code that allocates and fills in a
temporary struct lfs_direct to the top of ulfs_direnter; but it should
go away shortly.
 1.289  01-Sep-2015  dholland Add new accessors for the d_type and d_namlen fields of struct lfs_direct.
Napalm the old byteswap access logic for these.
 1.288  01-Sep-2015  dholland Use the lfs dinode accessors in place of the ufs-derived ones.
(Mostly.)

The ufs-derived ones are fake structure member macros, which are gross
and not very safe. Also, it seems that a lot of places in the lfs code
were using the ffsv1 branch of them unconditionally, and this way it's
guaranteed all those places have been updated.

Found while doing this: for non-devices, have getattr produce NODEV
in the rdev field instead of leaking the address of the first direct
block.
 1.287  19-Aug-2015  dholland Part two of dinodes; use the same union everywhere.
(previously the ufs-derived code had things set up slightly different)

Remove a bunch of associated mess.
 1.286  12-Aug-2015  dholland Hack up dinode usage to be 64 vs. 32 as needed. Part 1.

(This part changes the native lfs code; the ufs-derived code already
has 64 vs. 32 logic, but as aspects of it are unsafe, and don't
entirely interoperate cleanly with the lfs 64/32 stuff, pass 2 will be
rehashing that.)
 1.285  12-Aug-2015  dholland Make 32-bit and 64-bit versions of SEGSUM.
Also fix some of the FINFO handling as it's closely entangled.
 1.284  12-Aug-2015  dholland Make 32-bit and 64-bit versions of CLEANERINFO.

XXX: while this is written to disk, it seems like much of it would
XXX: be better set up as a commpage shared with the cleaner.
 1.283  12-Aug-2015  dholland Widen several of the fields of BLOCK_INFO to 64 bits.

Keep the old BLOCK_INFO as BLOCK_INFO_70, and version the fcntls that
use it.

Note that BLOCK_INFO_70 has 64-bit padding issues so that it's
different on 32-bit and 64-bit machines. This has been fixed. However,
BLOCK_INFO also contains a pointer, so compat32 stuff for 32-on-64 is
still needed and doesn't currently exist.
 1.282  12-Aug-2015  dholland Move the security checks for lfs_bmapv/lfs_markv into those functions.
(instead of the system call entry points)

Avoids duplication.

While touching these, pass the lwp around instead of the proc -- the
latter was there for no other reason than because once upon a time
struct proc was the first argument of all syscalls.

(For that matter, why not just use curlwp instead of passing it around
all over the place? The cost of passing it to every syscall probably
exceeds the cost of loading it from curcpu, even on machines where
it's not just kept in a register all the time.)
 1.281  03-Aug-2015  dholland Simplify some leftover code and remove some old assertions.

Last year when I killed off some evil dirop-related macros, I added
these assertions because if the things they asserted weren't true we'd
be leaking vnodes. Well, it seems that the code at the time did leak
vnodes, so certain failure cases (e.g. mkdir with disk full) would
assert. Nobody apparently tripped on this in the past fourteen months,
until I broke balloc so it always failed (unrelatedly) while working
on some LFS64 changes.

However, the vnode leak has since been removed by hannken@ as part of
the vnode cache changes, so the assertions are now superfluous;
instead, just make sure *vpp gets nulled on failure, and don't worry
about whether or not VU_DIROP is set as it shouldn't matter any more.

XXX: there's still a lot of gratuitous pointer aliasing in here that
should be tidied away.
 1.280  02-Aug-2015  dholland lfs_cleanint[] in the in-memory superblock needs to have 64-bit entries.
 1.279  02-Aug-2015  dholland Make i_eff_nblks in the in-memory inode 64 bits wide.
 1.278  28-Jul-2015  dholland Add a new lfs header file: lfs_accessors.h.

This contains all the accessor functions and macros out of lfs.h.
Add an include of lfs_accessors.h after all uses of lfs.h... except
for code that wants to define its own struct lfs-alike that the
accessors are supposed to play along with. For these, set STRUCT_LFS
and include lfs_accessors.h after the necessary structure has been
defined, so that lfs_accessors.h can emit functions in terms of it.
 1.277  26-Jul-2015  hannken lfs_flush_pchain: replace vget() with vcache_get().
 1.276  25-Jul-2015  martin Use accessors in DEBUG and DIAGNOSTIC code as well
 1.275  24-Jul-2015  dholland More lfs superblock accessors.
(This changes the rest of the code over; all the accessors were
already added.)

The difference between this commit and the previous one is arbitrary,
but the previous one passed the regression tests on its own so I'm
keeping it separate to help with any bisections that might be needed
in the future.
 1.274  24-Jul-2015  dholland Switch to accessor functions for elements of the LFS on-disk
superblock. This will allow switching between 32/64 bit forms on the
fly; it will also allow handling LFS_EI reasonably tidily. (That
currently doesn't work on the superblock.)

It also gets rid of cpp abuse in the form of fake structure member
macros.

Also, instead of doing sleep/wakeup on &lfs_avail and &lfs_nextseg
inside the on-disk superblock, add extra elements to the in-memory
struct lfs for this. (XXX: these should be changed to condvars, but
not right now)

XXX: this migrates a structure needed by the lfs code in libsa (struct
salfs) into lfs.h, where it doesn't belong, but for the time being
this is necessary in order to allow the accessors (and the various
lfs macros and other goop that relies on them) to compile.
 1.273  07-Jun-2015  hannken Fix copy and paste errors from last commits.
- Kernel i386/ALL and amd64/ALL compile again.
- Resolves CID 1304138 (DEADCODE) and 1304139 (IDENTICAL_BRANCHES).
 1.272  31-May-2015  hannken Change lfs from hash table to vcache.

- Change lfs_valloc() to return an inode number and version instead of
a vnode and move lfs_ialloc() and lfs_vcreate() to new lfs_init_vnode().

- Add lfs_valloc_fixed() to allocate a known inode, used by kernel
roll forward.

- Remove lfs_*ref(), these functions cannot coexist with vcache and
their commented behaviour is far away from their implementation.

- Add the cleaner lwp and blockinfo to struct ulfsmount so lfs_loadvnode()
may use hints from the cleaner.

- Remove vnode locks from ulfs_lookup() like we did with ufs_lookup().
 1.271  20-Apr-2015  riastradh Make VOP_LINK return directory still locked and referenced.

Ride 7.99.10 bump.
 1.270  27-Mar-2015  riastradh Disentangle buffer-cached I/O from page-cached I/O in UFS.

Page-cached I/O is used for regular files, and is initiated by VFS
users such as userland and NFS.

Buffer-cached I/O is used for directories and symlinks, and is issued
only internally by UFS.

New UFS routine ufs_bufio replaces vn_rdwr for internal use.
ufs_bufio is implemented by new UFS operations uo_bufrd/uo_bufwr,
which sit in ufs_readwrite.c alongside the VOP_READ/VOP_WRITE
implementations.

I preserved the code as much as possible and will leave further
simplification for future commits. I kept the ulfs_readwrite.c
copypasta close to ufs_readwrite.c in case we ever want to merge them
back; likewise ext2fs_readwrite.c.

No externally visible semantic change. All atf fs tests still pass.
 1.269  25-Jul-2014  dholland branches: 1.269.2; 1.269.4;
Add VOP_FALLOCATE and VOP_FDISCARD to every vnode ops table I can
find.

The filesystem ones all call genfs_eopnotsupp - right now I am only
implementing the plumbing and we can implement fallocate and/or
fdiscard for files later.

The device ones call spec_fallocate (which is also genfs_eopnotsupp)
and spec_fdiscard, which dispatches to the device-level op.

The fifo ones all call vn_fifo_bypass, which also ends up being
EOPNOTSUPP.
 1.268  17-May-2014  dholland Merge ulfs_create into lfs_create.
 1.267  17-May-2014  dholland Merge ulfs_mkdir into lfs_mkdir.
 1.266  17-May-2014  dholland Merge ulfs_symlink into lfs_symlink.
 1.265  17-May-2014  dholland Move the ulfs-level (copy of ufs) vnops for symlink, create, and mkdir
into lfs_vnops.c preparatory to folding them into the lfs entry points.

(lfs_vnops.c now has four licenses. sigh.)
 1.264  17-May-2014  dholland Remove the DIROP macros. They are evil, especially the CREATE ones.

This results in some duplicate logic in the creation vnops (symlink,
mknod, create, mkdir) but we will probably be able to factor it out in
a more sensible way later.

Now the creation vnops call getnewvnode explicitly instead of under
multiple layers of obscure gunk. Then we explicitly do lfs_set_dirop,
and afterwards lfs_unset_dirop.
 1.263  16-May-2014  dholland Move lfs_getpages and lfs_putpages to their own file.
 1.262  24-Mar-2014  hannken branches: 1.262.2;
- Make VI_XLOCK, VI_CLEAN and VI_LOCKSHARE private to kern/vfs_*.c.
- Make vwait() static.
- Add vdead_check() to check a vnode for being or becoming dead.

Discussed on tech-kern.

Welcome to 6.99.38
 1.261  23-Jan-2014  hannken Change vnode operations create, mknod, mkdir and symlink to return
the resulting vnode *vpp unlocked.

Discussed on tech-kern@

Welcome to 6.99.30
 1.260  17-Jan-2014  hannken Change vnode operations create, mknod, mkdir and symlink to keep the
directory node dvp locked on return.

Discussed on tech-kern@

Welcome to 6.99.29
 1.259  18-Oct-2013  christos use __USE() in the right place, instead of (void)var.
 1.258  17-Oct-2013  christos - remove unused variables
- add debug ifdefs for debugging variables
- __USE() where appropriate.
 1.257  29-Jul-2013  dholland Fix build both with and without options LFS_EI.
 1.256  29-Jul-2013  dholland Revert previous; it is wrong.
 1.255  28-Jul-2013  pgoyette Remove unused variable to fix the build.
 1.254  28-Jul-2013  dholland Merge the extattr VOPs from ffs.
As these do nothing besides dispatch to ulfs_extattr.c it wasn't
exactly hard.

This might just make extended attributes work on lfs...
 1.253  28-Jul-2013  dholland Migrate the miscellaneous ulfs-level info from struct ulfsmount to
struct lfs.

Put them inside #ifdef _KERNEL there. They are not the only such
members, gross as that is. Unfortunately, moving struct lfs to
lfs_kernel.h does not work.
 1.252  28-Jul-2013  dholland Add lfs_kernel.h for declarations that don't need to be exposed to userland.

lfs currently has the following headers:
lfs.h - on-disk structures and stuff needed for userlevel tools
lfs_inode.h - additional restricted materials for userlevel tools
that operate the fs (newfs_lfs, fsck_lfs, lfs_cleanerd)
lfs_kernel.h - stuff needed only in the kernel

and the following legacy headers that are expected to be mopped up and
folded into one of the above:
lfs_extern.h - function prototypes
ulfs_bswap.h - endian-independent support
ulfs_dinode.h - now contains very little
ulfs_dirhash.h - dirhash support
ulfs_extattr.h - extattr support
ulfs_extern.h - more function prototypes
ulfs_inode.h - assorted kernel-only declarations
ulfs_quota.h - quota support
ulfs_quota1.h - more quota support
ulfs_quota2.h - more quota support
ulfs_quotacommon.h - more quota support
ulfsmount.h - legacy copy of ufsmount material
 1.251  21-Jul-2013  dholland Merge logic from ulfs_close(), ulfs_getattr(), and ulfs_strategy()
into the preexisting lfs_*() versions of these functions, and delete
the unused ulfs copies.
 1.250  20-Jul-2013  dholland Merge ulfs_mknod into lfs_mknod, which was missing some bits.
 1.249  20-Jul-2013  dholland Collect the pieces of lfs rename into lfs_rename.c, and sprinkle static.
 1.248  18-Jun-2013  christos branches: 1.248.2; 1.248.4;
Prefix most of the cpp macros with lfs_ and LFS_ to avoid conflicts with ffs.
This was done so that boot blocks that want to compile both FFS and LFS in
the same file work.
 1.247  08-Jun-2013  dholland ulfs_dir.h has been emptied; remove it.
 1.246  08-Jun-2013  dholland Stick LFS_ in front of IFMT, IFIFO, IFREG, etc. so as not to conflict
with the UFS copies of these symbols. (Which themselves ought to have
UFS_ stuck on.)
 1.245  06-Jun-2013  dholland Split lfs from ufs step 4:

Massedit all ufs symbols to be "ulfs" instead, to make sure there are
no conflicts with ufs. Confirmed with grep.

(This required changing a few comments that maybe should have been
left alone to say "ulfs", but we'll survive that.)
 1.244  06-Jun-2013  dholland Split lfs from ufs, part 2:

Change all <ufs/ufs/foo.h> includes to <ufs/lfs/ulfs_foo.h>.
 1.243  22-Jan-2013  dholland Stuff UFS_ in front of a few of ufs's symbols to reduce namespace
pollution. Specifically:
ROOTINO -> UFS_ROOTINO
WINO -> UFS_WINO
NXADDR -> UFS_NXADDR
NDADDR -> UFS_NDADDR
NIADDR -> UFS_NIADDR
MAXSYMLINKLEN -> UFS_MAXSYMLINKLEN
MAXSYMLINKLEN_UFS[12] -> UFS[12]_MAXSYMLINKLEN (for consistency)

Sort out ext2fs's misuse of NDADDR and NIADDR; fortunately, these have
the same values in ext2fs and ffs.

No functional change intended.
 1.242  09-May-2012  riastradh branches: 1.242.2;
Adapt ffs, lfs, and ext2fs to use genfs_rename.

ok dholland, rmind
 1.241  13-Mar-2012  elad Replace the remaining KAUTH_GENERIC_ISSUSER authorization calls with
something meaningful. All relevant documentation has been updated or
written.

Most of these changes were brought up in the following messages:

http://mail-index.netbsd.org/tech-kern/2012/01/18/msg012490.html
http://mail-index.netbsd.org/tech-kern/2012/01/19/msg012502.html
http://mail-index.netbsd.org/tech-kern/2012/02/17/msg012728.html

Thanks to christos, manu, njoly, and jmmv for input.

Huge thanks to pgoyette for spinning these changes through some build
cycles and ATF.
 1.240  16-Feb-2012  perseant Pass t_renamerace and t_rmdirrace tests.

Adapt dholland@'s fix to ufs_rename to fix PR kern/43582. Address several
other MP locking issues discovered during the course of investigating the
same problem.

Removed extraneous vn_lock() calls on the Ifile, since the Ifile writes
are controlled by the segment lock.

Fix PR kern/45982 by deemphasizing the estimate of how much metadata
will fill the empty space on disk when the disk is nearly empty
(t_renamerace crates a lot of inode blocks on a tiny empty disk).
 1.239  02-Jan-2012  perseant branches: 1.239.2;

* Remove PGO_RECLAIM during lfs_putpages()' call to genfs_putpages(),
to avoid a live lock in the latter when reclaiming a vnode with
dirty pages.

* Add a new segment flag, SEGM_RECLAIM, to note when a segment is
being written for vnode reclamation, and record which inode is being
reclaimed, to aid in forensic debugging.

* Add a new segment flag, SEGM_SINGLE, so that opportunistic writes
can write a single segment's worth of blocks and then stop, rather
than writing all the way up to the cleaner's reserved number of
segments.

* Add assert statements to check mutex ownership is the way it ought
to be, mostly in lfs_putpages; fix problems uncovered by this.

* Don't clear VU_DIROP until the inode actually makes its way to disk,
avoiding a problem where dirop inodes could become separated
(uncovered by a modified version of the "ckckp" forensic regression
test).

* Move the vfs_getopsbyname() call into lfs_writerd. Prepare code to
make lfs_writerd notice when there are no more LFSs, and exit losing
the reference, so that, in theory, the module can be unloaded. This
code is not enabled, since it causes a crash on exit.

* Set IN_MODIFIED on inodes flushed by lfs_flush_dirops. Really we
only need to set IN_MODIFIED if we are going to write them again
(e.g., to write pages); need to think about this more.

Finally, several changes to help avoid "no clean segments" panics:

* In lfs_bmapv, note when a vnode is loaded only to discover whether
its blocks are live, so it can immediately be recycled. Since the
cleaner will try to choose ~empty segments over full ones, this
prevents the cleaner from (1) filling the vnode cache with junk, and
(2) squeezing any unwritten writes to disk and running the fs out of
segments.

* Overestimate by half the amount of metadata that will be required
to fill the clean segments. This will make the disk appear smaller,
but should help avoid a "no clean segments" panic.

* Rearrange lfs_writerd. In particular, lfs_writerd now pays
attention to the number of clean segments available, and holds off
writing until there is room.
 1.238  20-Sep-2011  chs branches: 1.238.2; 1.238.6;
strengthen the assertions about pages existing during block allocation,
which were incorrectly relaxed last year. add some comments so that
the intent of these is hopefully clearer.

in ufs_balloc_range(), don't free pages or mark them dirty if
allocating their backing store failed. this fixes PR 45369.
 1.237  12-Jul-2011  dholland Pass the ufs_lookup_results pointer around instead of fetching it from
the inode in the guts of ufs. Now, in VOPs where i_crap is used it is
used (directly) only immediately on entry to the VOP call and then
passed around by reference.

Except for rename, which needs explicit sorting out. The code in
ufs_wapbl_rename is unchanged in behavior but I'm increasingly
inclined to think it's wrong.
 1.236  11-Jul-2011  hannken Change VOP_BWRITE() to take a vnode as its first argument like all other
VOPs do. Layered file systems no longer have to modify bp->b_vp and run
into trouble when an async VOP_BWRITE() uses the wrong vnode.

- change all occurences of VOP_BWRITE(bp) to VOP_BWRITE(bp->b_vp, bp).
- remove layer_bwrite().
- welcome to 5.99.55

Adresses PR kern/38762 panic: vwakeup: neg numoutput

No objections from tech-kern@.
 1.235  12-Jun-2011  rmind Welcome to 5.99.53! Merge rmind-uvmplock branch:

- Reorganize locking in UVM and provide extra serialisation for pmap(9).
New lock order: [vmpage-owner-lock] -> pmap-lock.

- Simplify locking in some pmap(9) modules by removing P->V locking.

- Use lock object on vmobjlock (and thus vnode_t::v_interlock) to share
the locks amongst UVM objects where necessary (tmpfs, layerfs, unionfs).

- Rewrite and optimise x86 TLB shootdown code, make it simpler and cleaner.
Add TLBSTATS option for x86 to collect statistics about TLB shootdowns.

- Unify /dev/mem et al in MI code and provide required locking (removes
kernel-lock on some ports). Also, avoid cache-aliasing issues.

Thanks to Andrew Doran and Joerg Sonnenberger, as their initial patches
formed the core changes of this branch.
 1.234  05-Jan-2011  martin branches: 1.234.6;
Avoid NULL deref inside a KASSERT, as discussed on tech-kern.
 1.233  02-Jan-2011  dholland Remove the special refcount behavior (adding an extra reference to the
parent dir) associated with SAVESTART in relookup().

Check all call sites to make sure that SAVESTART wasn't set while
calling relookup(); if it was, adjust the refcount behavior. Remove
related references to SAVESTART.

The only code that was reaching the extra ref was msdosfs_rename,
where the refcount behavior was already fairly broken and/or gross;
repair it.

Add a dummy 4th argument to relookup to make sure code that hasn't
been inspected won't compile. (This will go away next time the
relookup semantics change, which they will.)
 1.232  18-Dec-2010  eeh Byebye deadlock.
 1.231  04-Aug-2010  hannken Free the on disk inodes in the reclaim routine.
 1.230  29-Jul-2010  hannken Add vm page flag PG_MARKER and use it to tag dummy marker pages
in genfs_do_putpages() and uao_put().
Use 'v_uobj.uo_npages' to check for an empty memq.
Put some assertions where these marker pages may not appear.

Ok: YAMAMOTO Takashi <yamt@netbsd.org>
 1.229  24-Jun-2010  hannken Clean up vnode lock operations pass 2:

VOP_UNLOCK(vp, flags) -> VOP_UNLOCK(vp): Remove the unneeded flags argument.

Welcome to 5.99.32.

Discussed on tech-kern.
 1.228  24-Jun-2010  hannken Clean up vnode lock operations:

- VOP_LOCK(vp, flags): Limit the set of allowed flags to LK_EXCLUSIVE,
LK_SHARED and LK_NOWAIT. LK_INTERLOCK is no longer allowed as it
makes no sense here.

- VOP_ISLOCKED(vp): Remove the for some time unused return value
LK_EXCLOTHER. Mark this operation as "diagnostic only".
Making a lock decision based on this operation is no longer allowed.

Discussed on tech-kern.
 1.227  29-Mar-2010  pooka Stop exposing fifofs internals and leave only fifo_vnodeop_p visible.
 1.226  07-Dec-2009  eeh branches: 1.226.2; 1.226.4;
Fix some more hangs and deadlocks.
 1.225  17-Nov-2009  eeh This should fix a deadlock.
 1.224  05-Nov-2009  pooka Include compat code by default.
 1.223  30-Oct-2009  christos compile without COMPAT_50
 1.222  29-Oct-2009  christos PR/42246: NAKAJIMA Yoshihiro: provide COMPAT_50 for LFS
 1.221  07-May-2009  elad Replace KAUTH_GENERIC_ISSUSER with a better alternative.
 1.220  22-Feb-2009  ad PR kern/26878 FFSv2 + softdep = livelock (no free ram)
PR kern/16942 panic with softdep and quotas
PR kern/19565 panic: softdep_write_inodeblock: indirect pointer #1 mismatch
PR kern/26274 softdep panic: allocdirect_merge: ...
PR kern/26374 Long delay before non-root users can write to softdep partitions
PR kern/28621 1.6.x "vp != NULL" panic in ffs_softdep.c:4653 while unmounting a softdep (+quota) filesystem
PR kern/29513 FFS+Softdep panic with unfsck-able file-corruption
PR kern/31544 The ffs softdep code appears to fail to write dirty bits to disk
PR kern/31981 stopping scsi disk can cause panic (softdep)
PR kern/32116 kernel panic in softdep (assertion failure)
PR kern/32532 softdep_trackbufs deadlock
PR kern/37191 softdep: locking against myself
PR kern/40474 Kernel panic after remounting raid root with softdep

Retire softdep, pass 2. As discussed and later formally announced on the
mailing lists.
 1.219  16-Jan-2009  yamt branches: 1.219.2;
one more change which i forgot to commit with
UVM_PAGE_HASH_PENALTY -> UVM_PAGE_TREE_PENALTY rename.
noticed by Andreas Wrede.
 1.218  24-Jun-2008  gmcgarry branches: 1.218.4; 1.218.6; 1.218.12;
fcntl(4) says the command is type int. lfs_fcntl() comment says u_long. The implementation says int. Synchronise comment with documentation and cast to int before comparison.
 1.217  04-Jun-2008  ad branches: 1.217.2;
vm_page: put TAILQ_ENTRY into a union with LIST_ENTRY, so we can use both.
 1.216  28-Apr-2008  martin branches: 1.216.2;
Remove clause 3 and 4 from TNF licenses
 1.215  25-Jan-2008  ad branches: 1.215.6; 1.215.8; 1.215.10;
Remove VOP_LEASE. Discussed on tech-kern.
 1.214  02-Jan-2008  ad Merge vmlocking2 to head.
 1.213  26-Nov-2007  pooka branches: 1.213.2; 1.213.6;
Remove the "struct lwp *" argument from all VFS and VOP interfaces.
The general trend is to remove it from all kernel interfaces and
this is a start. In case the calling lwp is desired, curlwp should
be used.

quick consensus on tech-kern
 1.212  10-Oct-2007  ad branches: 1.212.4;
Merge from vmlocking:

- Split vnode::v_flag into three fields, depending on field locking.
- simple_lock -> kmutex in a few places.
- Fix some simple locking problems.
 1.211  08-Oct-2007  ad Merge ffs locking & brelse changes from the vmlocking branch.
 1.210  29-Jul-2007  ad branches: 1.210.4; 1.210.6; 1.210.8; 1.210.10;
It's not a good idea for device drivers to modify b_flags, as they don't
need to understand the locking around that field. Instead of setting
B_ERROR, set b_error instead. b_error is 'owned' by whoever completes
the I/O request.
 1.209  27-Jul-2007  pooka Change unused fflags parameter in VOP_MMAP to prot and pass in
desired vm protection.
 1.208  10-Jul-2007  perseant branches: 1.208.2;
Move the "vp = NULL" assignment after the code that requires vp != NULL.
Reported by Chris Ross on current-users.
 1.207  09-Jul-2007  ad Merge some of the less invasive changes from the vmlocking branch:

- kthread, callout, devsw API changes
- select()/poll() improvements
- miscellaneous MT safety improvements
 1.206  24-Apr-2007  perseant Get rid of our own private copy of genfs_putpages, having adapted the real
genfs_putpages to suit our purposes.
 1.205  17-Apr-2007  perseant Fix another locking protocol error in lfs_fsync().
 1.204  17-Apr-2007  perseant Fix MP locking protocol violations introduced in my previous commit.
 1.203  17-Apr-2007  perseant Install a new sysctl, vfs.lfs.ignore_lazy_sync, which causes LFS to ignore
the "smooth" syncer, as if vfs.sync.*delay = 0, but only for LFS. The
default is "on", i.e., ignore lazy sync.

Reduce the amount of polling/busy-waiting done by lfs_putpages(). To
accomplish this, copied genfs_putpages() and modified it to indicate which
page it was that caused it to return with EDEADLK. fsync()/fdatasync()
should no longer ever fail with EAGAIN, and should not consume huge
quantities of cpu.

Also, try to make dirops less likely to be written as the result of a
VOP_PUTPAGES(), while ensuring that they are written regularly.
 1.202  05-Apr-2007  perseant correct comment for lfs_putpages
 1.201  04-Mar-2007  christos branches: 1.201.2; 1.201.4;
Kill caddr_t; there will be some MI fallout, but it will be fixed shortly.
 1.200  21-Feb-2007  thorpej Replace the Mach-derived boolean_t type with the C99 bool type. A
future commit will replace use of TRUE and FALSE with true and false.
 1.199  20-Feb-2007  ad Call genfs_node_destroy() where appropriate.
 1.198  09-Feb-2007  ad branches: 1.198.2;
Merge newlock2 to head.
 1.197  19-Jan-2007  hannken New file system suspension API to replace vn_start_write and vn_finished_write.
The suspension helpers are now put into file system specific operations.
This means every file system not supporting these helpers cannot be suspended
and therefore snapshots are no longer possible.

Implemented for file systems of type ffs.

The new API is enabled on a kernel option NEWVNGATE. This option is
not enabled by default in any kernel config.

Presented and discussed on tech-kern with much input from
Bill Studenmund <wrstuden@netbsd.org> and YAMAMOTO Takashi <yamt@netbsd.org>.

Welcome to 4.99.9 (new vfs op vfs_suspendctl).
 1.196  04-Jan-2007  elad Consistent usage of KAUTH_GENERIC_ISSUSER.
 1.195  03-Jan-2007  perseant Change VONWORKLST handling to better match its other uses; in particular,
check memq and clear VWRITEMAPDIRTY at the same time.
 1.194  09-Dec-2006  chs a smorgasbord of improvements to vnode locking and path lookup:
- LOCKPARENT is no longer relevant for lookup(), relookup() or VOP_LOOKUP().
these now always return the parent vnode locked. namei() works as before.
lookup() and various other paths no longer acquire vnode locks in the
wrong order via vrele(). fixes PR 32535.
as a nice side effect, path lookup is also up to 25% faster.
- the above allows us to get rid of PDIRUNLOCK.
- also get rid of WANTPARENT (just use LOCKPARENT and unlock it).
- remove an assumption in layer_node_find() that all file systems implement
a recursive VOP_LOCK() (unionfs doesn't).
- require that all file systems supply vfs_vptofh and vfs_fhtovp routines.
fill in eopnotsupp() for file systems that don't support being exported
and remove the checks for NULL. (layerfs calls these without checking.)
- in union_lookup1(), don't change refcounts in the ISDOTDOT case, just
adjust which vnode is locked. fixes PR 33374.
- apply fixes for ufs_rename() from ufs_vnops.c rev. 1.61 to ext2fs_rename().
 1.193  16-Nov-2006  christos branches: 1.193.2;
__unused removal on arguments; approved by core.
 1.192  20-Oct-2006  reinoud Replace the LIST structure mp->mnt_vnodelist to a TAILQ structure since all
vnodes were synced and processed backwards. This meant that the last
accessed node was processed first and the earlierst last.

An extra benefit is the removal of the ugly hack from the Berkly days on
LFS.

In the proces, i've also replaced the various variations hand written loops
by the TAILQ_FOREACH() macro's.
 1.191  12-Oct-2006  christos - sprinkle __unused on function decls.
- fix a couple of unused bugs
- no more -Wno-unused for i386
 1.190  28-Sep-2006  perseant Use lockstatus instead of a homebrewed locking system to control
LFCNWRAPSTOP and LFCNWRAPGO.

Be less verbose about the various looping checks: use log() rather than
printf(), and only log anything if we are really looping ("count = 2" is
not an error condition).

Allow dirops sleeping on available space to be interruptible.
 1.189  15-Sep-2006  perseant branches: 1.189.2;
Don't remark a locked inode with IN_MODIFIED after writing it to disk,
if we ourselves hold the lock. This prevents e.g. mknod from hanging
indefinitely.

Also, always use the return value from VOP_ISLOCKED to determine whether
we hold the lock or someone else does, rather than looking into the lock
structure ourselves.
 1.188  01-Sep-2006  perseant branches: 1.188.2;
Changes to help the roll-forward agent, to wit:

* Mark being-deleted files in the Ifile so we can finish deleting them
at fs mount time.
* Flag the Ifile with "cleaner must clean" when writers are waiting for
the cleaner, rather than relying solely on the cleaner's estimation of
whether it should clean or not.
* Note partial segments written by a user agent (in particular,
fsck_lfs) so that repeated rolls forward don't interfere with one
another.
* Add a new fcntl, LFCNPASS, that allows the log to wrap exactly once,
for better testing of the validity of checkpoints.
* Keep track of the on-disk nlink count when cleaning, so that we don't
partially complete directory operations while cleaning.
* Ensure that every single Ifile inode write represents a consistent
view of the filesystem. In particular, the accounting for the segment
we are writing the inode into must be correct, and the accounting for
the segment that inode used to reside in must be correct. Rather than
just rewriting the inode if we wrote it wrong, rewrite the necessary
ifile blocks before writing the inode so we never write it wrong.
* Don't unmark any VDIROP vnodes if we haven't written them to disk,
avoiding yet another problem with the "wait for the cleaner" error
return from lfs_putpages().

Also, move the last callback to an aiodone call, so we no longer do any
memory management from interrupt context.
 1.187  06-Aug-2006  martin Fix size confusion with lfs_fhandle - and as it now turns out to be the same
as the lfs compat_30_fhandle, g/c the latter.
Add an alias for the LFCNIFILEFH fcntl, so that binaries compiled in the
meantime (with too large lfs_fhandle) continue to work.

This makes vfs_cleanerd work again after the kernel checks filehandle size
more strictly (problem reported by Kurt Schreiner on current-users).
 1.186  31-Jul-2006  martin Make filehandles opaque to userland
 1.185  23-Jul-2006  ad Use the LWP cached credentials where sane.
 1.184  20-Jul-2006  perseant Move the kauth checks up front, so that all new LFS fcntl calls are subject
to the check for superuser privilege.
 1.183  13-Jul-2006  martin Apply _KERNEL_OPT
 1.182  13-Jul-2006  martin Version the lfs_cleanerd internal fcntl() for filehandles too,
so old cleaners should work with newer kernels.
 1.181  13-Jul-2006  martin Fix alignement problems for fhandle_t, exposed by gcc4.1.

While touching all vptofh/fhtovp functions, get rid of VFS_MAXFIDSIZ,
version the getfh(2) syscall and explicitly pass the size available in
the filehandle from userland.

Discussed on tech-kern, with lots of help from yamt (thanks!).
 1.180  29-Jun-2006  perseant Don't wake up the cleaner if the filesystem is unwrappable, and fix the
compatibility fcntls.

Also includes one-line fixes for an MP locking bug and a zero-length FINFO
problem that manifested during testing.
 1.179  24-Jun-2006  perseant Change LFCNWRAP{STOP,GO} to make them more suitable for snapshotting; in
particular, the caller can now choose whether to wait for the condition
to be met, and if the caller of LFCNWRAPSTOP dies or otherwise closes
the descriptor, the filesystem is started again. Updated the ckckp
regression test to use the new semantics.

dump_lfs(8) now uses the fcntls to implement LFS-style snapshotting through
the -X flag, addressing PR#33457 albeit not using fss(4). Fixed a couple
other problems with dump_lfs that manifested themselves during testing.
 1.178  18-May-2006  perseant branches: 1.178.4;
Break out the finfo array manipulation code into two new functions,
lfs_acquire_finfo() and lfs_release_finfo(). Add a debugging check
for zero-length finfo arrays in the segment summary to avoid future
regressions.
 1.177  17-May-2006  perseant Don't be quite so eager to error out from lfs_putpages() when pages are
busy; if we've sensed a possible 3-way deadlock and are not the pagedaemon,
relock and try again.
 1.176  14-May-2006  elad integrate kauth.
 1.175  12-May-2006  perseant Fixes to address the "vinvalbuf: dirty blocks" panic that can occur when
many inodes are cleaned at once. Make sure that we write all the pages
on vnodes that are being flushed, even if we don't think there's room;
drain v_numoutput before lfs_vflush() completes.

Also, don't allow a vnode that is in the process of being cleaned to be
chosen by getnewvnode(); this avoids a segment accounting panic in the case
that a large number of inodes are fed to lfs_markv() all at once.
 1.174  04-May-2006  perseant Change VOP_FCNTL to take an unlocked vnode. Approved by wrstuden@.
 1.173  04-May-2006  perseant Introduce another per-filesystem parameter, lfs_resvseg, to separate the
notion of "how many segments are reserved for the cleaner" from that of
"how many segments are not counted in lfs_bfree". The default value
used for existing filesystems is the same as the previous implicit value
of (lfs_minfreeseg / 2 + 1), modulo some sanity checking.

Count pending dirops on a per-filesystem basis, since once we start
writing them we can't stop until we're done. This seems to help stave off
the "no clean segments" panic in the case of filling the filesystem with
directories and small files (e.g. simultaneously unpacking more copies of
pkgsrc than will fit).
 1.172  02-May-2006  perseant Fix a "locking against myself": lfs_flush_dirops() doesn't need to lock the
vnodes to write their blocks, since it holds the segment lock.
 1.171  01-May-2006  perseant Don't ever partially write dirops, even if we need the cleaner to run.
This increases the chances of the "no clean segments" panic slightly,
but allows us to run the ckckp regression test successfully to completion.
 1.170  30-Apr-2006  perseant Postpone the segment accounting changes coming from truncation until the
inode that makes those changes valid is either written to disk by
lfs_writeinode() or discarded by lfs_vfree().

A couple of locking fixes are also included as well.
 1.169  18-Apr-2006  perseant Yet another MP locking issue.
 1.168  17-Apr-2006  perseant Introduce two fcntl calls that freeze the filesystem right at the point
where segment 0 is being considered for writing. This allows for automated
checkpoint vailidity scanning, and could be used (in conjunction with the
existing LFCNREWIND) for e.g. snapshot dumps as well.

Include a regression test that does such scanning.

When writing the Ifile, loop through the dirty block list three times to
make sure that the checkpoint is always consistent (the first and second
times the Ifile blocks can cross a segment boundary; not so the third time
unless the segments are very small). Discovered by using the aforementioned
regression test.
 1.167  13-Apr-2006  perseant Make lfs_vref/lfs_vunref not need to know about VXLOCK and VFREEING
explicitly (especially since we didn't know about VFREEING at all before),
but notice the EBUSY return from vget() instead.

Fix some more MP locking protocol issues, most of which were pointed out by
Christian Ehrhardt this morning on tech-kern.
 1.166  11-Apr-2006  perseant Another MP locking fix.
 1.165  10-Apr-2006  perseant Don't leak vnode references if we fail to lock a vnode in lfs_flush_pchain().
Also fix another (probably only academic) simple_lock protocol error.
 1.164  08-Apr-2006  perseant Implement a somewhat finer-grained mechanism for paging LFS-backed pages.
The writer daemon, if it does not need to flush the whole filesystem,
now only writes the vnodes for which the pagedaemon has requested pageouts
(although it does not pay attention to the page ranges the pagedaemon
supplies).
 1.163  07-Apr-2006  perseant Several minor bug fixes:

* Correct (weak) segment lock assertions in lfs_fragextend and lfs_putpages.
* Keep IN_MODIFIED set if we run out of avail in lfs_putpages.
* Don't try to (re)write buffers on a VBLK vnode; fixes a panic I found
while running with an LFS root.
* Raise priority of LFCNSEGWAIT to PVFS; PUSER is way too low for
something the pagedaemon is relying on.
 1.162  01-Apr-2006  perseant Make sure we unlock to zero when avoiding 3-way deadlock; otherwise we
simply have a different form of deadlock.
 1.161  31-Mar-2006  perseant Handle the "filesystem is clean" flag correctly when upgrading from
read-only to read-write mount. This makes "root on lfs" work for me,
although it looks like a different traceback from PR#32667.
 1.160  30-Mar-2006  yamt some cleanups after the introduction of GOP_SIZE_MEM flag.
- remove GOP_SIZE_READ/GOP_SIZE_WRITE flags.
they have not been used since the change.
- ufs_balloc_range: remove code which has been no-op since the change.
thanks Konrad Schroder for explaining the original intention of the code.
- ffs_gop_size: don't extend past eof, in the case of GOP_SIZE_MEM.
otherwise genfs_getpages end up to allocate pages past eof unnecessarily.
 1.159  28-Mar-2006  perseant Don't let the pagedaemon wait for pages, since that is just asking for
a deadlock.
 1.158  24-Mar-2006  perseant Improvements to LFS's paging mechanism, to wit:

* Acknowledge that sometimes there are more dirty pages to be written to
disk than clean segments. When we reach the danger line,
lfs_gop_write() now returns EAGAIN. The caller of VOP_PUTPAGES(), if
it holds the segment lock, drops it and waits for the cleaner to make
room before continuing.

* Note and avoid a three-way deadlock in lfs_putpages (a writer holding
a page busy blocks on the cleaner while the cleaner blocks on the
segment lock while lfs_putpages blocks on the page).
 1.157  11-Dec-2005  christos branches: 1.157.4; 1.157.6; 1.157.8; 1.157.10; 1.157.12;
merge ktrace-lwp.
 1.156  02-Nov-2005  yamt merge yamt-vop branch. remove following VOPs.

VOP_BLKATOFF
VOP_VALLOC
VOP_BALLOC
VOP_REALLOCBLKS
VOP_VFREE
VOP_TRUNCATE
VOP_UPDATE
 1.155  13-Sep-2005  christos branches: 1.155.2;
split out lfs_itimes(). It is used in fsck_lfs.
 1.154  12-Sep-2005  christos Use nanotime() to update the time fields in filesystems. Convert the code
from macros to real functions. Original patch and review from chuq.
Note: ext2fs only keeps seconds in the on-disk inode, and msdosfs does not
have enough precision for all fields, so this is not very useful for those
two.
 1.153  19-Aug-2005  christos 64 bit inode changes.
 1.152  29-May-2005  christos branches: 1.152.2;
- sprinkle const
- avoid shadow variables.
 1.151  20-May-2005  perseant VOP_LOCK drops the interlock; pick it up again to avoid an "already unlocked"
panic in lfs_putpages.
 1.150  27-Apr-2005  perseant Recognize that we hold the v_interlock when relocking after a flush in
lfs_putpages.
 1.149  25-Apr-2005  skrll Use the right arg structure for lfs_setattr, i.e. s/getattr/setattr/.
 1.148  23-Apr-2005  perseant Provide a resize_lfs(8), including kernel and cleaner support. The current
implementation requires the fs to be mounted while resizing. Tested in both
directions, and everything appears to work happily, but ymmv.
 1.147  19-Apr-2005  perseant Keep per-inode, per-fs, and subsystem-wide counts of blocks allocated through
lfs_balloc(), and use that to estimate the number of dirty pages belonging
to LFS (subsystem or filesystem). This is almost certainly wrong for
the case of a large mmap()ed region, but the accounting is tighter than
what we had before, and performs much better in the typical case of pages
dirtied through write().
 1.146  18-Apr-2005  perseant Check for the inode having been previously freed, in UNMARK_VNODE().
Avoids a panic when calling mkdir() on a full filesystem.
 1.145  16-Apr-2005  perseant Use splay trees, rather than a hash table, to manage the accounting of
blocks allocated through VOP_BALLOC() for pages to be written to disk.
This accounting no longer takes a noticeable fraction of the system CPU.
 1.144  16-Apr-2005  perseant Use lfs_malloc() to manage the blkiov arrays that the cleaner functions use,
since the cleaner is likely to operate in a low-memory condition.
 1.143  14-Apr-2005  perseant Tabify leading whitespace
 1.142  14-Apr-2005  perseant Consolidate the hash table we use to maintain the integrity of lfs_avail
into a single, system-wide table, rather than having a separate hash table
per inode. Significantly reduces the "system" cpu usage of your average
file write.
 1.141  01-Apr-2005  perseant Protect various per-fs structures with fs->lfs_interlock simple_lock, to
improve behavior in the multiprocessor case. Add debugging segment-lock
assertion statements.
 1.140  25-Mar-2005  perseant Don't sleep while holding the vnode interlock. Should take care of the
first panic case in PR #26043.
 1.139  24-Mar-2005  chs avoid the need for recursive locking lfs_flush_dirops() by unlocking
the vnode around the call to this in the caller.
 1.138  23-Mar-2005  perseant Make LFS dirops get their vnode first, before incrementing the dirop count,
to prevent a deadlock trying to call VOP_PUTPAGES() on a VDIROP vnode.
This can happen when a stacked filesystem is mounted on top of an LFS: an
LFS dirop needs to get a vnode, which is available from the upper layer.
The corresponding lower layer vnode, however, is VDIROP, so the upper layer
can't be cleaned out since its VOP_PUTPAGES() is passed through to the lower
layer, which waits for dirops to drain before it can proceed. Deadlock.

Tweak ufs_makeinode() and ufs_mkdir() to pass the a_vpp argument through
to VOP_VALLOC().

Partially addresses PR # 26043, though it probably does not completely fix
the problem described there.
 1.137  08-Mar-2005  simonb branches: 1.137.2;
Tab Police.
 1.136  08-Mar-2005  perseant Straighten out the maze of ifdefs. Instead, consolidate all the debugging
stuff under '#ifdef DEBUG', and use sysctl knobs to turn on/off particular
parts of the debugging reporting (if DEBUG is enabled). Re-enable the LFS
statistics in sysctl, while I'm there. A bit of a rototill.
 1.135  26-Feb-2005  perry nuke trailing whitespace
 1.134  26-Feb-2005  perseant Various minor LFS improvements:

* Note when lfs_putpages(9) thinks it is not going to be writing any
pages before calling genfs_putpages(9). This prevents a situation in
which blocks can be queued for writing without a segment header.
* Correct computation of NRESERVE(), though it is still a gross
overestimate in most cases. Note that if NRESERVE() is too high, it
may be impossible to create files on the filesystem. We catch this
case on filesystem mount and refuse to mount r/w.
* Allow filesystems to be mounted whose block size is == MAXBSIZE.
* Somewhere along the line, ufs_bmaparray(9) started mangling UNWRITTEN
entries in indirect blocks again, triggering a failed assertion "daddr
<= LFS_MAX_DADDR". Explicitly convert to and from int32_t to correct
this.
* Add a high-water mark for the number of dirty pages any given LFS can
hold before triggering a flush. This is settable by sysctl, but off
(zero) by default.
* Be more careful about the MAX_BYTES and MAX_BUFS computations so we
shouldn't see "please increase to at least zero" messages.
* Note that VBLK and VCHR vnodes can have nonzero values in di_db[0]
even though their v_size == 0. Don't panic when we see this.
* Change lfs_bfree to a signed quantity. The manner in which it is
processed before being passed to the cleaner means that sometimes it
may drop below zero, and the cleaner must be aware of this.
* Never report bfree < 0 (or higher than lfs_dsize) through
lfs_statvfs(9). This prevents df(1) from ever telling us that our full
filesystems have 16TB free.
* Account space allocated through lfs_balloc(9) that does not have
associated buffer headers, so that the pagedaemon doesn't run us out
of segments.
* Return ENOSPC from lfs_balloc(9) when bfree drops to zero.
* Address a deadlock in lfs_bmapv/lfs_markv when the filesystem is being
unmounted. Because vfs_busy() is a shared lock, and
lfs_bmapv/lfs_markv mark the filesystem vfs_busy(), the cleaner can be
holding the lock that umount() is blocking on, then try to vfs_busy()
again in getnewvnode().
 1.133  25-Jan-2005  wrstuden Extend fsync_range(2) to support the FDISKSYNC flag, which requests
that the sync be propogated out through the disk drive caches.
 1.132  22-Apr-2004  yamt branches: 1.132.4; 1.132.6;
check_dirty: fix another PHOLD leak. ("goto top" path)
 1.131  21-Apr-2004  christos Replace the statfs() family of system calls with statvfs().
Retain binary compatibility.
 1.130  20-Apr-2004  yamt check_dirty: plug a PHOLD leak. from Greg Oster.
 1.129  26-Feb-2004  yamt branches: 1.129.4;
lfs_putpages: fix a simple_lock mismatch.
 1.128  26-Jan-2004  hannken Fix xxx_strategy() to use the vnode arg instead of bp->b_vp.
 1.127  25-Jan-2004  hannken Make VOP_STRATEGY(bp) a real VOP as discussed on tech-kern.

VOP_STRATEGY(bp) is replaced by one of two new functions:

- VOP_STRATEGY(vp, bp) Call the strategy routine of vp for bp.
- DEV_STRATEGY(bp) Call the d_strategy routine of bp->b_dev for bp.

DEV_STRATEGY(bp) is used only for block-to-block device situations.
 1.126  16-Dec-2003  yamt - reduce code duplication.
- use boolean_t where appropriate.
 1.125  16-Dec-2003  yamt g/c lfs_no_inactive.
 1.124  25-Nov-2003  yamt use FINFOSIZE macro.
 1.123  30-Oct-2003  simonb Remove some assigned-to but otherwise unused variables.
 1.122  25-Oct-2003  christos Fix uninitialized variable warnings.
 1.121  21-Oct-2003  fvdl Correct preempt() calls.
 1.120  18-Oct-2003  yamt be more strict about sa->vp.
(make sure the last lfs_updatemata in lfs_putpages takes effect.)
 1.119  14-Oct-2003  dbj add mnt_iflag field to struct mount for internal flags
mv MNT_GONE, MNT_UNMOUNT and MNT_WANTRDWR to this field
additonally add mnt_writeopcountupper and mnt_writeopcountlower fields
in preparation for pending write suspension support work
bump kernel version to 1.6ZD
 1.118  24-Sep-2003  yamt fix a bug of lfs.

genfs_getpages() can read in more blocks than it should due to faked filesize
of lfs_gop_size(). it's a security problem and it makes gcc3 "internal error"

to fix this,
- in genfs_getpages(), always calculate diskeof and memeof separately
so that filesystems (in this case, lfs) can use different strategies
for them.
- introduce GOP_SIZE_MEM flag and use it to request in-core filesize.
(it was an intention of GOP_SIZE_READ,
but after the above change _READ is not a straightforward name)

after this, no one uses GOP_SIZE_{READ,WRITE} anymore but leave them for now.
 1.117  23-Sep-2003  yamt cleanup IN_ADIROP/VDIROP handling a little.
 1.116  23-Sep-2003  yamt remove unnecessary externs of lfs_do_flush.
 1.115  20-Sep-2003  yamt some comments
 1.114  07-Aug-2003  agc Move UCB-licensed code from 4-clause to 3-clause licence.

Patches provided by Joel Baker in PR 22364, verified by myself.
 1.113  12-Jul-2003  yamt more MP locks.
 1.112  12-Jul-2003  yamt - protect global resource counts with lfs_subsys_lock.
- clean up scattered externs a little.
 1.111  02-Jul-2003  yamt - add a new functions, lfs_writer_enter/leave, and use them instead of
duplicated code fragments.
- add an assertion.
 1.110  02-Jul-2003  yamt drain dirops before aqcuiring seglock. otherwise it might deadlocks.
PR/20676 (Karl Knutsson)
 1.109  29-Jun-2003  fvdl branches: 1.109.2;
Back out the lwp/ktrace changes. They contained a lot of colateral damage,
and need to be examined and discussed more.
 1.108  29-Jun-2003  thorpej Undo part of the ktrace/lwp changes. In particular:
* Remove the "lwp *" argument that was added to vget(). Turns out
that nothing actually used it!
* Remove the "lwp *" arguments that were added to VFS_ROOT(), VFS_VGET(),
and VFS_FHTOVP(); all they did was pass it to vget() (which, as noted
above, didn't use it).
* Remove all of the "lwp *" arguments to internal functions that were added
just to appease the above.
 1.107  28-Jun-2003  darrenr Pass lwp pointers throughtout the kernel, as required, so that the lwpid can
be inserted into ktrace records. The general change has been to replace
"struct proc *" with "struct lwp *" in various function prototypes, pass
the lwp through and use l_proc to get the process pointer when needed.

Bump the kernel rev up to 1.6V
 1.106  07-May-2003  ragge Add a missing ifdef DDB.
 1.105  02-May-2003  perseant Correct arguments to check_dirty, ensuring that all pages in a block are
written if any of them are dirty. Pointed out by yamt.
 1.104  27-Apr-2003  yamt fix a comment.
 1.103  23-Apr-2003  perseant Make LFS work better (though still not "well") as an NFS-exported
filesystem (and other things that needed to be fixed before the tests
would complete), to wit:

* Include the fs ident in the filehandle; improve stale filehandle checks.

* Change definition of blksize() to use the on-dinode size instead of
the inode's i_size, so that fsck_lfs will work properly again.

* Use b_interlock in lfs_vtruncbuf.

* Postpone dirop reclamation until after the seglock has been released,
so that lfs_truncate is not called with the segment lock held.

* Don't loop in lfs_fsync(), just write everything and wait.

* Be more careful about the interlock/uobjlock in lfs_putpages: when we
lose this lock, we have to resynchronize dirtiness of pages in each
block.

* Be sure to always write indirect blocks and update metadata in
lfs_putpages; fixes a bug that caused blocks to be accounted to the
wrong segment.
 1.102  02-Apr-2003  fvdl Add support for UFS2. UFS2 is an enhanced FFS, adding support for
64 bit block pointers, extended attribute storage, and a few
other things.

This commit does not yet include the code to manipulate the extended
storage (for e.g. ACLs), this will be done later.

Originally written by Kirk McKusick and Network Associates Laboratories for
FreeBSD.
 1.101  01-Apr-2003  yamt lfs_strategy is used only for read.
 1.100  28-Mar-2003  perseant Add a sleeper count, to prevent the cleaner from panicing the kernel
when the filesystem is unmounted, relocking the Ifile when its lock is
draining. (We can't use vfs_busy() since the process is sleeping for a
good long time.) Clean up / organize lfs.h, while I'm here.

In lfs_update_single, assert that disk addresses are either negative, or
are still positive when converted to int32_t, to prevent recurrence of a
negative/positive block problem.
 1.99  22-Mar-2003  perseant Unlock ifile inode during streamlined VOP_INACTIVE.
 1.98  21-Mar-2003  perseant KNF (space after keywords).
 1.97  21-Mar-2003  perseant Use VONWORKLST as a heuristic for vnode emptiness, rather than exhaustively
checking the memq.

Take greater care not to dirty the Ifile vnode when unmounting the filesystem.
This should fix a "(vp->v_flag & VONWORKLST) == 0" assertion panic in vgonel
that could occur when unmounting.

Do not allow the Ifile to be mapped for writing.
 1.96  15-Mar-2003  perseant Add simple_lock protection for lfs_seglock and lfs_subsys_pages; these will
be expanded to cover other per-fs and subsystem-wide data as well.

Fix a case of IN_MODIFIED being set without updating lfs_uinodes, resulting
in a "lfs_uinodes < 0" panic.

Fix a deadlock in lfs_putpages arising from the need to busy all pages in a
block; unbusy any that had already been busied before starting over.
 1.95  08-Mar-2003  perseant Take away "#ifdef LFS_UBC".
 1.94  08-Mar-2003  perseant Add an lfs_strategy() that checks to make sure we're not trying to read
where the cleaner is trying to write, instead of tying up the "live"
buffers (or pages).

Fix a bug in the LFS_UBC case where oversized buffers would not be
checksummed correctly, causing uncleanable segments.

Make sure that wakeup(fs->lfs_iocount) is done if fs->lfs_iocount is 1
as well as 0, since we wait in some places for it to drop to 1.

Activate all pages that make it into lfs_gop_write without the segment
lock held, since they must have been dirtied very recently, even if
PG_DELWRI is not set.
 1.93  04-Mar-2003  perseant Make sure we hold the uobjlock when checking for dirty pages, in lfs_vflush.
Note that pages can become dirty without our knowing it, anyway; don't
panic if that happens.
 1.92  02-Mar-2003  perseant Account SEGUSE_ACTIVE correctly so that the automatic segment cleaning
actually happens.

Add a new fcntl call that will write the minimum necessary to checkpoint
(i.e., for on-disk directory structure to be consistent, not including
updates to file data) so that the cleaner can clean segments more quickly
without sacrificing three-way commit for cleaning.
 1.91  01-Mar-2003  yamt use pid_t for pid.
 1.90  25-Feb-2003  perseant Make fs-specific fcntl macros take three arguments (approved wrstuden).
Let LFS use fcntl for cleaner functions.
 1.89  24-Feb-2003  perseant Add lfs_ioctl vnode op, with ioctls to take over cleaner system call
functionality (not including segment clean, since that is now done
automatically as checkpoints happen).
 1.88  23-Feb-2003  perseant Fix a buffer overflow bug in the LFS_UBC case that manifested itself
either as a mysterious UVM error or as "panic: dirty bufs". Verify
maximum size in lfs_malloc.

Teach lfs_updatemeta and lfs_shellsort about oversized cluster blocks from
lfs_gop_write.

When unwiring pages in lfs_gop_write, deactivate them, under the theory
that the pagedaemon wanted to free them last we knew.
 1.87  22-Feb-2003  yamt fix simple_lock/unlock mismatches.
 1.86  20-Feb-2003  perseant Tabify, and fix some comment alignment problems.
 1.85  19-Feb-2003  yamt wire the pages instead of just dequeue'ing them.
advised by Chuck Silvers.
 1.84  17-Feb-2003  perseant Add code to UBCify LFS. This is still behind "#ifdef LFS_UBC" for now
(there are still some details to work out) but expect that to go
away soon. To support these basic changes (creation of lfs_putpages,
lfs_gop_write, mods to lfs_balloc) several other changes were made, to
wit:

* Create a writer daemon kernel thread whose purpose is to handle page
writes for the pagedaemon, but which also takes over some of the
functions of lfs_check(). This thread is started the first time an
LFS is mounted.

* Add a "flags" parameter to GOP_SIZE. Current values are
GOP_SIZE_READ, meaning that the call should return the size of the
in-core version of the file, and GOP_SIZE_WRITE, meaning that it
should return the on-disk size. One of GOP_SIZE_READ or
GOP_SIZE_WRITE must be specified.

* Instead of using malloc(...M_WAITOK) for everything, reserve enough
resources to get by and use malloc(...M_NOWAIT), using the reserves if
necessary. Use the pool subsystem for structures small enough that
this is feasible. This also obsoletes LFS_THROTTLE.

And a few that are not strictly necessary:

* Moves the LFS inode extensions off onto a separately allocated
structure; getting closer to LFS as an LKM. "Welcome to 1.6O."

* Unified GOP_ALLOC between FFS and LFS.

* Update LFS copyright headers to correct values.

* Actually cast to unsigned in lfs_shellsort, like the comment says.

* Keep track of which segments were empty before the previous
checkpoint; any segments that pass two checkpoints both dirty and
empty can be summarily cleaned. Do this. Right now lfs_segclean
still works, but this should be turned into an effectless
compatibility syscall.
 1.83  03-Feb-2003  perseant Don't call a dirop within a dirop: if lfs_rename is actually deleting
a link, call lfs_remove directly before starting dirop rather than
having ufs_rename do it.
 1.82  30-Jan-2003  yamt there's no need to treat VOP_WHITEOUT as dirop
because it modifies only one inode.
 1.81  25-Jan-2003  kleink Fix further printf format warnings for DEBUG, in the wake of daddr_t
having changed.
 1.80  18-Jan-2003  thorpej Merge the nathanw_sa branch.
 1.79  08-Jan-2003  yamt for lfs_remove/lfs_rmdir, keep removed vnodes marked VDIROP.
(backout parts of rev.1.40)
otherwise, directory structures can be corrupted because checkpoints can
occur via eg. lfs_vflush before parent directory is written.
 1.78  08-Jan-2003  yamt in set_dirop/endop, use normal vref/vrele instead of lfs versions
so that we don't miss lfs_inactivate.
 1.77  08-Jan-2003  yamt add assertions.
 1.76  08-Jan-2003  yamt use lfs_unmark_vnode instead of duplicated code fragments.
 1.75  29-Dec-2002  yamt backout assertions in lfs_inactive.
they can be false when unmounting forcibly.
 1.74  28-Dec-2002  christos fix compile problem.
 1.73  28-Dec-2002  yamt avoid warnings without DIAGNOSTIC.

pointed by Andreas Wrede.
 1.72  28-Dec-2002  yamt dirop inode can't be passed to lfs_inactivate.
 1.71  28-Dec-2002  yamt - in lfs_reserve, vref vnodes that we're locking so that cleaner doesn't
try to reclaim them.
(workaround for deadlock noted in the comment in lfs_reserveavail)
- in lfs_rename, mark vnodes which are being moved as well as directry vnodes.
 1.70  26-Dec-2002  yamt - in lfs_reserve, reserve locked buffer count as well.
- don't wait for locking buf in lfs_bwrite_ext to avoid deadlocks.
- skip lfs_reserve when we're doing dirop.
reserve more (for lfs_truncate) in set_dirop instead.

this mostly solves PR 18972. (and hopefully PR 19196)
 1.69  24-Nov-2002  yamt correct locking for lfs_rmdir. PR 18976.
 1.68  23-Oct-2002  jdolecek merge kqueue branch into -current

kqueue provides a stateful and efficient event notification framework
currently supported events include socket, file, directory, fifo,
pipe, tty and device changes, and monitoring of processes and signals

kqueue is supported by all writable filesystems in NetBSD tree
(with exception of Coda) and all device drivers supporting poll(2)

based on work done by Jonathan Lemon for FreeBSD
initial NetBSD port done by Luke Mewburn and Jason Thorpe
 1.67  27-Sep-2002  provos remove trailing \n in panic(). approved perry.
 1.66  22-Sep-2002  jdolecek don't need <sys/conf.h> here
 1.65  16-Jun-2002  perseant For synchronous writes, keep separate i/o counters for each write, so
processes don't have to wait for one another to finish (e.g., nfsd seems
to be a little happier now, though I haven't measured the difference).
Synchronous checkpoints, however, must always wait for all i/o to finish.

Take the contents of the callback functions and have them run in thread
context instead (aiodoned thread). lfs_iocount no longer has to be
protected in splbio(), and quite a bit less of the segment construction
loop needs to be in splbio() as well.

If lfs_markv is handed a block that is not the correct size according to
the inode, refuse to process it. (Formerly it was extended to the "correct"
size.) This is possibly more prone to deadlock, but less prone to corruption.

lfs_segclean now outright refuses to clean segments that appear to have live
bytes in them. Again this may be more prone to deadlock but avoids
corruption.

Replace ufsspec_close and ufsfifo_close with LFS equivalents; this means
that no UFS functions need to know about LFS_ITIMES any more. Remove
the reference from ufs/inode.h.

Tested on i386, test-compiled on alpha.
 1.64  17-May-2002  perseant branches: 1.64.2;
use macros from <sys/queue.h>
 1.63  14-May-2002  perseant branches: 1.63.2;
Phase one of my three-phase plan to make LFS play nice with UBC, and bug-fixes
I found while making sure there weren't any new ones.

* Make the write clusters keep track of the buffers whose blocks they contain.
This should make it possible to (1) write clusters using a page mapping
instead of malloc, if desired, and (2) schedule blocks for rewriting
(somewhere else) if a write error occurs. Code is present to use
pagemove() to construct the clusters but that is untested and will go away
anyway in favor of page mapping.
* DEBUG now keeps a log of Ifile writes, so that any lingering instances of
the "dirty bufs" problem can be properly debugged.
* Keep track of whether the Ifile has been dirtied by various routines that
can be called by lfs_segwrite, and loop on that until it is clean, for
a checkpoint. Checkpoints need to be squeaky clean.
* Warn the user (once) if the Ifile grows larger than is reasonable for their
buffer cache. Both lfs_mountfs and lfs_unmount check since the Ifile can
grow.
* If an inode is not found in a disk block, try rereading the block, under
the assumption that the block was copied to a cluster and then freed.
* Protect WRITEINPROG() with splbio() to fix a hang in lfs_update.
 1.62  27-Apr-2002  perseant Make exported LFSes not panic on the first file create.
 1.61  11-Feb-2002  perseant Include the space taken by inodes in the count made by lfs_check();
make VOP_SETATTR call lfs_check. This prevents large numbers of inode
changes (say, at the end of tar(1)) from filling the buffer cache.
 1.60  18-Dec-2001  chs use the new compatibility routines to allow mmap() to work
(in the same non-coherent fashion that it worked pre-UBC)
until someone has time to do it the right way.
 1.59  23-Nov-2001  chs add spaces for KNF. confirmed to produce identical objects.
 1.58  08-Nov-2001  lukem add RCSID
 1.57  26-Oct-2001  lukem remove #include <ufs/ufs/quota.h> where it was just to appease
<ufs/ufs/inode.h>, since the latter now includes the former. leave the former
in source that obviously uses specific bits of it (for completeness.)
 1.56  22-Sep-2001  sommerfeld branches: 1.56.2;
Add fifo_putpages() placebo so that the vnode's uobj is unlocked.
 1.55  15-Sep-2001  chs a whole bunch of changes to improve performance and robustness under load:

- remove special treatment of pager_map mappings in pmaps. this is
required now, since I've removed the globals that expose the address range.
pager_map now uses pmap_kenter_pa() instead of pmap_enter(), so there's
no longer any need to special-case it.
- eliminate struct uvm_vnode by moving its fields into struct vnode.
- rewrite the pageout path. the pager is now responsible for handling the
high-level requests instead of only getting control after a bunch of work
has already been done on its behalf. this will allow us to UBCify LFS,
which needs tighter control over its pages than other filesystems do.
writing a page to disk no longer requires making it read-only, which
allows us to write wired pages without causing all kinds of havoc.
- use a new PG_PAGEOUT flag to indicate that a page should be freed
on behalf of the pagedaemon when it's unlocked. this flag is very similar
to PG_RELEASED, but unlike PG_RELEASED, PG_PAGEOUT can be cleared if the
pageout fails due to eg. an indirect-block buffer being locked.
this allows us to remove the "version" field from struct vm_page,
and together with shrinking "loan_count" from 32 bits to 16,
struct vm_page is now 4 bytes smaller.
- no longer use PG_RELEASED for swap-backed pages. if the page is busy
because it's being paged out, we can't release the swap slot to be
reallocated until that write is complete, but unlike with vnodes we
don't keep a count of in-progress writes so there's no good way to
know when the write is done. instead, when we need to free a busy
swap-backed page, just sleep until we can get it busy ourselves.
- implement a fast-path for extending writes which allows us to avoid
zeroing new pages. this substantially reduces cpu usage.
- encapsulate the data used by the genfs code in a struct genfs_node,
which must be the first element of the filesystem-specific vnode data
for filesystems which use genfs_{get,put}pages().
- eliminate many of the UVM pagerops, since they aren't needed anymore
now that the pager "put" operation is a higher-level operation.
- enhance the genfs code to allow NFS to use the genfs_{get,put}pages
instead of a modified copy.
- clean up struct vnode by removing all the fields that used to be used by
the vfs_cluster.c code (which we don't use anymore with UBC).
- remove kmem_object and mb_object since they were useless.
instead of allocating pages to these objects, we now just allocate
pages with no object. such pages are mapped in the kernel until they
are freed, so we can use the mapping to find the page to free it.
this allows us to remove splvm() protection in several places.

The sum of all these changes improves write throughput on my
decstation 5000/200 to within 1% of the rate of NetBSD 1.5
and reduces the elapsed time for "make release" of a NetBSD 1.5
source tree on my 128MB pc to 10% less than a 1.5 kernel took.
 1.54  24-Aug-2001  chs branches: 1.54.2;
disable mmap() for LFS until it is fixed.
 1.53  17-Aug-2001  chs add getpages/putpages entries for spec vnodes.
 1.52  24-Jul-2001  assar change vop_symlink and vop_mknod to return vpp (the created node)
refed, so that the caller can actually use it. update callers and
file systems that implement these vnode operations
 1.51  13-Jul-2001  perseant Merge the short-lived perseant-lfsv2 branch into the trunk.

Kernels and tools understand both v1 and v2 filesystems; newfs_lfs
generates v2 by default. Changes for the v2 layout include:

- Segments of non-PO2 size and arbitrary block offset, so these can be
matched to convenient physical characteristics of the partition (e.g.,
stripe or track size and offset).

- Address by fragment instead of by disk sector, paving the way for
non-512-byte-sector devices. In theory fragments can be as large
as you like, though in reality they must be smaller than MAXBSIZE in size.

- Use serial number and filesystem identifier to ensure that roll-forward
doesn't get old data and think it's new. Roll-forward is enabled for
v2 filesystems, though not for v1 filesystems by default.

- The inode free list is now a tailq, paving the way for undelete (undelete
is not yet implemented, but can be without further non-backwards-compatible
changes to disk structures).

- Inode atime information is kept in the Ifile, instead of on the inode;
that is, the inode is never written *just* because atime was changed.
Because of this the inodes remain near the file data on the disk, rather
than wandering all over as the disk is read repeatedly. This speeds up
repeated reads by a small but noticeable amount.

Other changes of note include:

- The ifile written by newfs_lfs can now be of arbitrary length, it is no
longer restricted to a single indirect block.

- Fixed an old bug where ctime was changed every time a vnode was created.
I need to look more closely to make sure that the times are only updated
during write(2) and friends, not after-the-fact during a segment write,
and certainly not by the cleaner.
 1.50  22-Jan-2001  jdolecek branches: 1.50.2; 1.50.4; 1.50.6;
make filesystem vnodeop, specop, fifoop and vnodeopv_* arrays const
 1.49  18-Nov-2000  toshii Make buildable again.
The previous commit was a backout of rev. 1.45, which must be an accident.
 1.48  17-Nov-2000  perseant Correct accounting of lfs_avail, locked_queue_count, and locked_queue_bytes.
(PR #11468). In the case of fragment allocation, check to see if enough
space is available before extending a fragment already scheduled for writing.

The locked_queue_* variables indicate the number of buffer headers and bytes,
respectively, that are unavailable to getnewbuf() because they are locked up
waiting for LFS to flush them; make sure that that is actually what we're
counting, i.e., never count malloced buffers, and always use b_bufsize instead
of b_bcount.

If DEBUG is defined, the periodic calls to lfs_countlocked will now complain
if either counter is incorrect. (In the future lfs_countlocked will not need
to be called at all if DEBUG is not defined.)
 1.47  12-Nov-2000  perseant Do not needlessly dirty segment table blocks during lfs_segwrite,
preventing needless disk activity when the filesystem is idle. (PR #10979.)
 1.46  14-Oct-2000  perseant In lfs_truncate, don't overcount the real blocks removed from the inode,
when deallocating a fragment that has not made it to disk yet.

Also, during dirops, give the directory vnode an extra reference in
SET_DIROP, to ensure its continued existence during SET_ENDOP, preventing
a possible NULL-dereference there.

These two changes should close PR #11064.
 1.45  19-Sep-2000  fvdl Adapt for VOP_FSYNC parameter change.
 1.44  09-Sep-2000  perseant Various bug-fixes to LFS, to wit:


Kernel:

* Add runtime quantity lfs_ravail, the number of disk-blocks reserved
for writing. Writes to the filesystem first reserve a maximum amount
of blocks before their write is allowed to proceed; after the blocks
are allocated the reserved total is reduced by a corresponding amount.

If the lfs_reserve function cannot immediately reserve the requested
number of blocks, the inode is unlocked, and the thread sleeps until
the cleaner has made enough space available for the blocks to be
reserved. In this way large files can be written to the filesystem
(or, smaller files can be written to a nearly-full but thoroughly
clean filesystem) and the cleaner can still function properly.

* Remove explicit switching on dlfs_minfreeseg from the kernel code; it
is now merely a fs-creation parameter used to compute dlfs_avail and
dlfs_bfree (and used by fsck_lfs(8) to check their accuracy). Its
former role is better assumed by a properly computed dlfs_avail.

* Bounds-check inode numbers submitted through lfs_bmapv and lfs_markv.
This prevents a panic, but, if the cleaner is feeding the filesystem
the wrong data, you are still in a world of hurt.

* Cleanup: remove explicit references of DEV_BSIZE in favor of
btodb()/dbtob().

lfs_cleanerd:

* Make -n mean "send N segments' blocks through a single call to
lfs_markv". Previously it had meant "clean N segments though N calls
to lfs_markv, before looking again to see if more need to be cleaned".
The new behavior gives better packing of direct data on disk with as
little metadata as possible, largely alleviating the problem that the
cleaner can consume more disk through inefficient use of metadata than
it frees by moving dirty data away from clean "holes" to produce
entirely clean segments.

* Make -b mean "read as many segments as necessary to write N segments
of dirty data back to disk", rather than its former meaning of "read
as many segments as necessary to free N segments worth of space". The
new meaning, combined with the new -n behavior described above,
further aids in cleaning storage efficiency as entire segments can be
written at once, using as few blocks as possible for segment summaries
and inode blocks.

* Make the cleaner take note of segments which could not be cleaned due
to error, and not attempt to clean them until they are entirely free
of dirty blocks. This prevents the case in which a cleanerd running
with -n 1 and without -b (formerly the default) would spin trying
repeatedly to clean a corrupt segment, while the remaining space
filled and deadlocked the filesystem.

* Update the lfs_cleanerd manual page to describe all the options,
including the changes mentioned here (in particular, the -b and -n
flags were previously undocumented).

fsck_lfs:

* Check, and optionally fix, lfs_avail (to an exact figure) and
lfs_bfree (within a margin of error) in pass 5.

newfs_lfs:

* Reduce the default dlfs_minfreeseg to 1/20 of the total segments.

* Add a warning if the sgs disklabel field is 16 (the default for FFS'
cpg, but not usually desirable for LFS' sgs: 5--8 is a better range).

* Change the calculation of lfs_avail and lfs_bfree, corresponding to
the kernel changes mentioned above.

mount_lfs:

* Add -N and -b options to pass corresponding -n and -b options to
lfs_cleanerd.

* Default to calling lfs_cleanerd with "-b -n 4".


[All of these changes were largely tested in the 1.5 branch, with the
idea that they (along with previous un-pulled-up work) could be applied
to the branch while it was still in ALPHA2; however my test system has
experienced corruption on another filesystem (/dev/console has gone
missing :^), and, while I believe this unrelated to the LFS changes, I
cannot with good conscience request that the changes be pulled up.]
 1.43  05-Jul-2000  perseant Clean up accounting of lfs_uinodes (dirty but unwritten inodes).

Make lfs_uinodes a signed quantity for debugging purposes, and set it to
zero as fs mount time.

Enclose setting/clearing of the dirty flags (IN_MODIFIED, IN_ACCESSED,
IN_CLEANING) in macros, and use those macros everywhere. Make
LFS_ITIMES use these macros; updated the ITIMES macro in inode.h to know
about this. Make ufs_getattr use ITIMES instead of FFS_ITIMES.
 1.42  01-Jul-2000  perseant Move SET_ENDOP after vrele to avoid deactivating vnode twice, if
SET_ENDOP triggers a write.
 1.41  28-Jun-2000  mrg remove include of <vm/vm.h> and <uvm/uvm_extern.h>
 1.40  27-Jun-2000  perseant Fixes associated with filling an LFS:

Change the space computation to appear to change the size of the *disk*
rather than the *bytes used* when more segment summaries and inode
blocks are written. Try to estimate the amount of space that these will
take up when more files are written, so the disk size doesn't change too
much.

Regularize error returns from lfs_valloc, lfs_balloc, lfs_truncate: they
now fail entirely, rather than succeeding half-way and leaving the fs in
an inconsistent state.

Rewrite lfs_truncate, mostly stealing from ffs_truncate. The old
lfs_truncate had difficulty truncating a large file to a non-zero size
(indirect blocks were not handled appropriately).

Unmark VDIROP on fvp after ufs_remove, ufs_rmdir, so these can be
reclaimed immediately: this vnode would not be written to disk again
anyway if the removal succeeded, and if it failed, no directory
operation occurred.

ufs_makeinode and ufs_mkdir now remove IN_ADIROP on error.
 1.39  22-Jun-2000  perseant Update lfs_vunref for the fact that now a vnode can be locked with no
references (locked for VOP_INACTIVE at the end of vrele) and it's okay.
Check the return value of lfs_vref where appropriate.
Fixes PR #s 10285 and 10352.
 1.38  31-May-2000  perseant branches: 1.38.2;
update for IN_ACCESSED changes
 1.37  27-May-2000  perseant branches: 1.37.2;
Prevent dirops from getting around lfs_check and wedging the buffer cache.
All the dirop vnops now mark the inodes with a new flag, IN_ADIROP, which
is removed as soon as the dirop is done (as opposed to VDIROP which stays
until the file is written). To address one issue raised in PR#9357.
 1.36  13-May-2000  perseant Change the sementics of the last parameter from a boolean ("waitfor") to
a set of flags ("flags"). Two flags are defined, UPDATE_WAIT and
UPDATE_DIROP.

Under the old semantics, VOP_UPDATE would block if waitfor were set,
under the assumption that directory operations should be done
synchronously. At least LFS and FFS+softdep do not make this
assumption; FFS+softdep got around the problem by enclosing all relevant
calls to VOP_UPDATE in a "if(!DOINGSOFTDEP(vp))", while LFS simply
ignored waitfor, one of the reasons why NFS-serving an LFS filesystem
did not work properly.

Under the new semantics, the UPDATE_DIROP flag is a hint to the
fs-specific update routine that the call comes from a dirop routine, and
should be wait for, or not, accordingly.

Closes PR#8996.
 1.35  30-Mar-2000  augustss Remove register declarations.
 1.34  15-Dec-1999  perseant Fix error returns on lfs vnops so that locks and reference counts are
preserved. Handle dirop accounting in lfs_vfree for this case as well.
May address PR#8823.
 1.33  03-Dec-1999  perseant Handle the case of a vnode flush while dirops are active correctly in
lfs_segwrite. Also, make sure a flush is called in SET_DIROP before sleeping
on its results. Addresses PR #8863.
 1.32  15-Nov-1999  fvdl Add Kirk McKusick's soft updates code to the trunk. Not enabled by
default, as the copyright on the main file (ffs_softdep.c) is such
that is has been put into gnusrc. options SOFTDEP will pull this
in. This code also contains the trickle syncer.

Bump version number to 1.4O
 1.31  06-Nov-1999  perseant branches: 1.31.2;
Address ufs_hashlock/ufs_ihashins protocol bug, discovered while doing a
post-mortem of a production machine. Also, take the active dirop
count off of the fs and make it global (since it is measuring a global
resource) and tie the threshold value LFS_MAXDIROP to desiredvnodes.
 1.30  05-Nov-1999  perseant Better fix for PR# 8577: before setting dirops, check for cross-device
rename and error out. This avoids possible problems with attempting
rename between two LFSs.
 1.29  01-Nov-1999  perseant Check that the destination vnode is on an LFS before trying to twiddle
its superblock. Fixes PR#8577.
 1.28  03-Sep-1999  perseant branches: 1.28.2; 1.28.4; 1.28.6;
Make changes that will allow an LFS filesystem to be used as the root
filesystem. In particular,

- Fix mknod deadlock, described in PR 8172.
- Enable lfs_mountroot.
- Make lfs_writevnodes treat filesystems mounted on lfs device nodes properly,
by flushing that device rather than trying to add blocks to the device inode.

This, in combination with lfs boot blocks, will allow operation of an all-lfs
system.
 1.27  03-Aug-1999  wrstuden Add support for fcntl(2) to generate VOP_FCNTL calls. Any fcntl
call with F_FSCTL set and F_SETFL calls generate calls to a new
fileop fo_fcntl. Add genfs_fcntl() and soo_fcntl() which return 0
for F_SETFL and EOPNOTSUPP otherwise. Have all leaf filesystems
use genfs_fcntl().

Reviewed by: thorpej
Tested by: wrstuden
 1.26  12-Apr-1999  perseant Disallow threshold-initiated cache flush when dirops are active. Also, make
SET_ENDOP use lfs_check instead of inlining most of it.
 1.25  29-Mar-1999  perseant branches: 1.25.2;
Fix the other missing dirop wakeup
 1.24  25-Mar-1999  perseant Since dirop vnodes can't be flushed, they hold a reference until their
dirop is completely written to disk. This means that ordinary calls to
ufs vnops which would ordinarily call VOP_INACTIVE through vrele/vput,
don't. This patch detects that condition after such vnops have been
run, and calls VOP_INACTIVE if it would ordinarily have been called by
the ufs call.
 1.23  25-Mar-1999  perseant clean up unused/required #ifdefs
 1.22  10-Mar-1999  perseant New sources should leave the LFS in a more-or-less working state. Changes
include:

- DIROP segregation is enabled, and greater care is taken
to make sure that a checkpoint completes. Fsck is not
needed to remount the filesystem.
- Several checks to make sure that the LFS subsystem does not
overuse various resources (memory, in particular).
- The cleaner routines, lfs_markv in particular, are completely
rewritten. A buffer overflow is removed. Greater care is taken
to ensure that inodes come from where lfs_cleanerd say they come
from (so we know nothing has changed since lfs_bmapv was called).
- Fragment allocation is fixed, so that writes beyond end-of-file
do the right thing.
 1.21  05-Mar-1999  mycroft Pass null pointers to VOP_UPDATE rather than having all the callers fetch the
current time themselves.
 1.20  06-Nov-1998  cgd argument to dbtob needs to be cast to u_quad_t here to avoid shift lossage
 1.19  01-Sep-1998  thorpej Use the pool allocator and the "nointr" pool page allocator for LFS inodes.
 1.18  24-Jun-1998  sommerfe Always include fifos; "not an option any more".
 1.17  22-Jun-1998  sommerfe defopt for options FIFO
 1.16  05-Jun-1998  kleink Convert fsync vnode operator implementations and usage from the old `waitfor'
argument and MNT_WAIT/MNT_NOWAIT to `flags' and FSYNC_WAIT.
 1.15  01-Mar-1998  fvdl Merge with Lite2 + local changes
 1.14  11-Jun-1997  bouyer Add support for ext2fs, this needed a few modifications to ufs/ufs/inode.h:
- added an "union inode_ext" to struct inode, for the per-fs extentions.
For now only ext2fs uses it.
- i_din is now an union:
union {
struct dinode ffs_din; /* 128 bytes of the on-disk dinode. */
struct ext2fs_dinode e2fs_din; /* 128 bytes of the on-disk dinode. */
} i_din
Added a lot of #define i_ffs_* and i_e2fs_* to access the fields.
- Added two macros: FFS_ITIMES and EXT2FS_ITIMES. ITIMES calls the rigth
macro, depending on the time of the inode. ITIMES is used where necessary,
FFS_ITIMES and EXT2FS_ITIMES in other places.
 1.13  07-Sep-1996  mycroft Implement poll(2).
 1.12  01-Sep-1996  mycroft Add a set of generic file system operations that most file systems use.
Also, fix some time stamp bogosities.
 1.11  11-May-1996  mycroft Change VOP_UPDATE() semantics:
* Make 2nd and 3rd args timespecs, not timevals.
* Consistently pass a Boolean as the 4th arg (except in LFS).
Also, fix ffs_update() and lfs_update() to actually change the nsec fields.
 1.10  09-Feb-1996  christos lfs prototypes
 1.9  09-Feb-1996  mycroft Fix vop_link, vop_symlink, and vop_remove semantics in several ways:
* Change the argument names to vop_link so they actually make sense.
* Implement vop_link and vop_symlink for all file systems, so they do proper
cleanup.
* Require the file system to decide whether or not linking and unlinking of
directories is allowed, and disable it for all current file systems.
 1.8  01-Feb-1996  jtc Rename struct timespec fields to conform to POSIX.1b
 1.7  15-Jun-1995  cgd compensate for timeval/timespec/stat structure changes.
 1.6  14-Dec-1994  mycroft Sync with CSRG.
 1.5  13-Dec-1994  mycroft Not ready for part of the previous change yet...
 1.4  13-Dec-1994  mycroft Turn lease_check() into a vnode op, per CSRG.
 1.3  20-Oct-1994  cgd update for new syscall args description mechanism, and deal safely
with wider types.
 1.2  29-Jun-1994  cgd New RCS ID's, take two. they're more aesthecially pleasant, and use 'NetBSD'
 1.1  08-Jun-1994  mycroft branches: 1.1.1;
Update to 4.4-Lite fs code, with local changes.
 1.1.1.2  01-Mar-1998  fvdl Import 4.4BSD-Lite2
 1.1.1.1  01-Mar-1998  fvdl Import 4.4BSD-Lite for reference
 1.25.2.6  15-Jan-2000  he Pull up revision 1.34 (requested by perseant):
Fix error returns on lfs vnops so that locks and reference counts
are preserved. Handle dirop accounting in lfs_vfree for this
case as well. Addresses PR#8823.
 1.25.2.5  15-Jan-2000  he Pull up revision 1.28 (requested by perseant):
Address problems related to using an LFS filesystem as the root
filesystem, including mknod hangs. Fixes PR#8172 and PR#9072.
 1.25.2.4  18-Dec-1999  he Pull up revision 1.33 (requested by perseant):
Handle the case of a vnode flush while dirops are active correctly
in lfs_segwrite. Also, make sure a flush is called in SET_DIROP
before sleeping on its results. Addresses PR#8863.
 1.25.2.3  17-Dec-1999  he Pull up revision 1.31 (requested by perseant):
Address locking protocol error for inode hash, and make the
maximum number of active dirops a global quantity.
 1.25.2.2  08-Nov-1999  cgd pull up revs 1.29-1.30 from trunk (requested by perseant):
Check for cross-device rename before setting up dirop markers in
lfs_rename. Addresses PR#8577.
 1.25.2.1  13-Apr-1999  perseant branches: 1.25.2.1.2;
Pull-up of changes made to the trunk on Sunday [1.25->1.26], to wit:

Take out the `#ifdef USE_UFSHASH'; use ufs_hashlock to lock the inode free
list instead of free_lock.

Fix inode reporting in lfs_statfs (the meaning of f_files and f_ffree was
reversed).

Fix "lfs_ifind: dinode xxx not found" panic. When inodes were freed, then
immediately reloaded, their dinodes were located in an inode block which
was not on disk at the advertized location, nor in the cache (although it
would be flushed to disk next segment write). Fix this by using getblk()
instead of lfs_newbuf() for inode blocks.

Better checking for held inode locks in lfs_fastvget, for a number of
error conditions. Also change the default setting of lfs_clean_vnhead to
0, which seems to make the locking problems go away (although this is
difficult to test as I can't reliably reproduce them).

Make sure that the wakeup occurs for vnodes that lfs_update might be
sleeping on (nodes which are not marked IN_MODIFIED/IN_CLEANING, but which
have dirty buffers), by marking them with the appropriate flag if
dirtybuffers were added while the write was in progress.

Fix block counting during file truncation, if not truncating to zero.

Disallow threshold-initiated cache flush when dirops are active. Also,
make SET_ENDOP use lfs_check instead of inlining most of it.

Improve the debugging printfs in the cleaner syscalls (in particular, make
it obvious that they're coming from lfs).

Check the superblock version field, and refuse to mount the filesystem if
the version number is higher than we know about. This allows, e.g.,
changes in the format of the ifile, segment size restrictions and
boundaries, etc., which would not affect existing fields in the
superblock, but which would drastically affect the filesystem, to be
smoothly integrated at a later date.
 1.25.2.1.2.5  31-Aug-1999  perseant Rudimentary support for LFS under UBC:

- LFS-specific VOP_BALLOC and VOP_PUTPAGES vnode ops.

- getblk VREG panic #ifdef'd out (can be reinstated when Ifile is
internalized and Ifile can be made another type from VREG)

- interface to VOP_PUTPAGES changed to pass all pager flags, not
just sync. FS putpages routines must know about the pager flags.

- new LFS magic disk address, -2 ("unwritten"), meaning accounted for
but not assigned to a fixed disk location (since LFS does these two
things separately, and the previous accounting method using buffer
headers no longer will work). Changed references to (foo == (daddr_t)-1)
to (foo < 0). Since disk drivers reject all addresses < 0, this should
not present a problem for other FSs.
 1.25.2.1.2.4  11-Jul-1999  chs add placeholders for getpages/putpages.
 1.25.2.1.2.3  02-Jul-1999  thorpej Take two at making a non-converted LFS work in a UBC kernel.
 1.25.2.1.2.2  21-Jun-1999  thorpej Pull in ffs_extern.h to get ffs_balloc_range() prototype for
ufs_readwrite.c
 1.25.2.1.2.1  21-Jun-1999  thorpej Sync w/ -current.
 1.28.6.2  27-Dec-1999  wrstuden Pull up to last week's -current.
 1.28.6.1  21-Dec-1999  wrstuden Initial commit of recent changes to make DEV_BSIZE go away.

Runs on i386, needs work on other arch's. Main kernel routines should be
fine, but a number of the stand programs need help.

cd, fd, ccd, wd, and sd have been updated. sd has been tested with non-512
byte block devices. vnd, raidframe, and lfs need work.

Non 2**n block support is automatic for LKM's and conditional for kernels
on "options NON_PO2_BLOCKS".
 1.28.4.2  15-Nov-1999  fvdl Sync with -current
 1.28.4.1  19-Oct-1999  fvdl Bring in Kirk McKusick's FFS softdep code on a branch.
 1.28.2.3  11-Feb-2001  bouyer Sync with HEAD.
 1.28.2.2  22-Nov-2000  bouyer Sync with HEAD.
 1.28.2.1  20-Nov-2000  bouyer Update thorpej_scsipi to -current as of a month ago
 1.31.2.2  06-Nov-1999  perseant Address ufs_hashlock/ufs_ihashins protocol bug, discovered while doing a
post-mortem of a production machine. Also, take the active dirop
count off of the fs and make it global (since it is measuring a global
resource) and tie the threshold value LFS_MAXDIROP to desiredvnodes.
 1.31.2.1  06-Nov-1999  perseant file lfs_vnops.c was added on branch comdex-fall-1999 on 1999-11-06 20:33:07 +0000
 1.37.2.1  22-Jun-2000  minoura Sync w/ netbsd-1-5-base.
 1.38.2.6  03-Feb-2001  he Pull up revisions 1.47-1.49 (requested by perseant):
o Don't write anything if the filesystem is idle (PR#10979).
o Close up accounting holes in LFS' accounting of immediately-
available-space, number of clean segments, and amount of dirty
space taken up by metadata (PR#11468, PR#11470, PR#11534).
 1.38.2.5  14-Dec-2000  he Pull up revision 1.45 (requested by fvdl):
Improve NFS performance, possibly with as much as 100% in
throughput. Please note: this implies a kernel interface change,
VOP_FSYNC gains two arguments.
 1.38.2.4  01-Nov-2000  tv Fix pullup of 1.46 [perseant, toshii]:
In lfs_truncate, don't overcount the real blocks removed from the inode,
when deallocating a fragment that has not made it to disk yet.

Also, during dirops, give the directory vnode an extra reference in
SET_DIROP, to ensure its continued existence during SET_ENDOP, preventing
a possible NULL-dereference there.

These two changes should close PR #11064.
 1.38.2.3  01-Nov-2000  tv Pullup 1.46 [perseant, toshii]:
In lfs_truncate, don't overcount the real blocks removed from the inode,
when deallocating a fragment that has not made it to disk yet.

Also, during dirops, give the directory vnode an extra reference in
SET_DIROP, to ensure its continued existence during SET_ENDOP, preventing
a possible NULL-dereference there.

These two changes should close PR #11064.
 1.38.2.2  14-Sep-2000  perseant Pull up recent LFS kernel changes (approved by thorpej):

ufs/ufs/inode.h, 1.20--1.22 (add i_lfs_effnblks extension ;
make ITIMES aware of LFS_ITIMES;
_LKM protection so userland progs
compile)
ufs/ufs/ufs_vnops.c, 1.69, 1.71 (remove IN_ADIROP;
use ITIMES instead of FFS_ITIMES)
ufs/ufs/ufs_readwrite.c, 1.27 (use lfs_reserve in lfs_write)
ufs/lfs/lfs.h, 1.26--1.32 (define LFS_EST_* macros ;
change MIN_FREE_SEGS to lfs_minfreesegs ;
add avail and bfree to CLEANERINFO ;
change lfs_uinodes to signed ;
change lfs_dmeta to signed ;
add whitespace to line up structure
members ;
explicit cast to int32_t in LFS_EST_*
macros)
ufs/lfs/lfs_alloc.c, back out 1.34.2.3 (pullups of 1.39, 1.40);
then pull up 1.38 (clean up on error)
1.39--1.43 (restore fvdl's ufs_hashlock fix ;
restore fvdl's ufs_hashlock fix ;
set i_lfs_effnblks ;
use UINO macros ;
add comments and fix long lines)
ufs/lfs/lfs_balloc.c, 1.19 (don't succeed halfway)
1.21--1.25 (use i_lfs_effnblks ;
fix i_lfs_effnblks computation and
quieten ;
fix i_ffs_blocks in unwritten fragment ;
remove useless debugging check ;
add comments and (c) 2000)
ufs/lfs/lfs_bio.c, 1.24--1.30 (cleanup and make lfs_flush_fs take
"struct lfs *" instead of "struct
mount *" ;
use lfs_minfreeseg instead of
MIN_FREE_SEGS ;
use UINO macros, and copy bfree/avail
to CLEANERINFO ;
add lfs_reserve function ;
1.28--1.30 fix printf formatting)
ufs/lfs/lfs_cksum.c, 1.13 (add (c) 2000)
ufs/lfs/lfs_debug.c, 1.11 (use btodb instead of DEV_BSIZE)
ufs/lfs/lfs_extern.h, 1.18, 1.20--1.21 (function prototype changes)
ufs/lfs/lfs_inode.c, 1.38 (rewrite lfs_truncate from
ffs_truncate)
1.40--1.44 (count written and unwritten blocks
seperately ;
use disk block units instead of bytes ;
remove unnecessary "mod" variable ;
correct B_DELWRI to avoid bawrite panic ;
use lfs_reserve)
ufs/lfs/lfs_segment.c, 1.52-1.59 (use lfs_dmeta to note used summaries ;
check for UNWRITTEN in indirect blocks ;
more debugging stuff inside #ifdef
DEBUG_LFS ;
use LK_CANRECURSE ;
don't drop dirty indirect blocks ;
use UINO macros ;
don't hose the free list ;
use btodb() instead of DEV_BSIZE ;
make it compile again (oops))
ufs/lfs/lfs_subr.c, 1.16--1.17 (check for locked inodes before
changing ;
use btodb() instead of DEV_BSIZE, (c)
2000)
ufs/lfs/lfs_syscalls.c, back out 1.41.4.2 (fvdl's ufs_hashlock fix);
then pull up 1.43 (use lfs_dmeta)
1.44--1.45 (restore fvdl's ufs_hashlock fix)
1.46--1.47 (fix lfs_avail leakage from sblock
segments ;
use UINO macros)
1.49 (bounds-check inode numbers in
lfs_markv)
ufs/lfs/lfs_vfsops.c, 1.53 (use LFS_EST_* macros in lfs_statfs)
1.56--1.58 (initialize lfs_minfreeseg, lfs_effnblk ;
initialize lfs_uinodes ;
initialize lfs_ravail)
ufs/lfs/lfs_vnops.c, 1.40 (remove VDIROP from removed files)
1.42--1.44 (move SET_ENDOP below the removal of
VDIROP ;
use UINO macros and add lfs_itimes
function ;
use lfs_reserve in dirops)
 1.38.2.1  22-Jun-2000  perseant Pull up lfs_vunref fix from the trunk.
 1.50.6.9  10-Oct-2002  jdolecek sync kqueue with -current; this includes merge of gehenna-devsw branch,
merge of i386 MP branch, and part of autoconf rototil work
 1.50.6.8  26-Sep-2002  jdolecek hook in genfs_kqfilter(), kevents seem to work fine
 1.50.6.7  23-Sep-2002  jdolecek add spec kqfilter vnode op
 1.50.6.6  22-Sep-2002  jdolecek add fifo_kqfilter() to fifo ops, to switch on support for kevents
 1.50.6.5  23-Jun-2002  jdolecek catch up with -current on kqueue branch
 1.50.6.4  16-Mar-2002  jdolecek Catch up with -current.
 1.50.6.3  10-Jan-2002  thorpej Sync kqueue branch with -current.
 1.50.6.2  25-Aug-2001  thorpej Merge Aug 24 -current into the kqueue branch.
 1.50.6.1  03-Aug-2001  lukem update to -current
 1.50.4.2  02-Jul-2001  perseant Change disk addressing unit to be the fragment, instead of the disk sector.
All quantities in the superblock, inodes, indirect blocks, etc. refer now
to this abstract unit (called "fsb" as it is in FFS) instead of disk sectors;
as a consequence segment summary blocks have to be multiples of a fragment in
size. In v1 filesystems, compatibility code ensures that 1 fsb == 1 sector,
regardless of fragment size.

Fragments can now range in size between 512 and 32k; in the event that
LFS_LABELPAD (8k) is smaller than the disk address unit size, an extra
proto-superblock is kept at 8k from the beginning of the disk, to be used
*only* to locate the real superblocks. (Not all of the userland knows about
this yet.)

Almost all of this was done not by me, but by joff.
 1.50.4.1  29-Jun-2001  perseant Get rid of __P(), protoizing where it had not already been done
 1.50.2.15  08-Jan-2003  thorpej Sync with HEAD.
 1.50.2.14  08-Jan-2003  thorpej Sync with HEAD.
 1.50.2.13  29-Dec-2002  thorpej Sync with HEAD.
 1.50.2.12  11-Dec-2002  thorpej Sync with HEAD.
 1.50.2.11  11-Nov-2002  nathanw Catch up to -current
 1.50.2.10  18-Oct-2002  nathanw Catch up to -current.
 1.50.2.9  24-Jun-2002  nathanw Curproc->curlwp renaming.

Change uses of "curproc->l_proc" back to "curproc", which is more like the
original use. Bare uses of "curproc" are now "curlwp".

"curproc" is now #defined in proc.h as ((curlwp) ? (curlwp)->l_proc) : NULL)
so that it is always safe to reference curproc (*de*referencing curproc
is another story, but that's always been true).
 1.50.2.8  20-Jun-2002  nathanw Catch up to -current.
 1.50.2.7  28-Feb-2002  nathanw Catch up to -current.
 1.50.2.6  08-Jan-2002  nathanw Catch up to -current.
 1.50.2.5  14-Nov-2001  nathanw Catch up to -current.
 1.50.2.4  26-Sep-2001  nathanw Catch up to -current.
Again.
 1.50.2.3  21-Sep-2001  nathanw Catch up to -current.
 1.50.2.2  24-Aug-2001  nathanw Catch up with -current.
 1.50.2.1  05-Mar-2001  nathanw Initial commit of scheduler activations and lightweight process support.
 1.54.2.1  01-Oct-2001  fvdl Catch up with -current.
 1.56.2.1  12-Nov-2001  thorpej Sync the thorpej-mips-cache branch with -current.
 1.63.2.2  20-Jun-2002  gehenna catch up with -current.
 1.63.2.1  30-May-2002  gehenna Catch up with -current.
 1.64.2.1  20-Jun-2002  lukem Pull up revision 1.65 (requested by perseant in ticket #325):
For synchronous writes, keep separate i/o counters for each write, so
processes don't have to wait for one another to finish (e.g., nfsd seems
to be a little happier now, though I haven't measured the difference).
Synchronous checkpoints, however, must always wait for all i/o to finish.
Take the contents of the callback functions and have them run in thread
context instead (aiodoned thread). lfs_iocount no longer has to be
protected in splbio(), and quite a bit less of the segment construction
loop needs to be in splbio() as well.
If lfs_markv is handed a block that is not the correct size according to
the inode, refuse to process it. (Formerly it was extended to the "correct"
size.) This is possibly more prone to deadlock, but less prone to corruption.
lfs_segclean now outright refuses to clean segments that appear to have live
bytes in them. Again this may be more prone to deadlock but avoids
corruption.
Replace ufsspec_close and ufsfifo_close with LFS equivalents; this means
that no UFS functions need to know about LFS_ITIMES any more. Remove
the reference from ufs/inode.h.
Tested on i386, test-compiled on alpha.
 1.109.2.11  10-Nov-2005  skrll Sync with HEAD. Here we go again...
 1.109.2.10  01-Apr-2005  skrll Sync with HEAD.
 1.109.2.9  08-Mar-2005  skrll Sync with HEAD.
 1.109.2.8  04-Mar-2005  skrll Sync with HEAD.

Hi Perry!
 1.109.2.7  04-Feb-2005  skrll Sync with HEAD.
 1.109.2.6  30-Oct-2004  skrll Reduced diff to HEAD by restoring the struct proc * argument to lfs_bmapv
 1.109.2.5  21-Sep-2004  skrll Fix the sync with head I botched.
 1.109.2.4  18-Sep-2004  skrll Sync with HEAD.
 1.109.2.3  24-Aug-2004  skrll Undo part of the ktrace/lwp changes. In particular:
* Remove the "lwp *" argument that was added to vget(). Turns out
that nothing actually used it!
* Remove the "lwp *" arguments that were added to VFS_ROOT(), VFS_VGET(),
and VFS_FHTOVP(); all they did was pass it to vget() (which, as noted
above, didn't use it).
* Remove all of the "lwp *" arguments to internal functions that were added
just to appease the above.
 1.109.2.2  03-Aug-2004  skrll Sync with HEAD
 1.109.2.1  02-Jul-2003  darrenr Apply the aborted ktrace-lwp changes to a specific branch. This is just for
others to review, I'm concerned that patch fuziness may have resulted in some
errant code being generated but I'll look at that later by comparing the diff
from the base to the branch with the file I attempt to apply to it. This will,
at the very least, put the changes in a better context for others to review
them and attempt to tinker with removing passing of 'struct lwp' through
the kernel.
 1.129.4.1  10-May-2005  riz Pull up the following revisions (requested by perseant in ticket #1281):

1.8 sys/ufs/lfs/TODO
1.75 sys/ufs/lfs/lfs.h (via patch)
1.74 sys/ufs/lfs/lfs_alloc.c (via patch)
1.49, 1.51 sys/ufs/lfs/lfs_balloc.c (1.51 via patch)
1.78 sys/ufs/lfs/lfs_bio.c
1.62 sys/ufs/lfs/lfs_extern.h (via patch)
1.156 sys/ufs/lfs/lfs_segment.c (via patch)
1.48 sys/ufs/lfs/lfs_subr.c
1.101 sys/ufs/lfs/lfs_syscalls.c
1.163 sys/ufs/lfs/lfs_vfsops.c (via patch)
1.134 sys/ufs/lfs/lfs_vnops.c (via patch)
1.61 sys/ufs/ufs/ufs_readwrite.c (via patch)

1.20 libexec/lfs_cleanerd/clean.h (via patch)
1.52 libexec/lfs_cleanerd/cleanerd.c (via patch)
1.41 libexec/lfs_cleanerd/library.c (via patch)

1.4 regress/sys/fs/lfs/newfs_fsck/Makefile
1.2 regress/sys/fs/lfs/newfs_fsck/mkfs_mount
1.2 regress/sys/fs/lfs/newfs_fsck/smallfiles
1.3 sbin/fsck_lfs/bufcache.c
1.3 sbin/fsck_lfs/bufcache.h
1.3 sbin/fsck_lfs/lfs.h
1.8 sbin/fsck_lfs/lfs.c (via patch)
1.8 sbin/fsck_lfs/pass3.c (via patch)
1.18 sbin/fsck_lfs/pass0.c (via patch)
1.18 sbin/fsck_lfs/utilities.c (via patch)
1.7 sbin/fsck_lfs/segwrite.c
1.19 sbin/fsck_lfs/setup.c (via patch)
1.3 sbin/newfs_lfs/Makefile
0 sbin/newfs_lfs/lfs.c (yes, remove it)
1.1 sbin/newfs_lfs/make_lfs.c
1.15 sbin/newfs_lfs/newfs.c (via patch)

Various minor LFS improvements.

Kernel:

* Note when lfs_putpages(9) thinks it is not going to be writing any
pages before calling genfs_putpages(9). This prevents a situation in
which blocks can be queued for writing without a segment header.
* Correct computation of NRESERVE(), though it is still a gross
overestimate in most cases. Note that if NRESERVE() is too high, it
may be impossible to create files on the filesystem. We catch this
case on filesystem mount and refuse to mount r/w.
* Allow filesystems to be mounted whose block size is == MAXBSIZE.
* Somewhere along the line, ufs_bmaparray(9) started mangling UNWRITTEN
entries in indirect blocks again, triggering a failed assertion "daddr
<= LFS_MAX_DADDR". Explicitly convert to and from int32_t to correct
this. Should fix PR #29045.
* Add a high-water mark for the number of dirty pages any given LFS can
hold before triggering a flush. This is settable by sysctl, but off
(zero) by default.
* Be more careful about the MAX_BYTES and MAX_BUFS computations so we
shouldn't see "please increase to at least zero" messages.
* Note that VBLK and VCHR vnodes can have nonzero values in di_db[0]
even though their v_size == 0. Don't panic when we see this.
Fixes PR #26680.
* Change lfs_bfree to a signed quantity. The manner in which it is
processed before being passed to the cleaner means that sometimes it
may drop below zero, and the cleaner must be aware of this.
* Never report bfree < 0 (or higher than lfs_dsize) through
lfs_statfs(9). This prevents df(1) from ever telling us that our full
filesystems have 16TB free.
* Account space allocated through lfs_balloc(9) that does not have
associated buffer headers, so that the pagedaemon doesn't run us out
of segments.
* Return ENOSPC from lfs_balloc(9) when bfree drops to zero.
* Address a deadlock in lfs_bmapv/lfs_markv when the filesystem is being
unmounted. Because vfs_busy() is a shared lock, and
lfs_bmapv/lfs_markv mark the filesystem vfs_busy(), the cleaner can be
holding the lock that umount() is blocking on, then try to vfs_busy()
again in getnewvnode().

cleaner:

* Adapt lfs_cleanerd to use the fcntl call to get the Ifile filehandle,
so it need not be in the namespace.
* Make lfs_cleanerd be more careful when there are very few available
segments.
* Make lfs_cleanerd less verbose when the filesystem is unmounted.

newfs_lfs, fsck_lfs, and regression:

* Extend the lfs library from fsck_lfs(8) so that it can be used with a
not-yet-existent LFS. Make newfs_lfs(8) use this library, so it can
create LFSs whose Ifile is larger than one segment. Addresses PR #11110.
* Make newfs_lfs(8) use strsuftoi64() for its arguments, a la newfs(8).
* Make fsck_lfs(8) respect the "file system is clean" flag.
* Don't let fsck_lfs(8) think it has dirty blocks when invoked with the
-n flag.
* Remove the Ifile from the filesystem namespace. The cleaner now uses
a fcntl call on the root inode to find the Ifile filehandle. (As a
side-effect, addresses PR #29144.)
 1.132.6.3  26-Mar-2005  yamt sync with head.
 1.132.6.2  19-Mar-2005  yamt sync with head. xen and whitespace. xen part is not finished.
 1.132.6.1  12-Feb-2005  yamt sync with head.
 1.132.4.1  29-Apr-2005  kent sync with -current
 1.137.2.25  10-Aug-2006  tron Apply patch (requested by fair in perseant #1457):
Bring LFS up to current, including a patch (1.95 lfs_alloc.c) that
should prevent the inode free list errors seen on the STABLE branch
subsequent to pullup ticket #1327.
 1.137.2.24  20-May-2006  riz Pull up following revision(s) (requested by perseant in ticket #1327):
sys/ufs/lfs/lfs_vnops.c: revision 1.177
Don't be quite so eager to error out from lfs_putpages() when pages are
busy; if we've sensed a possible 3-way deadlock and are not the pagedaemon,
relock and try again.
 1.137.2.23  20-May-2006  riz Pull up following revision(s) (requested by perseant in ticket #1327):
sys/ufs/lfs/lfs_alloc.c: revision 1.93
sys/ufs/lfs/lfs.h: revision 1.106
sys/ufs/lfs/lfs_vfsops.c: revision 1.209
sys/ufs/lfs/lfs_vnops.c: revision 1.175
sys/ufs/lfs/lfs_segment.c: revision 1.178
Fixes to address the "vinvalbuf: dirty blocks" panic that can occur when
many inodes are cleaned at once. Make sure that we write all the pages
on vnodes that are being flushed, even if we don't think there's room;
drain v_numoutput before lfs_vflush() completes.
Also, don't allow a vnode that is in the process of being cleaned to be
chosen by getnewvnode(); this avoids a segment accounting panic in the case
that a large number of inodes are fed to lfs_markv() all at once.
 1.137.2.22  20-May-2006  riz Pull up following revision(s) (requested by perseant in ticket #1327):
sys/ufs/lfs/lfs_alloc.c: revision 1.92
sys/ufs/lfs/lfs.h: revision 1.105
sys/ufs/lfs/lfs_vfsops.c: revision 1.207
sys/ufs/lfs/lfs_subr.c: revision 1.59
sys/ufs/lfs/lfs_vnops.c: revision 1.173
sys/ufs/lfs/lfs_bio.c: revision 1.92
Introduce another per-filesystem parameter, lfs_resvseg, to separate the
notion of "how many segments are reserved for the cleaner" from that of
"how many segments are not counted in lfs_bfree". The default value
used for existing filesystems is the same as the previous implicit value
of (lfs_minfreeseg / 2 + 1), modulo some sanity checking.
Count pending dirops on a per-filesystem basis, since once we start
writing them we can't stop until we're done. This seems to help stave off
the "no clean segments" panic in the case of filling the filesystem with
directories and small files (e.g. simultaneously unpacking more copies of
pkgsrc than will fit).
 1.137.2.21  20-May-2006  riz Pull up following revision(s) (requested by perseant in ticket #1327):
sys/ufs/lfs/lfs_vnops.c: revision 1.172
Fix a "locking against myself": lfs_flush_dirops() doesn't need to lock the
vnodes to write their blocks, since it holds the segment lock.
 1.137.2.20  20-May-2006  riz Pull up following revision(s) (requested by perseant in ticket #1327):
sys/ufs/lfs/lfs_vnops.c: revision 1.171
sys/ufs/lfs/lfs_extern.h: revision 1.81
sys/ufs/lfs/lfs_segment.c: revision 1.177
Don't ever partially write dirops, even if we need the cleaner to run.
This increases the chances of the "no clean segments" panic slightly,
but allows us to run the ckckp regression test successfully to completion.
 1.137.2.19  20-May-2006  riz Pull up following revision(s) (requested by perseant in ticket #1327):
sys/ufs/lfs/lfs.h: revision 1.104
sys/ufs/lfs/lfs_vfsops.c: revision 1.206
sys/ufs/lfs/lfs_vnops.c: revision 1.170
sys/ufs/lfs/lfs_extern.h: revision 1.80
sys/ufs/lfs/lfs_segment.c: revision 1.176
sys/ufs/lfs/lfs_inode.c: revision 1.103 via patch
sys/ufs/lfs/lfs_alloc.c: revision 1.90
Postpone the segment accounting changes coming from truncation until the
inode that makes those changes valid is either written to disk by
lfs_writeinode() or discarded by lfs_vfree().
A couple of locking fixes are also included as well.
 1.137.2.18  20-May-2006  riz Pull up following revision(s) (requested by perseant in ticket #1327):
sys/ufs/lfs/lfs_vnops.c: revision 1.169
Yet another MP locking issue.
 1.137.2.17  20-May-2006  riz Pull up following revision(s) (requested by perseant in ticket #1327):
sys/ufs/lfs/lfs.h: revision 1.103
sys/ufs/lfs/lfs_segment.c: revision 1.174
sys/ufs/lfs/lfs_vnops.c: revision 1.168
Introduce two fcntl calls that freeze the filesystem right at the point
where segment 0 is being considered for writing. This allows for automated
checkpoint vailidity scanning, and could be used (in conjunction with the
existing LFCNREWIND) for e.g. snapshot dumps as well.
Include a regression test that does such scanning.
When writing the Ifile, loop through the dirty block list three times to
make sure that the checkpoint is always consistent (the first and second
times the Ifile blocks can cross a segment boundary; not so the third time
unless the segments are very small). Discovered by using the aforementioned
regression test.
 1.137.2.16  20-May-2006  riz Pull up following revision(s) (requested by perseant in ticket #1327):
sys/ufs/lfs/lfs.h: revision 1.102
sys/ufs/lfs/lfs_segment.c: revision 1.173
sys/ufs/lfs/lfs_vnops.c: revision 1.167 via patch
sys/ufs/lfs/lfs_bio.c: revision 1.91
Make lfs_vref/lfs_vunref not need to know about VXLOCK and VFREEING
explicitly (especially since we didn't know about VFREEING at all before),
but notice the EBUSY return from vget() instead.
Fix some more MP locking protocol issues, most of which were pointed out by
Christian Ehrhardt this morning on tech-kern.
 1.137.2.15  20-May-2006  riz Pull up following revision(s) (requested by perseant in ticket #1327):
sys/ufs/lfs/lfs_vnops.c: revision 1.166
Another MP locking fix.
 1.137.2.14  20-May-2006  riz Pull up following revision(s) (requested by perseant in ticket #1327):
sys/ufs/lfs/lfs_vnops.c: revision 1.165
Don't leak vnode references if we fail to lock a vnode in lfs_flush_pchain().
Also fix another (probably only academic) simple_lock protocol error.
 1.137.2.13  20-May-2006  riz Pull up following revision(s) (requested by perseant in ticket #1327):
sys/ufs/lfs/lfs_vfsops.c: revision 1.200
sys/ufs/lfs/lfs_vnops.c: revision 1.164
sys/ufs/lfs/lfs_inode.c: revision 1.101
sys/ufs/lfs/lfs_extern.h: revision 1.78
sys/ufs/lfs/lfs.h: revision 1.100
Implement a somewhat finer-grained mechanism for paging LFS-backed pages.
The writer daemon, if it does not need to flush the whole filesystem,
now only writes the vnodes for which the pagedaemon has requested pageouts
(although it does not pay attention to the page ranges the pagedaemon
supplies).
 1.137.2.12  20-May-2006  riz Pull up following revision(s) (requested by perseant in ticket #1327):
sys/ufs/lfs/lfs_balloc.c: revision 1.60
sys/ufs/lfs/lfs_syscalls.c: revision 1.111
sys/ufs/lfs/lfs_segment.c: revision 1.172
sys/ufs/lfs/lfs_vnops.c: revision 1.163
Several minor bug fixes:
* Correct (weak) segment lock assertions in lfs_fragextend and lfs_putpages.
* Keep IN_MODIFIED set if we run out of avail in lfs_putpages.
* Don't try to (re)write buffers on a VBLK vnode; fixes a panic I found
while running with an LFS root.
* Raise priority of LFCNSEGWAIT to PVFS; PUSER is way too low for
something the pagedaemon is relying on.
 1.137.2.11  20-May-2006  riz Pull up following revision(s) (requested by perseant in ticket #1327):
sys/ufs/lfs/lfs_vnops.c: revision 1.162
Make sure we unlock to zero when avoiding 3-way deadlock; otherwise we
simply have a different form of deadlock.
 1.137.2.10  20-May-2006  riz Pull up following revision(s) (requested by perseant in ticket #1327):
sys/ufs/lfs/lfs_vfsops.c: revision 1.198
sys/ufs/lfs/lfs_vnops.c: revision 1.161
Handle the "filesystem is clean" flag correctly when upgrading from
read-only to read-write mount. This makes "root on lfs" work for me,
although it looks like a different traceback from PR#32667.
 1.137.2.9  20-May-2006  riz Pull up following revision(s) (requested by perseant in ticket #1327):
sys/ufs/lfs/lfs_vnops.c: revision 1.159
Don't let the pagedaemon wait for pages, since that is just asking for
a deadlock.
 1.137.2.8  20-May-2006  riz Pull up following revision(s) (requested by perseant in ticket #1327):
sys/ufs/lfs/lfs_vnops.c: revision 1.158
sys/ufs/lfs/lfs_subr.c: revision 1.57
sys/ufs/lfs/lfs_segment.c: revision 1.171
sys/ufs/lfs/lfs.h: revision 1.97
sys/ufs/lfs/lfs_vfsops.c: revision 1.195
sys/ufs/lfs/lfs_extern.h: revision 1.76
Improvements to LFS's paging mechanism, to wit:
* Acknowledge that sometimes there are more dirty pages to be written to
disk than clean segments. When we reach the danger line,
lfs_gop_write() now returns EAGAIN. The caller of VOP_PUTPAGES(), if
it holds the segment lock, drops it and waits for the cleaner to make
room before continuing.
* Note and avoid a three-way deadlock in lfs_putpages (a writer holding
a page busy blocks on the cleaner while the cleaner blocks on the
segment lock while lfs_putpages blocks on the page).
 1.137.2.7  20-May-2006  riz Pull up following revision(s) (requested by perseant in ticket #1327):
sys/ufs/lfs/lfs_vnops.c: revision 1.153
sys/ufs/lfs/lfs_debug.c: revision 1.32
sys/ufs/lfs/lfs_alloc.c: revision 1.84
sys/ufs/lfs/lfs_vfsops.c: revision 1.185
sys/ufs/lfs/lfs_segment.c: revision 1.165
64 bit inode changes.
 1.137.2.6  20-May-2006  riz Pull up following revision(s) (requested by perseant in ticket #1327):
sys/ufs/lfs/lfs_vnops.c: revision 1.152
sys/ufs/lfs/lfs_debug.c: revision 1.31
sys/ufs/lfs/lfs_subr.c: revision 1.53
sys/ufs/lfs/lfs_extern.h: revision 1.68
sys/ufs/lfs/lfs_inode.c: revision 1.96
sys/ufs/lfs/lfs_bio.c: revision 1.86
sys/ufs/lfs/lfs_alloc.c: revision 1.83
sys/ufs/lfs/lfs_vfsops.c: revision 1.181
sys/ufs/lfs/lfs.h: revision 1.88
sys/ufs/lfs/lfs_segment.c: revision 1.164
- sprinkle const
- avoid shadow variables.
 1.137.2.5  20-May-2006  riz Pull up following revision(s) (requested by perseant in ticket #1327):
sys/ufs/lfs/lfs_vnops.c: revision 1.151
VOP_LOCK drops the interlock; pick it up again to avoid an "already unlocked"
panic in lfs_putpages.
 1.137.2.4  07-May-2005  tron Apply patch (requested by perseant in ticket #242):
* fsck_lfs buffer cache fixes, including PR #29151
* Change fsck_lfs phase 0 message to reflect reality
* fsck_lfs: check phase 5 (cleanerinfo accounting) even on
roll-forward
* Keep better track of the free list during roll-forward, avoiding
a core dump
* Improve hash table use for fsck_lfs buffer and vnode cache
* Document fsck_lfs flag -f, and implement -q
* Add resize_lfs, including kernel support
* Add LFS to mountd's list of exportable filesystem types
* Make the LFS lkm work again [christos@]
* Add MP locking to the LFS kernel subsystem
* Fix pager_map deadlock in lfs_putpages()
* Avoid incomplete file extension that looks like "partial
truncation" to fsck
* Use lfs_malloc for cleaner malloc, since the cleaner often runs
in low-memory conditions.
* Use splay trees, not hash table, to track page allocation for
write.
* Fix mkdir panic on full fs
* Fix page accounting leak by counting differently.
* Use rightly named structure for lfs_getattr [skrll@]
* Cosmetic changes for readability.
 1.137.2.3  30-Mar-2005  tron Pull up revision 1.140 (requested by perseant in ticket #74):
Don't sleep while holding the vnode interlock. Should take care of the
first panic case in PR #26043.
 1.137.2.2  30-Mar-2005  tron Pull up revision 1.139 (requested by perseant in ticket #74):
avoid the need for recursive locking lfs_flush_dirops() by unlocking
the vnode around the call to this in the caller.
 1.137.2.1  30-Mar-2005  tron Pull up revision 1.138 (requested by perseant in ticket #74):
Make LFS dirops get their vnode first, before incrementing the dirop
count, to prevent a deadlock trying to call VOP_PUTPAGES() on a VDIROP
vnode. This can happen when a stacked filesystem is mounted on top of an
LFS: an LFS dirop needs to get a vnode, which is available from the upper
layer. The corresponding lower layer vnode, however, is VDIROP, so the
upper layer can't be cleaned out since its VOP_PUTPAGES() is passed
through to the lower layer, which waits for dirops to drain before it can
proceed. Deadlock.
Tweak ufs_makeinode() and ufs_mkdir() to pass the a_vpp argument through
to VOP_VALLOC().
Partially addresses PR # 26043, though it probably does not completely fix
the problem described there.
 1.152.2.8  04-Feb-2008  yamt sync with head.
 1.152.2.7  21-Jan-2008  yamt sync with head
 1.152.2.6  07-Dec-2007  yamt sync with head
 1.152.2.5  27-Oct-2007  yamt sync with head.
 1.152.2.4  03-Sep-2007  yamt sync with head.
 1.152.2.3  26-Feb-2007  yamt sync with head.
 1.152.2.2  30-Dec-2006  yamt sync with head.
 1.152.2.1  21-Jun-2006  yamt sync with head.
 1.155.2.2  29-Oct-2005  yamt use lfs_* directly rather than via ufs_ops.
suggested by Chuck Silvers.
 1.155.2.1  20-Oct-2005  yamt adapt ufs.
 1.157.12.3  24-May-2006  tron Merge 2006-05-24 NetBSD-current into the "peter-altq" branch.
 1.157.12.2  31-Mar-2006  tron Merge 2006-03-31 NetBSD-current into the "peter-altq" branch.
 1.157.12.1  28-Mar-2006  tron Merge 2006-03-28 NetBSD-current into the "peter-altq" branch.
 1.157.10.5  11-May-2006  elad sync with head
 1.157.10.4  06-May-2006  christos - Move kauth_cred_t declaration to <sys/types.h>
- Cleanup struct ucred; forward declarations that are unused.
- Don't include <sys/kauth.h> in any header, but include it in the c files
that need it.

Approved by core.
 1.157.10.3  19-Apr-2006  elad sync with head.
 1.157.10.2  10-Mar-2006  elad generic_authorize() -> kauth_authorize_generic().
 1.157.10.1  08-Mar-2006  elad Adapt to kernel authorization KPI.
 1.157.8.6  03-Sep-2006  yamt sync with head.
 1.157.8.5  11-Aug-2006  yamt sync with head
 1.157.8.4  26-Jun-2006  yamt sync with head.
 1.157.8.3  24-May-2006  yamt sync with head.
 1.157.8.2  11-Apr-2006  yamt sync with head
 1.157.8.1  01-Apr-2006  yamt sync with head.
 1.157.6.2  01-Jun-2006  kardel Sync with head.
 1.157.6.1  22-Apr-2006  simonb Sync with head.
 1.157.4.1  09-Sep-2006  rpaulo sync with head
 1.178.4.1  13-Jul-2006  gdamore Merge from HEAD.
 1.188.2.4  01-Feb-2007  ad Sync with head.
 1.188.2.3  30-Jan-2007  ad Remove support for SA. Ok core@.
 1.188.2.2  12-Jan-2007  ad Sync with head.
 1.188.2.1  18-Nov-2006  ad Sync with head.
 1.189.2.2  10-Dec-2006  yamt sync with head.
 1.189.2.1  22-Oct-2006  yamt sync with head
 1.193.2.3  25-Nov-2007  xtraeme Pull up following revision(s) (requested by christos in ticket #994):
sys/ufs/lfs/lfs_vnops.c: revision 1.208 (patch)
Move the "vp = NULL" assignment after the code that requires vp != NULL.
Reported by Chris Ross on current-users.
 1.193.2.2  05-Jun-2007  bouyer Pull up following revision(s) (requested by perseant in ticket #703):
sys/miscfs/genfs/genfs.h 1.21
sys/miscfs/genfs/genfs_vnops.c 1.151
sys/ufs/lfs/lfs.h 1.119, 1.120
sys/ufs/lfs/lfs_bio.c 1.99-101
sys/ufs/lfs/lfs_extern.h 1.89
sys/ufs/lfs/lfs_inode.c 1.108, 1.109
sys/ufs/lfs/lfs_segment.c 1.197, 1.199, 1.200
sys/ufs/lfs/lfs_subr.c 1.69, 1.70
sys/ufs/lfs/lfs_syscalls.c 1.119
sys/ufs/lfs/lfs_vfsops.c 1.234, 1.235
sys/ufs/lfs/lfs_vnops.c 1.195, 1.196, 1.200, 1.202-206

Reduce busy waiting in lfs_putpages(), and other LFS improvements.
 1.193.2.1  17-Feb-2007  tron branches: 1.193.2.1.2;
Apply patch (requested by chs in ticket #422):
- Fix various deadlock problems with nullfs and unionfs.
- Speed up path lookups by upto 25%.
 1.193.2.1.2.2  06-Jan-2008  wrstuden Catch up to netbsd-4.0 release.
 1.193.2.1.2.1  03-Sep-2007  wrstuden Sync w/ NetBSD-4-RC_1
 1.198.2.4  07-May-2007  yamt sync with head.
 1.198.2.3  15-Apr-2007  yamt sync with head.
 1.198.2.2  12-Mar-2007  rmind Sync with HEAD.
 1.198.2.1  27-Feb-2007  yamt - sync with head.
- move sched_changepri back to kern_synch.c as it doesn't know PPQ anymore.
 1.201.4.1  11-Jul-2007  mjf Sync with head.
 1.201.2.12  16-Sep-2007  ad - Checkpoint work in progress on the vnode lifecycle and reference counting
stuff. This makes it work properly without kernel_lock and fixes a few
quite old bugs. See vfs_subr.c 1.283.2.17 for details.

- Fix some problems with softdep. Unfortunately our softdep code appears
to have some longstanding bugs that cause it fail under stress test.
 1.201.2.11  20-Aug-2007  ad Sync with HEAD.
 1.201.2.10  19-Aug-2007  ad - Back out the biodone() changes.
- Eliminate B_ERROR (from HEAD).
 1.201.2.9  15-Jul-2007  ad Sync with head.
 1.201.2.8  23-Jun-2007  ad - Lock v_cleanblkhd, v_dirtyblkhd, v_numoutput with the vnode's interlock.
Get rid of global_v_numoutput_lock. Partially incomplete as the buffer
cache locking doesn't work very well and needs an overhaul.
- Some changes to try and make softdep MP safe. Untested.
 1.201.2.7  17-Jun-2007  ad - Increase the number of thread priorities from 128 to 256. How the space
is set up is to be revisited.
- Implement soft interrupts as kernel threads. A generic implementation
is provided, with hooks for fast-path MD code that can run the interrupt
threads over the top of other threads executing in the kernel.
- Split vnode::v_flag into three fields, depending on how the flag is
locked (by the interlock, by the vnode lock, by the file system).
- Miscellaneous locking fixes and improvements.
 1.201.2.6  08-Jun-2007  ad Sync with head.
 1.201.2.5  13-May-2007  ad - Pass the error number and residual count to biodone(), and let it handle
setting error indicators. Prepare to eliminate B_ERROR.
- Add a flag argument to brelse() to be set into the buf's flags, instead
of doing it directly. Typically used to set B_INVAL.
- Add a "struct cpu_info *" argument to kthread_create(), to be used to
create bound threads. Change "bool mpsafe" to "int flags".
- Allow exit of LWPs in the IDL state when (l != curlwp).
- More locking fixes & conversion to the new API.
 1.201.2.4  10-Apr-2007  ad Sync with head.
 1.201.2.3  09-Apr-2007  ad - Add two new arguments to kthread_create1: pri_t pri, bool mpsafe.
- Fork kthreads off proc0 as new LWPs, not new processes.
 1.201.2.2  21-Mar-2007  ad - Replace more simple_locks, and fix up in a few places.
- Use condition variables.
- LOCK_ASSERT -> KASSERT.
 1.201.2.1  13-Mar-2007  ad Pull in the initial set of changes for the vmlocking branch.
 1.208.2.1  15-Aug-2007  skrll Sync with HEAD.
 1.210.10.2  29-Jul-2007  ad It's not a good idea for device drivers to modify b_flags, as they don't
need to understand the locking around that field. Instead of setting
B_ERROR, set b_error instead. b_error is 'owned' by whoever completes
the I/O request.
 1.210.10.1  29-Jul-2007  ad file lfs_vnops.c was added on branch matt-mips64 on 2007-07-29 13:31:16 +0000
 1.210.8.1  14-Oct-2007  yamt sync with head.
 1.210.6.3  23-Mar-2008  matt sync with HEAD
 1.210.6.2  09-Jan-2008  matt sync with HEAD
 1.210.6.1  06-Nov-2007  matt sync with HEAD
 1.210.4.2  27-Nov-2007  joerg Sync with HEAD. amd64 Xen support needs testing.
 1.210.4.1  26-Oct-2007  joerg Sync with HEAD.

Follow the merge of pmap.c on i386 and amd64 and move
pmap_init_tmp_pgtbl into arch/x86/x86/pmap.c. Modify the ACPI wakeup
code to restore CR4 before jumping back into kernel space as the large
page option might cover that.
 1.212.4.2  18-Feb-2008  mjf Sync with HEAD.
 1.212.4.1  08-Dec-2007  mjf Sync with HEAD.
 1.213.6.1  02-Jan-2008  bouyer Sync with HEAD
 1.213.2.4  19-Dec-2007  ad Use a global lfs_lock.
 1.213.2.3  19-Dec-2007  ad Fix some more problems w/lfs on this branch.
 1.213.2.2  19-Dec-2007  ad Get lfs mostly working.
 1.213.2.1  04-Dec-2007  ad Pull the vmlocking changes into a new branch.
 1.215.10.5  11-Aug-2010  yamt sync with head.
 1.215.10.4  11-Mar-2010  yamt sync with head
 1.215.10.3  16-May-2009  yamt sync with head
 1.215.10.2  04-May-2009  yamt sync with head.
 1.215.10.1  16-May-2008  yamt sync with head.
 1.215.8.2  17-Jun-2008  yamt sync with head.
 1.215.8.1  18-May-2008  yamt sync with head.
 1.215.6.4  17-Jan-2009  mjf Sync with HEAD.
 1.215.6.3  29-Jun-2008  mjf Sync with HEAD.
 1.215.6.2  05-Jun-2008  mjf Sync with HEAD.

Also fix build.
 1.215.6.1  02-Jun-2008  mjf Sync with HEAD.
 1.216.2.2  18-Sep-2008  wrstuden Sync with wrstuden-revivesa-base-2.
 1.216.2.1  23-Jun-2008  wrstuden Sync w/ -current. 34 merge conflicts to follow.
 1.217.2.1  27-Jun-2008  simonb Sync with head.
 1.218.12.1  29-Feb-2012  matt Deal with UVM_PAGE_OWN changes.
 1.218.6.1  19-May-2012  riz Apply patch (requested by buhrow in ticket #1759):


sys/ufs/lfs/lfs_vnops.c patch
sys/ufs/ufs/inode.h patch
sys/ufs/ufs/ufs_extern.h patch
sys/ufs/ufs/ufs_lookup.c patch
sys/ufs/ufs/ufs_vnops.c patch
sys/ufs/ufs/ufs_wapbl.c patch

Port dholland's ufs_rename locking changes to netbsd-5.
[buhrow, ticket #1759]

Hello. More testing has revealed a minor misunderstanding between the
vnode API in -current and 5.x. The below patch, against NetBSD-5.1
sources, rolls all the accumulated patches into one patch set. With this
patch, I believe you can now run with WAPBL, softdep or traditional ufs
semantics with heavy file loads and avoid panics due to resource exhaustion
and/or tstile deadlocks. Testing has been done on I386, both uniprocessor
and multiprocessor, and on Sparc machines in uniprocessor mode, though I
think multiprocessor Sparc would be fine as well. Since these changes are
machine independent, I don't anticipate any issues on any platform. It is
my hope that modulo any final issues that come up in the final round of
testing I'm currently performing, these patches will be ready to be pulled
up into the NetBSD-5 branch.
Finally, I'd like to thank mouse@ and hannken@ for their help and
patience in helping me track down and test the final versions of these
patches. With their assistance, I'm confident these patches make NetBSD-5
a much more stable and robust operating environment in a variety of
setings.
 1.218.4.2  03-Mar-2009  skrll Sync with HEAD.
 1.218.4.1  19-Jan-2009  skrll Sync with HEAD.
 1.219.2.1  13-May-2009  jym Sync with HEAD.

Commit is split, to avoid a "too many arguments" protocol error.
 1.226.4.6  21-May-2011  rmind Fix the build.
 1.226.4.5  19-May-2011  rmind Implement sharing of vnode_t::v_interlock amongst vnodes:
- Lock is shared amongst UVM objects using uvm_obj_setlock() or getnewvnode().
- Adjust vnode cache to handle unsharing, add VI_LOCKSHARE flag for that.
- Use sharing in tmpfs and layerfs for underlying object.
- Simplify locking in ubc_fault().
- Sprinkle some asserts.

Discussed with ad@.
 1.226.4.4  05-Mar-2011  rmind sync with head
 1.226.4.3  03-Jul-2010  rmind sync with head
 1.226.4.2  30-May-2010  rmind sync with head
 1.226.4.1  16-Mar-2010  rmind Change struct uvm_object::vmobjlock to be dynamically allocated with
mutex_obj_alloc(). It allows us to share the locks among UVM objects.
 1.226.2.2  17-Aug-2010  uebayasi Sync with HEAD.
 1.226.2.1  30-Apr-2010  uebayasi Sync with HEAD.
 1.234.6.1  23-Jun-2011  cherry Catchup with rmind-uvmplock merge.
 1.238.6.3  02-Jun-2012  mrg sync to latest -current.
 1.238.6.2  05-Apr-2012  mrg sync to latest -current.
 1.238.6.1  18-Feb-2012  mrg merge to -current.
 1.238.2.6  22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.238.2.5  23-Jan-2013  yamt sync with head
 1.238.2.4  23-May-2012  yamt sync with head.
 1.238.2.3  17-Apr-2012  yamt sync with head
 1.238.2.2  06-Nov-2011  yamt remove pg->listq and uobj->memq
 1.238.2.1  02-Nov-2011  yamt page cache related changes

- maintain object pages in radix tree rather than rb tree.
- reduce unnecessary page scan in putpages. esp. when an object has a ton of
pages cached but only a few of them are dirty.
- reduce the number of pmap operations by tracking page dirtiness more
precisely in uvm layer.
- fix nfs commit range tracking.
- fix nfs write clustering. XXX hack
 1.239.2.2  27-Aug-2016  bouyer Pull up following revision(s) (requested by dholland in ticket #1389):
sys/ufs/lfs/lfs_vnops.c: revision 1.304
Fix a deadlock
ok dholland@
 1.239.2.1  17-Mar-2012  bouyer Pull up following revision(s) (requested by perseant in ticket #116):
sys/ufs/lfs/lfs_alloc.c: revision 1.112
tests/fs/vfs/t_rmdirrace.c: revision 1.9
tests/fs/vfs/t_renamerace.c: revision 1.25
sys/ufs/lfs/lfs_vnops.c: revision 1.240
sys/ufs/lfs/lfs_segment.c: revision 1.224
sys/ufs/lfs/lfs_bio.c: revision 1.122
sys/ufs/lfs/lfs_vfsops.c: revision 1.294
sbin/newfs_lfs/make_lfs.c: revision 1.19
sys/ufs/lfs/lfs.h: revision 1.136
Pass t_renamerace and t_rmdirrace tests.
Adapt dholland@'s fix to ufs_rename to fix PR kern/43582. Address several
other MP locking issues discovered during the course of investigating the
same problem.
Removed extraneous vn_lock() calls on the Ifile, since the Ifile writes
are controlled by the segment lock.
Fix PR kern/45982 by deemphasizing the estimate of how much metadata
will fill the empty space on disk when the disk is nearly empty
(t_renamerace crates a lot of inode blocks on a tiny empty disk).
 1.242.2.4  03-Dec-2017  jdolecek update from HEAD
 1.242.2.3  20-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.242.2.2  23-Jun-2013  tls resync from head
 1.242.2.1  25-Feb-2013  tls resync with head
 1.248.4.1  23-Jul-2013  riastradh sync with HEAD
 1.248.2.2  18-May-2014  rmind sync with head
 1.248.2.1  28-Aug-2013  rmind sync with head
 1.262.2.1  10-Aug-2014  tls Rebase.
 1.269.4.6  28-Aug-2017  skrll Sync with HEAD
 1.269.4.5  05-Oct-2016  skrll Sync with HEAD
 1.269.4.4  09-Jul-2016  skrll Sync with HEAD
 1.269.4.3  22-Sep-2015  skrll Sync with HEAD
 1.269.4.2  06-Jun-2015  skrll Sync with HEAD
 1.269.4.1  06-Apr-2015  skrll Sync with HEAD
 1.269.2.2  14-Jul-2016  martin Pull up following revision(s) (requested by dholland in ticket #1205):
sys/ufs/lfs/lfs_vnops.c: revision 1.304
Fix a deadlock
ok dholland@
 1.269.2.1  06-Aug-2015  snj Apply patch (requested by dholland in ticket #935):
Comment out some KASSERTs.
 1.304.4.1  21-Apr-2017  bouyer Sync with HEAD
 1.304.2.2  26-Apr-2017  pgoyette Sync with HEAD
 1.304.2.1  20-Mar-2017  pgoyette Sync with HEAD
 1.315.2.1  30-Oct-2017  snj Pull up following revision(s) (requested by maya in ticket #330):
sbin/fsck_lfs/inode.c: 1.69
sbin/fsck_lfs/lfs.c: 1.73
sbin/fsck_lfs/pass6.c: 1.50
sbin/fsck_lfs/segwrite.c: 1.46
sys/ufs/lfs/lfs.h: 1.202-1.203
sys/ufs/lfs/lfs_accessors.h: 1.48
sys/ufs/lfs/lfs_alloc.c: 1.136-1.137
sys/ufs/lfs/lfs_balloc.c: 1.94
sys/ufs/lfs/lfs_bio.c: 1.141
sys/ufs/lfs/lfs_extern.h: 1.113
sys/ufs/lfs/lfs_inode.c: 1.156-1.157
sys/ufs/lfs/lfs_inode.h: 1.20, 1.21, 1.23
sys/ufs/lfs/lfs_itimes.c: 1.20
sys/ufs/lfs/lfs_pages.c: 1.13-1.15
sys/ufs/lfs/lfs_rename.c: 1.22
sys/ufs/lfs/lfs_segment.c: 1.270-1.275
sys/ufs/lfs/lfs_subr.c: 1.94-1.97
sys/ufs/lfs/lfs_syscalls.c: 1.175
sys/ufs/lfs/lfs_vfsops.c: 1.360
sys/ufs/lfs/lfs_vnops.c: 1.316-1.321
sys/ufs/lfs/ulfs_inode.c: 1.20
sys/ufs/lfs/ulfs_inode.h: 1.24
sys/ufs/lfs/ulfs_lookup.c: 1.41
sys/ufs/lfs/ulfs_quota2.c: 1.31
sys/ufs/lfs/ulfs_readwrite.c: 1.24
sys/ufs/lfs/ulfs_vnops.c: 1.49-1.50
Update inode member i_flag --> i_state to keep up with kernel changes
Move definition of IN_ALLMOD near the flag it's a mask for.
Now we can see that it doesn't match all the flags, but changing that will
require more careful thought.
Correct confusion between i_flag and i_flags
These will have to be renamed.
Spotted by Riastradh, thanks!
Add an XXX about the missing flags so it's not buried in a commit
message.
now the XXX count for LFS is 260
Rename i_flag to i_state.
The similarity to i_flags has previously caused errors.
Use continue to denote the no-op loop to match netbsd style
newline for extra clarity.
It isn't safe to drain dirops with seglock held, it'll deadlock if there
are any dirops. drain before grabbing seglock.
lfs_dirops == 0 is always true (as we already drained dirops), so omit
that part of the comparison.
Fixes a lot of LFS deadlocks. PR kern/52301
Many thanks to dholland for help analyzing coredumps
Ifdef out KDASSERT which fires on my machine.
Deduplicate sanity check that seglock is held on segunlock
Revert r1.272 fix to PR kern/52301, the performance hit is making things
unusable.
change lfs_nextsegsleep and lfs_allclean_wakeup to use condvar
XXX had to use lfs_lock in lfs_segwait, removed kernel_lock, is this
appropriate?
fix buffer overflow/KASSERT when cookies are supplied
lfs no longer uses the ffs-style struct direct, use the correct minimum
size
from dholland
XXX more wrong
Consistently use {,UN}MARK_VNODE macros rather than function calls.
Not much point doing anything after a panic call
Ask some question about the code in a XXX comment
XXX question our double-flushing of dirops
Fix typo in comment
 1.321.4.4  21-Apr-2020  martin Sync with HEAD
 1.321.4.3  13-Apr-2020  martin Mostly merge changes from HEAD upto 20200411
 1.321.4.2  08-Apr-2020  martin Merge changes from current as of 20200406
 1.321.4.1  10-Jun-2019  christos Sync with HEAD
 1.321.2.2  18-Jan-2019  pgoyette Synch with HEAD
 1.321.2.1  06-Sep-2018  pgoyette Sync with HEAD

Resolve a couple of conflicts (result of the uimin/uimax changes)
 1.324.2.1  17-Aug-2020  martin Pull up following revision(s) (requested by riastradh in ticket #1050):

sys/ufs/lfs/lfs_subr.c: revision 1.101
sys/ufs/lfs/lfs_subr.c: revision 1.102
sys/ufs/lfs/lfs_inode.c: revision 1.158
sys/ufs/lfs/lfs_inode.h: revision 1.25
sys/ufs/lfs/lfs_balloc.c: revision 1.95
sys/ufs/lfs/lfs_pages.c: revision 1.21
sys/ufs/lfs/lfs_vnops.c: revision 1.330
sys/ufs/lfs/lfs_alloc.c: revision 1.140 (patch)
sys/ufs/lfs/lfs_alloc.c: revision 1.141 (patch)
lib/libp2k/p2k.c: revision 1.72
sys/ufs/lfs/lfs.h: revision 1.205
sys/ufs/lfs/lfs.h: revision 1.206
sys/ufs/lfs/lfs_segment.c: revision 1.284
sys/ufs/lfs/lfs.h: revision 1.207
sys/ufs/lfs/lfs_segment.c: revision 1.285
sys/ufs/lfs/lfs_debug.c: revision 1.55
sys/ufs/lfs/lfs_rename.c: revision 1.23
usr.sbin/dumplfs/dumplfs.c: revision 1.65
sys/ufs/lfs/lfs_vfsops.c: revision 1.371
sys/arch/i386/stand/efiboot/bootx64/Makefile: revision 1.3
sys/ufs/lfs/lfs_vfsops.c: revision 1.372
sys/ufs/lfs/lfs_vfsops.c: revision 1.373
sbin/fsck_lfs/pass1.c: revision 1.46
sys/ufs/lfs/lfs_vnops.c: revision 1.326
sys/ufs/lfs/lfs_vnops.c: revision 1.327
sys/ufs/lfs/lfs_vfsops.c: revision 1.375 (patch)
sys/ufs/lfs/lfs_vnops.c: revision 1.328
sys/ufs/lfs/lfs_subr.c: revision 1.98
sys/ufs/lfs/lfs_extern.h: revision 1.116
sys/ufs/lfs/lfs_vnops.c: revision 1.329
sys/ufs/lfs/lfs_subr.c: revision 1.99
sys/ufs/lfs/lfs_extern.h: revision 1.117
sys/ufs/lfs/lfs_accessors.h: revision 1.49
sys/ufs/lfs/lfs_extern.h: revision 1.118
sys/rump/fs/lib/liblfs/Makefile: revision 1.15
sys/ufs/lfs/lfs_bio.c: revision 1.146 (patch)
sys/ufs/lfs/lfs_bio.c: revision 1.147
sys/ufs/lfs/lfs_subr.c: revision 1.100

Fix kassert in lfs by initializing vp first.

Use a marker node to iterate lfs_dchainhd / i_lfs_dchain.

I believe elements can be removed while the lock is dropped,
including the next node we're hanging on to.

Just use VOP_BWRITE for lfs_bwrite_log.
Hope this doesn't cause trouble with vfs_suspend.

Teach lfs to transition ro<->rw.

Prevent new dirops while we issue lfs_flush_dirops.

lfs_flush_dirops assumes (by KASSERT((ip->i_state & IN_ADIROP) == 0))
that vnodes on the dchain will not become involved in active dirops
even while holding no other locks (lfs_lock, v_interlock), so we must
set lfs_writer here. All other callers already set lfs_writer.

We set fs->lfs_writer++ without explicitly doing lfs_writer_enter
because
(a) we already waited for the dirops to drain, and
(b) we hold lfs_lock and cannot drop it before setting lfs_writer.

Assert lfs_writer where I think we can now prove it.

Serialize access to the splay tree with lfs_lock.

Change some cheap KDASSERT into KASSERT.

Take a reference and fix assertions in lfs_flush_dirops.
Fixes panic:
KASSERT((ip->i_state & IN_ADIROP) == 0) at lfs_vnops.c:1670
lfs_flush_dirops
lfs_check
lfs_setattr
VOP_SETATTR
change_mode
sys_fchmod
syscall

This assertion -- and the assertion that vp->v_uflag has VU_DIROP set
-- is valid only until we release lfs_lock, because we may race with
lfs_unmark_dirop which will remove the nodes and change the flags.

Further, vp itself is valid only as long as it is referenced, which it
is as long as it's on the dchain, but lfs_unmark_dirop drops the
dchain's reference.

Don't lfs_writer_enter while holding v_interlock.

There's no need to lfs_writer_enter at all here, as far as I can see.
lfs_flush_fs will do it for us.

Break deadlock in PR kern/52301.

The lock order is lfs_writer -> lfs_seglock. The problem in 52301 is
that lfs_segwrite violates this lock order by sometimes doing
lfs_seglock -> lfs_writer, either (a) when doing a checkpoint or (b),
opportunistically, when there are no dirops pending. Both cases can
deadlock, because dirops sometimes take the seglock (lfs_truncate,
lfs_valloc, lfs_vfree):
(a) There may be dirops pending, and they may be waiting for the
seglock, so we can't wait for them to complete while holding the
seglock.
(b) The test for fs->lfs_dirops == 0 happens unlocked, and the state
may change by the time lfs_writer_enter acquires lfs_lock.

To resolve this in each case:
(a) Do lfs_writer_enter before lfs_seglock, since we will need it
unconditionally anyway. The worst performance impact of this should
be that some dirops get delayed a little bit.
(b) Create a new lfs_writer_tryenter to use at this point so that the
test for fs->lfs_dirops == 0 and the acquisition of lfs_writer happen
atomically under lfs_lock.

Initialize/destroy lfs_allclean_wakeup in modcmd, not lfs_mountfs.

Fixes reloading lfs.kmod.

In lfs_update, hold lfs_writer around lfs_vflush.

Otherwise, we might do
lfs_vflush
-> lfs_seglock
-> lfs_segwait(SEGM_CKP)
-> lfs_writer_enter
which is the reverse of the lfs_writer -> lfs_seglock ordering.

Call lfs_orphan in lfs_rename while we're still in the dirop.
lfs_writer_enter can't fail; keep it simple and don't pretend it can.

Assert that mtsleep can't fail either -- it doesn't catch signals and
there's no timeout.

Teach LFS_ORPHAN_NEXTFREE about lfs64.

Dust off the orphan detection code and try to make it work.

Fix !DIAGNOSTIC compile

Fix userland references to LFS_ORPHAN_NEXTFREE.

Forgot to grep for these or do a full distribution build, oops!

Fix missing <sys/evcnt.h> by removing the evcnts instead.

Just wanted to confirm that a race might happen, and indeed it did.
These serve little diagnostic value otherwise.

OR into bp->b_cflags; don't overwrite.

CTASSERT lfs on-disk structure sizes.

Avoid misaligned access to lfs64 on-disk records in memory.
lfs64 directory entries are only 32-bit aligned in order to conserve
space in directory blocks, and we had a hack to stuff a 64-bit inode
in them. This replaces the hack by __aligned(4) __packed, and goes
further:

1. It's not clear that all the other lfs64 data structures are 64-bit
aligned on disk to begin with. We can go through these later and
upgrade them from
struct foo64 {
...
} __aligned(4) __packed;
union foo {
struct foo64 f64;
...
};
to
struct foo64 {
...
};
union foo {
struct foo64 f64 __aligned(8);
...
} __aligned(4) __packed;
if we really want to take advantage of 64-bit memory accesses.
However, the __aligned(4) __packed must remain on the union
because:
2. We access even the lfs32 data structures via a union that has
lfs64 members, and it turns out that compilers will assume access
through a union with 64-bit aligned members implies the whole
union has 64-bit alignment, even if we're only accessing a 32-bit
aligned member.

Fix clang build after packed lfs64 accessor change.

Suppress spurious address-of-packed error in rump lfs too.
 1.325.2.1  29-Feb-2020  ad Sync with head.
 1.331.4.1  20-Apr-2020  bouyer Sync with HEAD
 1.336.6.1  01-Aug-2021  thorpej Sync with HEAD.

RSS XML Feed