branches: 1.1.2;
move msdosfs code from sys/msdosfs to sys/fs/msdosfs

branches: 1.1.2;
move msdosfs code from sys/msdosfs to sys/fs/msdosfs

s/fist/first/

u_int8_t -> uint8_t

branches: 1.5.14;
Stylistic changes in comments/strings:
"FAT" and "fat" are different things, use the appropriate case.
"GEMDOS" is all caps.

branches: 1.4.92; 1.4.102;
When mounting an MSDOS fileysystem, do not require that bytes 252 and
253 of the superblock be zero. Searching the net failed to find any
justification for checking these bytes; all available references say
that they are part of the boot code and not BOOTSIG2 and BOOTSIG3.

Modify the MSDOS 7.1 bootsector definition to have 420 bytes of boot
code and no BOOTSIG[23], rather than 418 bytes of boot code, to follow
available references and apparent Windows practice. A test build
showed that these defines are not used other than in the check removed
by this commit.

Patch tested on netbsd-3, and enabled mounting of a 4 GB CF formatted
under Windows XP and then in a digital camera. The CF was previously
unmountable.

Concept approved on tech-kern by christos@ and martin@.

branches: 1.3.4; 1.3.8;
- protect userland exported files against multiple inclusion.
- make sure that kernel only files don't compile in userland using #error
- XXX: some kernel only files still get installed.
- XXX: some files used in userland, don't get installed.

branches: 1.2.16;
Consistently use "bsBPB" as the struct member for the BIOS parameter block.

branches: 1.1.2; 1.1.4;
move msdosfs code from sys/msdosfs to sys/fs/msdosfs

u_int{8,16,32}_t -> uint{8,16,32}_t, also u_int -> unsigned and
u_char -> unsigned char.

branches: 1.7.14;
Stylistic changes in comments/strings:
"FAT" and "fat" are different things, use the appropriate case.
"GEMDOS" is all caps.

branches: 1.6.80; 1.6.90;
Use the endian conversion functions from sys/endian.h.

PR/35151: Christian Groessler: file size reported incorrectly for files > 2GB
on a FAT32 file system on a 64bit host

branches: 1.4.20; 1.4.22;
- protect userland exported files against multiple inclusion.
- make sure that kernel only files don't compile in userland using #error
- XXX: some kernel only files still get installed.
- XXX: some files used in userland, don't get installed.

branches: 1.3.12;
cleanup casts in get/set u short/long so that we don't have const castaways.

add bpbReserved fields

branches: 1.1.2; 1.1.4;
move msdosfs code from sys/msdosfs to sys/fs/msdosfs

Sanitize the symbol namespace. NFC.

Convert msdosfs_rename() to use genfs_sane_rename().

Based on work by Taylor R Campbell.

Factor out the lookup results from struct denode.

No functional change.

Abolish all the silly indirection macros for initializing vnode ops tables.

These are things of the form #define foofs_op genfs_op, or #define
foofs_op genfs_eopnotsupp, or similar. They serve no purpose besides
obfuscation, and have gotten cutpasted all over everywhere.

branches: 1.25.32;
Be explicit about how we're placing part of the on-disk name into
the extension, so it doesn't appear like we are overrunning an array.
Appeases coverity, NFC.

ok riastradh

branches: 1.24.4; 1.24.8;
Change msdosfs from hashlist to vcache:
- Use (dir_cluster, dir_offset, dir_generation) as key, where
dir_generation is non-zero and unique for unlinked but open nodes.
- Change deget() to return a vnode as it is unsafe to return a
referenced but unlocked denode.

branches: 1.23.10;
fix fstat build.

more cross-compile friendly.

expose more stuff if MAKEFS is defined for the headers, and arrange for
the source file to be compilable from userland.

Stylistic changes in comments/strings:
"FAT" and "fat" are different things, use the appropriate case.
"GEMDOS" is all caps.

branches: 1.19.4; 1.19.14;
Msdosfs on-disk meta data is not sufficient to create or validate file handles.

Maintain a tree of file handles, create nodes from msdosfs_vptofh() and keep
them until either the file gets unlinked or the file system gets unmounted.

Fixes the msdosfs part of PR #43745 (fhopen of an unlinked file causes problems
on multiple file systems)

branches: 1.18.2;
In genfs where available.

The only functional change is mknod now returning EOPNOTSUPP instead
of EINVAL. I make this sacrifice willingly and with a clean conscience.

Use genfs instead of homegrown stuff where possible.

branches: 1.16.18; 1.16.38; 1.16.40;
Remove the "struct lwp *" argument from all VFS and VOP interfaces.
The general trend is to remove it from all kernel interfaces and
this is a start. In case the calling lwp is desired, curlwp should
be used.

quick consensus on tech-kern

branches: 1.15.8; 1.15.10; 1.15.16;
fix comment: struct fid is in fstypes.h now

branches: 1.14.4; 1.14.8; 1.14.10;
Apply patch from PR kern/34583 sent by Rhialto, quoting him:

"Add a 3rd entry in the cache, which keeps the end position
from just before extending a file.
This has the desired effect of keeping the write speed constant."

And yes, that helps a lot copying large files... always at full speed
now. This closes my PR kern/30868 "Poor performance copying large files
on msdosfs".

Also remove a 2 if-statements testing the same condition, combine them.

All that from Rhialto, thank you very much.

branches: 1.13.6; 1.13.8; 1.13.10;
integrate kauth.

branches: 1.12.4; 1.12.6; 1.12.8; 1.12.10; 1.12.12;
merge ktrace-lwp.

- protect userland exported files against multiple inclusion.
- make sure that kernel only files don't compile in userland using #error
- XXX: some kernel only files still get installed.
- XXX: some files used in userland, don't get installed.

PR/32003: Brian Buhrow: msdosfs doesn't properly zero out high cluster data
on non-FAT32 msdos filesystems.

merge yamt-vop branch. remove following VOPs.

VOP_BLKATOFF
VOP_VALLOC
VOP_BALLOC
VOP_REALLOCBLKS
VOP_VFREE
VOP_TRUNCATE
VOP_UPDATE

branches: 1.8.2;
Use nanotime() to update the time fields in filesystems. Convert the code
from macros to real functions. Original patch and review from chuq.
Note: ext2fs only keeps seconds in the on-disk inode, and msdosfs does not
have enough precision for all fields, so this is not very useful for those
two.

Remove __P()

The maximum file size on MS-DOS filesystems is 4 GB - 1 byte, so
don't bother trying to write files bigger than this. Just return
EFBIG to caller, rather than panic()ing later.

From OpenBSD.

This closes my PR kern/30864: "panic when copying files of >4GB on msdosfs"

update file timestamps for nfsd loaned-read and mmap.
PR/25279. discussed on tech-kern@.

branches: 1.4.2; 1.4.6; 1.4.14; 1.4.16;
add -t option for gmt time offset (normally MS-DOS filesystem has timestamp
in localtime, not GMT). PR kern/22717

branches: 1.3.2;
Back out the lwp/ktrace changes. They contained a lot of colateral damage,
and need to be examined and discussed more.

Pass lwp pointers throughtout the kernel, as required, so that the lwpid can
be inserted into ktrace records. The general change has been to replace
"struct proc *" with "struct lwp *" in various function prototypes, pass
the lwp through and use l_proc to get the process pointer when needed.

Bump the kernel rev up to 1.6V

branches: 1.1.2;
move msdosfs code from sys/msdosfs to sys/fs/msdosfs

Sanitize the symbol namespace. NFC.

We can't depend on dp->d_namlen existing for the parts that are used in
makefs(8).

Add support to msdosfs and makefs to generate correct Unicode (UCS-2) directory
entries from UTF8 encoded file names.

Apparently a recent round of vandalism by the C++ standards committee
includes disallowing the standard and historic type name "unsigned". Add
a gratuitous "int" to recent changes.

C++ is really a blight on the world.

u_int{8,16,32}_t -> uint{8,16,32}_t, also u_int -> unsigned and
u_char -> unsigned char.

branches: 1.7.6;
provide a function to access the name and extension as a single array as
opposed depend on array index overflow.

branches: 1.6.2;
expose more stuff if MAKEFS is defined for the headers, and arrange for
the source file to be compilable from userland.

branches: 1.5.110; 1.5.120;
- protect userland exported files against multiple inclusion.
- make sure that kernel only files don't compile in userland using #error
- XXX: some kernel only files still get installed.
- XXX: some files used in userland, don't get installed.

Use nanotime() to update the time fields in filesystems. Convert the code
from macros to real functions. Original patch and review from chuq.
Note: ext2fs only keeps seconds in the on-disk inode, and msdosfs does not
have enough precision for all fields, so this is not very useful for those
two.

Remove __P()

branches: 1.2.16;
add -t option for gmt time offset (normally MS-DOS filesystem has timestamp
in localtime, not GMT). PR kern/22717

branches: 1.1.2; 1.1.4;
move msdosfs code from sys/msdosfs to sys/fs/msdosfs

Sanitize the symbol namespace. NFC.

src is too big these days to tolerate superfluous apostrophes. It's
"its", people!

expose more stuff if MAKEFS is defined for the headers, and arrange for
the source file to be compilable from userland.

Stylistic changes in comments/strings:
"FAT" and "fat" are different things, use the appropriate case.
"GEMDOS" is all caps.

And correct a typo that wasn't corrected in previous.

Correct constant in comment to match the expression it is derived from.

branches: 1.4.110; 1.4.120;
- protect userland exported files against multiple inclusion.
- make sure that kernel only files don't compile in userland using #error
- XXX: some kernel only files still get installed.
- XXX: some files used in userland, don't get installed.

Remove __P()

branches: 1.2.4;
nuke trailing whitespace

branches: 1.1.2; 1.1.4; 1.1.12; 1.1.14;
move msdosfs code from sys/msdosfs to sys/fs/msdosfs

Move msdosfs_rename() and doscheckpath() to new file msdosfs_rename.c.

No functional change.

Split case folding table into separate source file and add full
copyright and permission notice from http://www.unicode.org/copyright.html

branches: 1.2.2;
Define filesystem attributes with vfs dependency.

branches: 1.1.2; 1.1.152;
move msdosfs code from sys/msdosfs to sys/fs/msdosfs

follow the steps of Andrew Doran (ad) commit and fix more s/loose/lose/ typos.
also s/beyound/beyond/ and few others along the way, mainly in comments.

s/chacters/characters/ in comments.

Sanitize the symbol namespace. NFC.

Fix false positives when comparing long file names that have the
same first 13 (or some multiple thereof) characters.

Use KASSERT for conditions that cannot be met with current parameters.

Split case folding table into separate source file and add full
copyright and permission notice from http://www.unicode.org/copyright.html

- split a long line.
- remove extra test.
- move d_namlen setting to msdosfs_vnops.c to avoid the ifdef.

Avoid unsigned/signed comparision warning to fix the build.

We can't depend on dp->d_namlen existing for the parts that are used in
makefs(8).

Add support to msdosfs and makefs to generate correct Unicode (UCS-2) directory
entries from UTF8 encoded file names.

branches: 1.10.2;
Make msdosfs time conversion use the y/m/d/h/m/s conversion functions
from clock_subr.c and compile that into the userland (and tools)
makefs as well.

branches: 1.9.12;
more cross-compile friendly.

expose more stuff if MAKEFS is defined for the headers, and arrange for
the source file to be compilable from userland.

branches: 1.7.12; 1.7.22;
ansify function definitions

Change about 4500 of the K&R function definitions to ANSI ones.
There are still about 1600 left, but they have ',' or /* ... */
in the actual variable definitions - which my awk script doesn't handle.
There are also many that need () -> (void).
(The script does handle misordered arguments.)

branches: 1.5.74; 1.5.84; 1.5.90;
merge ktrace-lwp.

Use nanotime() to update the time fields in filesystems. Convert the code
from macros to real functions. Original patch and review from chuq.
Note: ext2fs only keeps seconds in the on-disk inode, and msdosfs does not
have enough precision for all fields, so this is not very useful for those
two.

branches: 1.3.4;
nuke trailing whitespace

branches: 1.2.8; 1.2.10;
add -t option for gmt time offset (normally MS-DOS filesystem has timestamp
in localtime, not GMT). PR kern/22717

branches: 1.1.2; 1.1.4;
move msdosfs code from sys/msdosfs to sys/fs/msdosfs

Sanitize the symbol namespace. NFC.

PR kern/54759 (vm.ubc_direct deadlock when read()/write() into mapping of itself)

- Add new flag UBC_ISMAPPED which tells ubc_uiomove() the object is mmap()ed
somewhere. Use it to decide whether to do direct-mapped copy, rather than
poking around directly in the vnode in ubc_uiomove(), which is ugly and
doesn't work for tmpfs. It would be nicer to contain all this in UVM but
the filesystem provides the needed locking here (VV_MAPPED) and to
reinvent that would suck more.

- Rename UBC_UNMAP_FLAG() to UBC_VNODE_FLAGS(). Pass in UBC_ISMAPPED where
appropriate.

Replace most uses of vp->v_usecount with a call to vrefcnt(vp), a function
that hides the details and does atomic_load_relaxed(). Signature matches
FreeBSD.

branches: 1.57.2; 1.57.12;
add a genfs method to allow a file system to limit the range of pages
that are given to a single GOP_WRITE() call. needed by ZFS.

branches: 1.56.8;
Eliminate crusty debugging sludge.

We have a mostly sane vnode lifecycle now. If this needs debugging,
it should be done once at the call site of VOP_RECLAIM.

Make VOP_RECLAIM do the last unlock of the vnode.

VOP_RECLAIM naturally has exclusive access to the vnode, so having it
locked on entry is not strictly necessary -- but it means if there
are any final operations that must be done on the vnode, such as
ffs_update, requiring exclusive access to it, we can now kassert that
the vnode is locked in those operations.

We can't just have the caller release the last lock because some file
systems don't use genfs_lock, and require the vnode to remain valid
for VOP_UNLOCK to work, notably unionfs.

Make VOP_INACTIVE preserve vnode lock on return.

Discussed on tech-kern:
https://mail-index.netbsd.org/tech-kern/2017/04/01/msg021751.html

Ride 7.99.68, a bumpy bus of incremental vfs improvements!

Remove now redundant calls to fstrans_start()/fstrans_done().

branches: 1.52.2;
Remove now obsolete operation vcache_remove().

Welcome to 7.99.36

branches: 1.51.2;
Remove the 'cred' argument from bread(). Remove a now unused var in
ffs_snapshot.c. Update the man page accordingly.

ok hannken@

branches: 1.50.4;
Change msdosfs from hashlist to vcache:
- Use (dir_cluster, dir_offset, dir_generation) as key, where
dir_generation is non-zero and unique for unlinked but open nodes.
- Change deget() to return a vnode as it is unsafe to return a
referenced but unlocked denode.

msdosfs_reclaim(): add missing fstrans and protect change
of v_data with v_interlock as msdosfs_sync() now needs it.

branches: 1.48.10;
Change bread() and breadn() to never return a buffer on
error and modify all callers to not brelse() on error.

Welcome to 6.99.16

PR kern/46282 (6.0_BETA crash: msdosfs_bmap -> pcbmap -> bread -> bio_doread)

Stylistic changes in comments/strings:
"FAT" and "fat" are different things, use the appropriate case.
"GEMDOS" is all caps.

branches: 1.46.2; 1.46.12;
Rename uvm_vnp_zerorange(struct vnode *, off_t, size_t) to
ubc_zerorange(struct uvm_object *, off_t, size_t, int) changing
the first argument to an uvm_object and adding a flags argument.

Modify tmpfs_reg_resize() to zero the backing store (aobj) instead
of the vnode. Ubc_purge() no longer panics when unmounting tmpfs.

Keep uvm_vnp_zerorange() until the next kernel version bump.

Welcome to 5.99.53! Merge rmind-uvmplock branch:

- Reorganize locking in UVM and provide extra serialisation for pmap(9).
New lock order: [vmpage-owner-lock] -> pmap-lock.

- Simplify locking in some pmap(9) modules by removing P->V locking.

- Use lock object on vmobjlock (and thus vnode_t::v_interlock) to share
the locks amongst UVM objects where necessary (tmpfs, layerfs, unionfs).

- Rewrite and optimise x86 TLB shootdown code, make it simpler and cleaner.
Add TLBSTATS option for x86 to collect statistics about TLB shootdowns.

- Unify /dev/mem et al in MI code and provide required locking (removes
kernel-lock on some ports). Also, avoid cache-aliasing issues.

Thanks to Andrew Doran and Joerg Sonnenberger, as their initial patches
formed the core changes of this branch.

branches: 1.44.2;
Remove cache_purge(9) calls from reclamation routines in the file systems,
as vclean(9) performs it for us since Lite2 merge.

Msdosfs on-disk meta data is not sufficient to create or validate file handles.

Maintain a tree of file handles, create nodes from msdosfs_vptofh() and keep
them until either the file gets unlinked or the file system gets unmounted.

Fixes the msdosfs part of PR #43745 (fhopen of an unlinked file causes problems
on multiple file systems)

When truncating a file purge the fat cache after setting the new size
and after all io but before actually updating the cluster chain.

Both uvm_vnp_zerorange() and vtruncbuf() call get/putpages -> bmap -> pcbmap
and here the fat cache gets updated with information no longer valid after
truncation.

When extending a file, either by truncating or by writing past EOF make
sure the unallocated remainder of the last page gets zeroed.

Detected by fsx.

branches: 1.40.2;
Make holding v_interlock mandatory for callers of vget().

Announced some time ago on tech-kern.

Clean up vnode lock operations pass 2:

VOP_UNLOCK(vp, flags) -> VOP_UNLOCK(vp): Remove the unneeded flags argument.

Welcome to 5.99.32.

Discussed on tech-kern.

Add file system suspension support (vfs_suspend(9)) to msdosfs.

Reviewed by: Antti Kantee <pooka@netbsd.org>

branches: 1.37.2; 1.37.4;
The VATTR_NULL/VREF/VHOLD/HOLDRELE() macros lost their will to live
years ago when the kernel was modified to not alter ABI based on
DIAGNOSTIC, and now just call the respective function interfaces
(in lowercase). Plenty of mix'n match upper/lowercase has creeped
into the tree since then. Nuke the macros and convert all callsites
to lowercase.

no functional change

ansify function definitions

ANSIfy another 1261 function definitions.
The only ones left in sys are beyond by sed script!
(or in sys/dist or sys/external)
Mostly they have function pointer parameters.

Change about 4500 of the K&R function definitions to ANSI ones.
There are still about 1600 left, but they have ',' or /* ... */
in the actual variable definitions - which my awk script doesn't handle.
There are also many that need () -> (void).
(The script does handle misordered arguments.)

branches: 1.33.6; 1.33.12;
Make sure all cached buffers with valid, not yet written data have been
run through copy-on-write. Call fscow_run() with valid data where possible.

The LP_UFSCOW hack is no longer needed to protect ffs_copyonwrite() against
endless recursion.

- Add a flag B_MODIFY to bread(), breada() and breadn(). If set the caller
intends to modify the buffer returned.

- Always run copy-on-write on buffers returned from ffs_balloc().

- Add new function ffs_getblk() that gets a buffer, assigns a new blkno,
may clear the buffer and runs copy-on-write. Process possible errors
from getblk() or fscow_run(). Part of PR kern/38664.

Welcome to 4.99.63

Reviewed by: YAMAMOTO Takashi <yamt@netbsd.org>

branches: 1.32.2;
- Convert hashinit() to use kmem_alloc(). The hash tables can be large
and it's better to not have them in kmem_map.
- Convert a couple of minor items along the way to kmem_alloc().
- Fix some memory leaks.

branches: 1.31.6; 1.31.8; 1.31.10;
Fix dodgy tests of v_usecount.

Merge vmlocking2 to head.

Use uvm_vnp_setwritesize() on file extending to instruct UVM to NOT read-in
the newly added space first. This significantly speeds up write speed for
msdosfs and making it at par with ffs wich already had this patched.

Speed increase measured on my IDE disc from 2Mb/sec to 32 Mb/sec

branches: 1.28.4;
Merge ihash locking changes from the vmlocking branch.

branches: 1.27.2;
Remove the "struct lwp *" argument from all VFS and VOP interfaces.
The general trend is to remove it from all kernel interfaces and
this is a start. In case the calling lwp is desired, curlwp should
be used.

quick consensus on tech-kern

branches: 1.26.4;
Merge from vmlocking:

- Split vnode::v_flag into three fields, depending on field locking.
- simple_lock -> kmutex in a few places.
- Fix some simple locking problems.

Merge brelse() changes from the vmlocking branch.

Release vnode being created in error branch instead of leaving it
locked. Ideally the function should be rewritten to do things in
a different order, but this tries to keep changes minimal aiming
for a possible netbsd-4 pullup.

fixes PR kern/37034

Avoid stack allocation of large dirent structures in foo_readdir().

branches: 1.22.4; 1.22.6; 1.22.8; 1.22.10;
Also set writesize when creating node.

noticed by Markus W Kilbinger

branches: 1.21.2;
Using POOL_INIT here makes no sense, since file systems always have
an init method. So get rid of it and #ifdef _LKM and just always
init in the init method. Give malloc types the same treatment.
Makes file systems nicer to work with in linksetless environments
and fixes a few LKM discrepancies.

branches: 1.20.2; 1.20.4;
Pass an ipl argument to pool_init/POOL_INIT to be used when initializing
the pool's lock.

branches: 1.19.2;
Kill caddr_t; there will be some MI fallout, but it will be fixed shortly.

Call genfs_node_destroy() where appropriate.

branches: 1.17.2; 1.17.4; 1.17.6;
Support FAT filesystems on non-DEV_BSIZE media.

Based on the patches provided in PR kern/17398 by Trevin Beattie.

__unused removal on arguments; approved by core.

- sprinkle __unused on function decls.
- fix a couple of unused bugs
- no more -Wno-unused for i386

branches: 1.14.8; 1.14.10;
integrate kauth.

Coverity CID 2508: Initialize variable

branches: 1.12.4; 1.12.6; 1.12.8; 1.12.10; 1.12.12;
merge ktrace-lwp.

merge yamt-vop branch. remove following VOPs.

VOP_BLKATOFF
VOP_VALLOC
VOP_BALLOC
VOP_REALLOCBLKS
VOP_VFREE
VOP_TRUNCATE
VOP_UPDATE

branches: 1.10.2;
Remove __P()

update file timestamps for nfsd loaned-read and mmap.
PR/25279. discussed on tech-kern@.

branches: 1.8.2;
- constify genfs_ops.
- use member designators.

branches: 1.7.10;
Explicitly call pool_init() (and pool_destroy()) when being built as
an _LKM.

This adds pools to the list of things that lkms must do manually
because they're set up with link sets. Not that there's anything
wrong with link sets, but that we need to try harder to remember that
lkms are second class citizens. Of a sort.

Initialise (most) pools from a link set instead of explicit calls
to pool_init. Untouched pools are ones that either in arch-specific
code, or aren't initialiased during initial system startup.

Convert struct session, ucred and lockf to pools.

Manually attach malloc types when being built as an lkm.

branches: 1.4.2;
Back out the lwp/ktrace changes. They contained a lot of colateral damage,
and need to be examined and discussed more.

Undo part of the ktrace/lwp changes. In particular:
* Remove the "lwp *" argument that was added to vget(). Turns out
that nothing actually used it!
* Remove the "lwp *" arguments that were added to VFS_ROOT(), VFS_VGET(),
and VFS_FHTOVP(); all they did was pass it to vget() (which, as noted
above, didn't use it).
* Remove all of the "lwp *" arguments to internal functions that were added
just to appease the above.

Pass lwp pointers throughtout the kernel, as required, so that the lwpid can
be inserted into ktrace records. The general change has been to replace
"struct proc *" with "struct lwp *" in various function prototypes, pass
the lwp through and use l_proc to get the process pointer when needed.

Bump the kernel rev up to 1.6V

branches: 1.1.2;
move msdosfs code from sys/msdosfs to sys/fs/msdosfs

s/contigous/contiguous/ in comment.

Sanitize the symbol namespace. NFC.

avoid an uninit warning with GCC 9.

Rename min/max -> uimin/uimax for better honesty.

These functions are defined on unsigned int. The generic name
min/max should not silently truncate to 32 bits on 64-bit systems.
This is purely a name change -- no functional change intended.

HOWEVER! Some subsystems have

#define min(a, b) ((a) < (b) ? (a) : (b))
#define max(a, b) ((a) > (b) ? (a) : (b))

even though our standard name for that is MIN/MAX. Although these
may invite multiple evaluation bugs, these do _not_ cause integer
truncation.

To avoid `fixing' these cases, I first changed the name in libkern,
and then compile-tested every file where min/max occurred in order to
confirm that it failed -- and thus confirm that nothing shadowed
min/max -- before changing it.

I have left a handful of bootloaders that are too annoying to
compile-test, and some dead code:

cobalt ews4800mips hp300 hppa ia64 luna68k vax
acorn32/if_ie.c (not included in any kernels)
macppc/if_gm.c (superseded by gem(4))

It should be easy to fix the fallout once identified -- this way of
doing things fails safe, and the goal here, after all, is to _avoid_
silent integer truncations, not introduce them.

Maybe one day we can reintroduce min/max as type-generic things that
never silently truncate. But we should avoid doing that for a while,
so that existing code has a chance to be detected by the compiler for
conversion to uimin/uimax without changing the semantics until we can
properly audit it all. (Who knows, maybe in some cases integer
truncation is actually intended!)

Avoid undefined behavior semantics in msdosfs_fat.c

Do not change signedness bit with left shift.
While there avoid signed integer overflow.
Address both issues with using unsigned type.

msdosfs_fat.c:512:42, left shift of 1 by 31 places cannot be represented in type 'int'
msdosfs_fat.c:521:44, left shift of 1 by 31 places cannot be represented in type 'int'
msdosfs_fat.c:744:14, left shift of 1 by 31 places cannot be represented in type 'int'
msdosfs_fat.c:744:24, signed integer overflow: -2147483648 - 1 cannot be represented in type 'int [20]'
msdosfs_fat.c:840:13, left shift of 1 by 31 places cannot be represented in type 'int'
msdosfs_fat.c:840:36, signed integer overflow: -2147483648 - 1 cannot be represented in type 'int [20]'

Detected with micro-UBSan in the user mode.

branches: 1.32.2; 1.32.4;
Need strings.h for ffs()
Resolves implict declaration warning of ffs() when building tools via build.sh

branches: 1.31.10;
fix DEBUG build

Validate FAT entries to avoid some panics caused by a corrupted FAT.

Also print FAT write errors when mount is synchronous (-o sync). This
reveals problems caused by a write protected disklabel on sector 1.

Remove the 'cred' argument from bread(). Remove a now unused var in
ffs_snapshot.c. Update the man page accordingly.

ok hannken@

branches: 1.28.12; 1.28.14;
A little more debugging.

don't need sys/mount.h in userland.

tidy up debugging printfs; no functional change.

more cross-compile friendly.

expose more stuff if MAKEFS is defined for the headers, and arrange for
the source file to be compilable from userland.

Revert rev. 1.20 now that bread() has been fixed.

PR kern/46282 (6.0_BETA crash: msdosfs_bmap -> pcbmap -> bread -> bio_doread)

Change bread() and breadn() to never return a buffer on
error and modify all callers to not brelse() on error.

Welcome to 6.99.16

PR kern/46282 (6.0_BETA crash: msdosfs_bmap -> pcbmap -> bread -> bio_doread)

Stylistic changes in comments/strings:
"FAT" and "fat" are different things, use the appropriate case.
"GEMDOS" is all caps.

branches: 1.20.2;
pcbmap(): We cannot use bread() here as for the pagedaemon getblk()
may fail leading to a panic in bread().
Replace bread() with getblk() / VOP_STRATEGY() and return
an error if getblk() fails.

Fixes PR#46282: 6.0_BETA crash: msdosfs_bmap -> pcbmap -> bread -> bio_doread

This is an interim solution for easy pullup. The final solution
is be to change bread() to not return a buffer on error. As
we have to change all callers of bread() this will not qualify
for a pullup.

branches: 1.19.12; 1.19.16; 1.19.18;
On broken filesystems the fillinmap inner loop may have never read a
block, so don't try to release it if bp==NULL.

ANSIfy another 1261 function definitions.
The only ones left in sys are beyond by sed script!
(or in sys/dist or sys/external)
Mostly they have function pointer parameters.

Change about 4500 of the K&R function definitions to ANSI ones.
There are still about 1600 left, but they have ',' or /* ... */
in the actual variable definitions - which my awk script doesn't handle.
There are also many that need () -> (void).
(The script does handle misordered arguments.)

branches: 1.16.6; 1.16.12;
Make sure all cached buffers with valid, not yet written data have been
run through copy-on-write. Call fscow_run() with valid data where possible.

The LP_UFSCOW hack is no longer needed to protect ffs_copyonwrite() against
endless recursion.

- Add a flag B_MODIFY to bread(), breada() and breadn(). If set the caller
intends to modify the buffer returned.

- Always run copy-on-write on buffers returned from ffs_balloc().

- Add new function ffs_getblk() that gets a buffer, assigns a new blkno,
may clear the buffer and runs copy-on-write. Process possible errors
from getblk() or fscow_run(). Part of PR kern/38664.

Welcome to 4.99.63

Reviewed by: YAMAMOTO Takashi <yamt@netbsd.org>

branches: 1.15.18; 1.15.20; 1.15.22; 1.15.24;
Merge brelse() changes from the vmlocking branch.

branches: 1.14.6; 1.14.8; 1.14.10;
fix typos

branches: 1.13.2; 1.13.4;
Kill caddr_t; there will be some MI fallout, but it will be fixed shortly.

branches: 1.12.4;
Support FAT filesystems on non-DEV_BSIZE media.

Based on the patches provided in PR kern/17398 by Trevin Beattie.

Apply patch from PR kern/34583 sent by Rhialto, quoting him:

"Add a 3rd entry in the cache, which keeps the end position
from just before extending a file.
This has the desired effect of keeping the write speed constant."

And yes, that helps a lot copying large files... always at full speed
now. This closes my PR kern/30868 "Poor performance copying large files
on msdosfs".

Also remove a 2 if-statements testing the same condition, combine them.

All that from Rhialto, thank you very much.

branches: 1.10.6; 1.10.8; 1.10.10;
integrate kauth.

Coverity CID 839: Add KASSERT before deref.

Coverity CID 838: Add KASSERT before deref.

branches: 1.7.4; 1.7.6; 1.7.8; 1.7.10; 1.7.12;
Remove leading __ from __(const|inline|signed|volatile) -- it is obsolete.

merge ktrace-lwp.

Remove __P()

branches: 1.4.2; 1.4.4;
nuke trailing whitespace

branches: 1.3.4; 1.3.6;
Replace the statfs() family of system calls with statvfs().
Retain binary compatibility.

Fix a panic that occurred when trying to traverse a corrupt msdosfs
filesystem. With this particular corruption, the code in pcbmap()
would compute an offset into an array that was way out of bounds,
so check the bounds before trying to access and return an error if
the offset would be out of bounds.

branches: 1.1.2; 1.1.4;
move msdosfs code from sys/msdosfs to sys/fs/msdosfs

s/blity/bility/ in various words, mainly in comments.

Sanitize the symbol namespace. NFC.

Move msdosfs_rename() and doscheckpath() to new file msdosfs_rename.c.

No functional change.

Factor out the lookup results from struct denode.

No functional change.

Fix all remaining typos, mainly in comments but also in few definitions and log messages, reported by me in PR kern/54889.
Also fixed some additional typos in comments, found on review of same files or typos.

branches: 1.36.8;
Merge the remaining changes from the ad-namecache branch, affecting namei()
and getcwd():

- push vnode locking back as far as possible.
- do most lookups directly in the namecache, avoiding vnode locks & refs.
- don't block new refs to vnodes across VOP_INACTIVE().
- get shared locks for VOP_LOOKUP() if the file system supports it.
- correct lock types for VOP_ACCESS() / VOP_GETATTR() in a few places.

Possible future enhancements:

- make the lookups lockless.
- support dotdot lookups by being lockless and inferring absence of chroot.
- maybe make it work for layered file systems.
- avoid vnode references at the root & cwd.

branches: 1.35.18; 1.35.24;
Add support to msdosfs and makefs to generate correct Unicode (UCS-2) directory
entries from UTF8 encoded file names.

Remove the 'cred' argument from bread(). Remove a now unused var in
ffs_snapshot.c. Update the man page accordingly.

ok hannken@

branches: 1.33.4;
Change msdosfs from hashlist to vcache:
- Use (dir_cluster, dir_offset, dir_generation) as key, where
dir_generation is non-zero and unique for unlinked but open nodes.
- Change deget() to return a vnode as it is unsafe to return a
referenced but unlocked denode.

branches: 1.32.2;
Change vnode operation lookup to return the resulting vnode *vpp unlocked.
Change cache_lookup() to return an unlocked vnode.

Discussed on tech-kern@

Welcome to 6.99.31

Change vnode operations create, mknod, mkdir and symlink to return
the resulting vnode *vpp unlocked.

Discussed on tech-kern@

Welcome to 6.99.30

don't treat adjacent members as a larger array
Coverity CID 977367

branches: 1.29.2;
more cross-compile friendly.

expose more stuff if MAKEFS is defined for the headers, and arrange for
the source file to be compilable from userland.

Change bread() and breadn() to never return a buffer on
error and modify all callers to not brelse() on error.

Welcome to 6.99.16

PR kern/46282 (6.0_BETA crash: msdosfs_bmap -> pcbmap -> bread -> bio_doread)

Excise struct componentname from the namecache.

This uglifies the interface, because several operations need to be
passed the namei flags and cache_lookup also needs for the time being
to be passed cnp->cn_nameiop. Nonetheless, it's a net benefit.

The glop should be able to go away eventually but requires structural
cleanup elsewhere first.

This change requires a kernel bump.

Disentangle the namecache from the internals of namei.

- Move the namecache's hash computation to inside the namecache code,
instead of being spread out all over the place. Remove cn_hash from
struct componentname and delete all uses of it.

- It is no longer necessary (if it ever was) for cache_lookup and
cache_lookup_raw to clear MAKEENTRY from cnp->cn_flags for the cases
that cache_enter already checks for.

- Rearrange the interface of cache_lookup (and cache_lookup_raw) to
make it somewhat simpler, to exclude certain nonexistent error
conditions, and (most importantly) to make it not require write access
to cnp->cn_flags.

This change requires a kernel bump.

branches: 1.24.2;
Move some the test for MAKEENTRY into the cache_enter(9). Make some
variables in vfs_cache.c static, __read_mostly, etc.

No objection on tech-kern@.

branches: 1.23.8; 1.23.14;
Abolish the SAVENAME and HASBUF flags. There is now always a buffer,
so the path in a struct componentname is now always valid during VOP
calls.

Return EINVAL for rename and delete operations to the
root directory instead of the erroneous EROFS.

Clean up vnode lock operations pass 2:

VOP_UNLOCK(vp, flags) -> VOP_UNLOCK(vp): Remove the unneeded flags argument.

Welcome to 5.99.32.

Discussed on tech-kern.

branches: 1.20.2; 1.20.4;
The VATTR_NULL/VREF/VHOLD/HOLDRELE() macros lost their will to live
years ago when the kernel was modified to not alter ABI based on
DIAGNOSTIC, and now just call the respective function interfaces
(in lowercase). Plenty of mix'n match upper/lowercase has creeped
into the tree since then. Nuke the macros and convert all callsites
to lowercase.

no functional change

ANSIfy another 1261 function definitions.
The only ones left in sys are beyond by sed script!
(or in sys/dist or sys/external)
Mostly they have function pointer parameters.

Change about 4500 of the K&R function definitions to ANSI ones.
There are still about 1600 left, but they have ',' or /* ... */
in the actual variable definitions - which my awk script doesn't handle.
There are also many that need () -> (void).
(The script does handle misordered arguments.)

branches: 1.17.2;
From FreeBSD, 10 years ago;

Now empty DOS filesystems default to long file names. Non-empty filesystems
without traces of Win95 default to short file names, as before.

branches: 1.16.6; 1.16.8;
Make sure all cached buffers with valid, not yet written data have been
run through copy-on-write. Call fscow_run() with valid data where possible.

The LP_UFSCOW hack is no longer needed to protect ffs_copyonwrite() against
endless recursion.

- Add a flag B_MODIFY to bread(), breada() and breadn(). If set the caller
intends to modify the buffer returned.

- Always run copy-on-write on buffers returned from ffs_balloc().

- Add new function ffs_getblk() that gets a buffer, assigns a new blkno,
may clear the buffer and runs copy-on-write. Process possible errors
from getblk() or fscow_run(). Part of PR kern/38664.

Welcome to 4.99.63

Reviewed by: YAMAMOTO Takashi <yamt@netbsd.org>

branches: 1.15.14; 1.15.16; 1.15.18; 1.15.20;
Remove the "struct lwp *" argument from all VFS and VOP interfaces.
The general trend is to remove it from all kernel interfaces and
this is a start. In case the calling lwp is desired, curlwp should
be used.

quick consensus on tech-kern

branches: 1.14.4;
Merge from vmlocking:

- Split vnode::v_flag into three fields, depending on field locking.
- simple_lock -> kmutex in a few places.
- Fix some simple locking problems.

Merge brelse() changes from the vmlocking branch.

branches: 1.12.2; 1.12.14; 1.12.16; 1.12.18;
Kill caddr_t; there will be some MI fallout, but it will be fixed shortly.

branches: 1.11.2;
a smorgasbord of improvements to vnode locking and path lookup:
- LOCKPARENT is no longer relevant for lookup(), relookup() or VOP_LOOKUP().
these now always return the parent vnode locked. namei() works as before.
lookup() and various other paths no longer acquire vnode locks in the
wrong order via vrele(). fixes PR 32535.
as a nice side effect, path lookup is also up to 25% faster.
- the above allows us to get rid of PDIRUNLOCK.
- also get rid of WANTPARENT (just use LOCKPARENT and unlock it).
- remove an assumption in layer_node_find() that all file systems implement
a recursive VOP_LOCK() (unionfs doesn't).
- require that all file systems supply vfs_vptofh and vfs_fhtovp routines.
fill in eopnotsupp() for file systems that don't support being exported
and remove the checks for NULL. (layerfs calls these without checking.)
- in union_lookup1(), don't change refcounts in the ISDOTDOT case, just
adjust which vnode is locked. fixes PR 33374.
- apply fixes for ufs_rename() from ufs_vnops.c rev. 1.61 to ext2fs_rename().

branches: 1.10.2;
Support FAT filesystems on non-DEV_BSIZE media.

Based on the patches provided in PR kern/17398 by Trevin Beattie.

branches: 1.9.8; 1.9.10;
integrate kauth.

branches: 1.8.4; 1.8.6; 1.8.8; 1.8.10; 1.8.12;
merge ktrace-lwp.

mention what is the failure case, from FreeBSD revision 1.46 cvs log

PR/31312: Juan RP: Don't use negative cacheing on msdosfs, because of the
evil case preserving and case sensitive semantics. From FreeBSD.

branches: 1.5.2;
- rename variables to avoid shadowing.
- add a few const.

nuke trailing whitespace

branches: 1.3.2; 1.3.10; 1.3.12;
Back out the lwp/ktrace changes. They contained a lot of colateral damage,
and need to be examined and discussed more.

Pass lwp pointers throughtout the kernel, as required, so that the lwpid can
be inserted into ktrace records. The general change has been to replace
"struct proc *" with "struct lwp *" in various function prototypes, pass
the lwp through and use l_proc to get the process pointer when needed.

Bump the kernel rev up to 1.6V

branches: 1.1.2;
move msdosfs code from sys/msdosfs to sys/fs/msdosfs

From genfs_rename.c:

* XXX Want a better equality test. `tcnp->cn_cred == cred'
* hoses p2k because puffs transmits the creds separately and
* allocates distinct but equivalent structures for them.

Fixes rename crash in rump_msdos.

branches: 1.3.4;
Sanitize the symbol namespace. NFC.

Convert msdosfs_rename() to use genfs_sane_rename().

Based on work by Taylor R Campbell.

Move msdosfs_rename() and doscheckpath() to new file msdosfs_rename.c.

No functional change.

branches: 1.2.2; 1.2.18;
Toolify.

Split case folding table into separate source file and add full
copyright and permission notice from http://www.unicode.org/copyright.html

Lock vnode for vinvalbuf().

Sanitize the symbol namespace. NFC.

Enable to mount Raspberry Pi Pico's USB mass storage partition

Fix PR kern/55985.
O.k. by thorpej@.

Pull-up to netbsd-8 and netbsd-9.

branches: 1.135.2;
Replace most uses of vp->v_usecount with a call to vrefcnt(vp), a function
that hides the details and does atomic_load_relaxed(). Signature matches
FreeBSD.

branches: 1.134.2;
Merge the remaining changes from the ad-namecache branch, affecting namei()
and getcwd():

- push vnode locking back as far as possible.
- do most lookups directly in the namecache, avoiding vnode locks & refs.
- don't block new refs to vnodes across VOP_INACTIVE().
- get shared locks for VOP_LOOKUP() if the file system supports it.
- correct lock types for VOP_ACCESS() / VOP_GETATTR() in a few places.

Possible future enhancements:

- make the lookups lockless.
- support dotdot lookups by being lockless and inferring absence of chroot.
- maybe make it work for layered file systems.
- avoid vnode references at the root & cwd.

Use the module subsystem's ability to process SYSCTL_SETUP() entries to
automate installation of sysctl nodes.

Note that there are still a number of device and pseudo-device modules
that create entries tied to individual device units, rather than to the
module itself. These are not changed.

Tighten up the locking around vp->v_iflag a little more after the recent
split of vmobjlock & v_interlock.

VFS_VGET(), VFS_ROOT(), VFS_FHTOVP(): give them a "int lktype" argument, to
allow us to get shared locks (or no lock) on the returned vnode. Matches
FreeBSD.

branches: 1.130.4; 1.130.6;
Rename min/max -> uimin/uimax for better honesty.

These functions are defined on unsigned int. The generic name
min/max should not silently truncate to 32 bits on 64-bit systems.
This is purely a name change -- no functional change intended.

HOWEVER! Some subsystems have

#define min(a, b) ((a) < (b) ? (a) : (b))
#define max(a, b) ((a) > (b) ? (a) : (b))

even though our standard name for that is MIN/MAX. Although these
may invite multiple evaluation bugs, these do _not_ cause integer
truncation.

To avoid `fixing' these cases, I first changed the name in libkern,
and then compile-tested every file where min/max occurred in order to
confirm that it failed -- and thus confirm that nothing shadowed
min/max -- before changing it.

I have left a handful of bootloaders that are too annoying to
compile-test, and some dead code:

cobalt ews4800mips hp300 hppa ia64 luna68k vax
acorn32/if_ie.c (not included in any kernels)
macppc/if_gm.c (superseded by gem(4))

It should be easy to fix the fallout once identified -- this way of
doing things fails safe, and the goal here, after all, is to _avoid_
silent integer truncations, not introduce them.

Maybe one day we can reintroduce min/max as type-generic things that
never silently truncate. But we should avoid doing that for a while,
so that existing code has a chance to be detected by the compiler for
conversion to uimin/uimax without changing the semantics until we can
properly audit it all. (Who knows, maybe in some cases integer
truncation is actually intended!)

branches: 1.129.2; 1.129.4;
relax sanity check. It's ok to have more FAT sectors than needed.

Add more sanity checks for BPB parameters. Handle FAT12 format for media
with sectors >= 32kByte.

Does fix PR 52485.

branches: 1.127.2; 1.127.4;
Remove unused argument "nextp" from vfs_busy() and vfs_unbusy().
Remove argument "keepref" from vfs_unbusy() and add vfs_ref() where needed.

Add vfs_ref(mp) and vfs_rele(mp) to add or remove a reference to
struct mount. Rename vfs_destroy(mp) to vfs_rele(mp) and replace
incrementing mp->mnt_refcnt with vfs_ref(mp).

KASSERT(mutex_owned(vp->v_interlock)) in vnode iterator selector.

Remove now redundant calls to fstrans_start()/fstrans_done().

Enable fstrans on all file systems.

Welcome to 7.99.61

Add generic genfs_suspendctl() and use it for all file systems.
Layered file systems need work.

Untangle VFS_SYNC() from VFS_SUSPENDCTL().

Take vnode lock for VOP_FSYNC().

branches: 1.119.2;
Move vnode members "v_freelisthd" and "v_freelist" from "struct vnode"
to "struct vnode_impl" and rename to "vi_lrulisthd" and "vi_lrulist".

No functional change intended.

Welcome to 7.99.48

branches: 1.118.2;
Remove the 'cred' argument from bread(). Remove a now unused var in
ffs_snapshot.c. Update the man page accordingly.

ok hannken@

catch up with DPRINTF change

add some more paranoid checks about secsize and struct use.

branches: 1.115.2; 1.115.4;
Make DPRINTF more understandable, and replace my previous #ifdef DIAGNOSTIC...

Limit the minimum size of a disk sector to 512 bytes, to prevent memory
overflow on extremely low secsize. This normally conforms to the old standard
(for which there doesn't seem to be a clear spec). Since 2011, IDEMA's Advanced
Format standardizes it to 4k, so this change won't cause any trouble on
new devices.

Put the printf under DIAGNOSTIC temporarily to see if someone complains.

after a quick discussion on tech-kern

Correct the bread size of struct fsinfo from Gerald Lee at DELL dot com

Minor changes:
- malloc()+memset() -> malloc(|M_ZERO)
- rename 'vers' to 'FSVers'
- declare 'ExtFlags' instead of calling getushort() two times

Remove ROOTNAME (unused).

- Perform sanity checks not just for GEMDOSFS, but for all FAT devices. This
also fixes a division-by-zero bug that could crash the system.
- Define GEMDOSFS_BSIZE instead of a hard-coded 512 value, and remove 'bsize'.
- Rename 'tmp' to 'BlkPerSec'.

From me, FreeBSD, OpenBSD and the FAT specification.

ok christos@

Change msdosfs from hashlist to vcache:
- Use (dir_cluster, dir_offset, dir_generation) as key, where
dir_generation is non-zero and unique for unlinked but open nodes.
- Change deget() to return a vnode as it is unsafe to return a
referenced but unlocked denode.

Introduce a selector function to the vfs vnode iterator so that we don't
need to vget() vnodes that we are not interested at, and optimize locking
a bit. Iterator changes reviewed by Hannken (thanks), the rest of the bugs
are mine.

An (un)privileged user can easily make the kernel dereference a NULL
pointer.

The kernel allows 'data' to be NULL; it's the fs's responsibility to
ensure that it isn't NULL (if the fs actually needs data).

ok christos@

branches: 1.106.2;
Change all vfsops to use C99 designated initializers.

No functional changes intended.

Change msdosfs_sync() to use vfs_vnode_iterator.

Ensure that the top level sysctl nodes (kern, vfs, net, ...) exist before
the sysctl link sets are processed, and remove redundancy.

Shaves >13kB off of an amd64 GENERIC, not to mention >1k duplicate
lines of code.

change the mountlist CIRCLEQ into a TAILQ

Replace macro v_specmountpoint with two functions spec_node_getmountedfs()
and spec_node_setmountedfs() to manage the file system mounted on a device.
Assert the device is a block device.

Welcome to 6.99.24

Discussed on tech-kern@ some time ago.

Reviewed by: David Holland <dholland@netbsd.org>

branches: 1.101.4;
Don't attempt to mount file system with clusters larger than MAXBSIZE.

Stylistic changes in comments/strings:
"FAT" and "fat" are different things, use the appropriate case.
"GEMDOS" is all caps.

Check that the FSInfo block's next free cluster suggestion is actually
a cluster within the bounds of the volume too.

((u_long)-1) will not always be 0xffffffff, which is what we actually
want to test against to determine if the FSInfo block's next free
cluster suggestion is valid

We don't actually want to round the number of elements in the bitmap
down. Fixes a self-inflicted buffer overrun.

(This was detected by chance that the top of the bitmap coincided with
a page boundary.)

branches: 1.96.2;
Revert rev 1.95 since getdisksize() no longer returns secsize=0.

Add a sanity check if secsize returned from getdisksize() isn't bogus.
This prevent possible panic "panic: buf mem pool index 23" later in
vfs_bio.c:buf_mempoolidx().
(I'm not sure if it's okay for getdisksize() to assume that
partinfo taken from DIOCGPART is properly initialized
on all disk(9) devices or not)

See also:
http://mail-index.NetBSD.org/source-changes/2012/06/30/msg035298.html

Replace the remaining KAUTH_GENERIC_ISSUSER authorization calls with
something meaningful. All relevant documentation has been updated or
written.

Most of these changes were brought up in the following messages:

http://mail-index.netbsd.org/tech-kern/2012/01/18/msg012490.html
http://mail-index.netbsd.org/tech-kern/2012/01/19/msg012502.html
http://mail-index.netbsd.org/tech-kern/2012/02/17/msg012728.html

Thanks to christos, manu, njoly, and jmmv for input.

Huge thanks to pgoyette for spinning these changes through some build
cycles and ATF.

branches: 1.93.4; 1.93.6;
VOP_OPEN() needs a locked vnode. All these copy-and-pasted xxxfs_mount()
implementations need more review.

branches: 1.92.2;
As vnalloc() always allocates with PR_WAITOK there is no longer the need
to test its result for NULL.

Welcome to 5.99.53! Merge rmind-uvmplock branch:

- Reorganize locking in UVM and provide extra serialisation for pmap(9).
New lock order: [vmpage-owner-lock] -> pmap-lock.

- Simplify locking in some pmap(9) modules by removing P->V locking.

- Use lock object on vmobjlock (and thus vnode_t::v_interlock) to share
the locks amongst UVM objects where necessary (tmpfs, layerfs, unionfs).

- Rewrite and optimise x86 TLB shootdown code, make it simpler and cleaner.
Add TLBSTATS option for x86 to collect statistics about TLB shootdowns.

- Unify /dev/mem et al in MI code and provide required locking (removes
kernel-lock on some ports). Also, avoid cache-aliasing issues.

Thanks to Andrew Doran and Joerg Sonnenberger, as their initial patches
formed the core changes of this branch.

branches: 1.90.2;
Msdosfs on-disk meta data is not sufficient to create or validate file handles.

Maintain a tree of file handles, create nodes from msdosfs_vptofh() and keep
them until either the file gets unlinked or the file system gets unmounted.

Fixes the msdosfs part of PR #43745 (fhopen of an unlinked file causes problems
on multiple file systems)

branches: 1.89.2;
Extend the range of fstrans transactions to a sequence of vnode operations
on a locked vnode. This leaves a suspended file system and therefore a
snapshot with either all or no operations of such a sequence done.

msdosfs_sync: check for dead vnode (denode == NULL) before testing denode flags.

Make holding v_interlock mandatory for callers of vget().

Announced some time ago on tech-kern.

Clean up vnode lock operations pass 2:

VOP_UNLOCK(vp, flags) -> VOP_UNLOCK(vp): Remove the unneeded flags argument.

Welcome to 5.99.32.

Discussed on tech-kern.

polish previous

If getdisksize() fails (happens e.g. on fss block devices), don't
give up if we don't really need the information provided by it.

The *_modcmd functions use the module name as prefix.

Make module name match MOUNT_NAME. Inspired by PR kern/43110.

Add file system suspension support (vfs_suspend(9)) to msdosfs.

Reviewed by: Antti Kantee <pooka@netbsd.org>

* Don't care about VOP_CLOSE() error in unmount. In the extremely
unlike event it did fail, the kernel would double lutz to doom
(in failure devvp now remains unmountable until reboot. fans
of complicated & untested error branches may attempt to gunk this
up. i'm not one of them).
* cosmetic surgery: cut extra ;

branches: 1.79.2; 1.79.4;
Replace individual queries for partition information with
new helper function.

Drop two uses of disk label data.

msdosfs and cd9660 are the only filesystems that verify the filesystem
type in the label. This is the wrong place, sanity checks should only
rely on the inner structure of the filesystem (like signatures or
magic numbers).

msdosfs also used the device type information from the label to
deduce a filesystem parameter heuristically for the gemdos variant.
If there is no information inside the filesystem data itself, this
should be an explicit mount option.

Fetch sector size also from wedges.

Convert 67 namei call sites to use namei_simple, in these functions:

check_console, veriexecclose, veriexec_delete, veriexec_file_add,
emul_find_root, coff_load_shlib (sh3 version), coff_load_shlib,
compat_20_sys_statfs, compat_20_netbsd32_statfs,
ELFNAME2(netbsd32,probe_noteless), darwin_sys_statfs,
ibcs2_sys_statfs, ibcs2_sys_statvfs, linux_sys_uselib,
osf1_sys_statfs, sunos_sys_statfs, sunos32_sys_statfs,
ultrix_sys_statfs, do_sys_mount, fss_create_files (3 of 4),
adosfs_mount, cd9660_mount, coda_ioctl, coda_mount, ext2fs_mount,
ffs_mount, filecore_mount, hfs_mount, lfs_mount, msdosfs_mount,
ntfs_mount, sysvbfs_mount, udf_mount, union_mount, sys_chflags,
sys_lchflags, sys_chmod, sys_lchmod, sys_chown, sys_lchown,
sys___posix_chown, sys___posix_lchown, sys_link, do_sys_pstatvfs,
sys_quotactl, sys_revoke, sys_truncate, do_sys_utimes, sys_extattrctl,
sys_extattr_set_file, sys_extattr_set_link, sys_extattr_get_file,
sys_extattr_get_link, sys_extattr_delete_file,
sys_extattr_delete_link, sys_extattr_list_file, sys_extattr_list_link,
sys_setxattr, sys_lsetxattr, sys_getxattr, sys_lgetxattr,
sys_listxattr, sys_llistxattr, sys_removexattr, sys_lremovexattr

All have been scrutinized (several times, in fact) and compile-tested,
but not all have been explicitly tested in action.

XXX: While I haven't (intentionally) changed the use or nonuse of
XXX: TRYEMULROOT in any of these places, I'm not convinced all the
XXX: uses are correct; an audit might be desirable.

Wrap some long lines in a comment, pointed out by ad@ - thanks!

While here, wrap two more long lines.

Add genfs_can_mount() and use it to prevent some more code duplication of
the security checks when mounting a device (VOP_ACCESS() + kauth(9) call)).

Proposed with no objections on tech-kern@:

http://mail-index.netbsd.org/tech-kern/2009/04/20/msg004859.html

The vnode is always expected to be locked, so no locking is done outside
the file-system code.

ansify function definitions

Change about 4500 of the K&R function definitions to ANSI ones.
There are still about 1600 left, but they have ',' or /* ... */
in the actual variable definitions - which my awk script doesn't handle.
There are also many that need () -> (void).
(The script does handle misordered arguments.)

branches: 1.71.2;
Remove additional GEMDOS only check on fstype, and update comment

Remove pm_Heads check in msdosfs_mountfs; this value isn't really used,
FreeBSD and OpenBSD both removed this check years ago, and this makes an
8GB Memory Stick Pro Duo formatted with a PSP work.

The great QUOTA purge of '08: get rid of those #ifdef QUOTA and
#include "opt_quota.h" which do exactly nothing. Speeds up kernel
compilation by 1.375*10^-20001 seconds. But leave the most moxious
comment in msdosfs_vfsops untouched.

branches: 1.68.4; 1.68.6;
Create sysctl entries during module initialisation and destroy them
appropriately.

Many of these file systems are now ready for modularisation.

branches: 1.67.2;
Make sure all cached buffers with valid, not yet written data have been
run through copy-on-write. Call fscow_run() with valid data where possible.

The LP_UFSCOW hack is no longer needed to protect ffs_copyonwrite() against
endless recursion.

- Add a flag B_MODIFY to bread(), breada() and breadn(). If set the caller
intends to modify the buffer returned.

- Always run copy-on-write on buffers returned from ffs_balloc().

- Add new function ffs_getblk() that gets a buffer, assigns a new blkno,
may clear the buffer and runs copy-on-write. Process possible errors
from getblk() or fscow_run(). Part of PR kern/38664.

Welcome to 4.99.63

Reviewed by: YAMAMOTO Takashi <yamt@netbsd.org>

Convert file systems to dynamically attach with the new module interface.
Make VFS hooks dynamic while we're here and say farewell to VFS_ATTACH and
VFS_HOOKS_ATTACH linksets.

As a consequence, most of the file systems can now be loaded as new style
modules.

Quick sanity check by ad@.

branches: 1.65.2;
PR kern/38141 lookup/vfs_busy acquire rwlock recursively

Simplify the mount locking. Remove all the crud to deal with recursion on
the mount lock, and crud to deal with unmount as another weirdo lock.

Hopefully this will once and for all fix the deadlocks with this. With this
commit there are two locks on each mount:

- krwlock_t mnt_unmounting. This is used to prevent unmount across critical
sections like getnewvnode(). It's only ever read locked with rw_tryenter(),
and is only ever write locked in dounmount(). A write hold can't be taken
on this lock if the current LWP could hold a vnode lock.

- kmutex_t mnt_updating. This is taken by threads updating the mount, for
example when going r/o -> r/w, and is only present to serialize updates.
In order to take this lock, a read hold must first be taken on
mnt_unmounting, and the two need to be held across the operation.

One effect of this change: previously if an unmount failed, we would make a
half hearted attempt to back out of it gracefully, but that was unlikely to
work in a lot of cases. Now while an unmount that will be aborted is in
progress, new file operations within the mount will fail instead of being
delayed. That is unlikely to be a problem though, because if the admin
requests unmount of a file system then s(he) has made a decision to deny
access to the resource.

Make various bits of debug code compile again.

PR kern/38135 vfs_busy/vfs_trybusy confusion

The previous fix worked, but it opened a window where mounts could have
disappeared from mountlist while the caller was traversing it using
vfs_trybusy(). Fix that.

PR kern/38057 ffs makes assuptions about devvp file system
PR kern/33406 softdeps get stuck in endless loop

Introduce VFS_FSYNC() and call it when syncing a block device, if it
has a mounted file system.

branches: 1.61.6; 1.61.8; 1.61.10;
PR kern/37706 (forced unmount of file systems is unsafe):

- Do reference counting for 'struct mount'. Each vnode associated with a
mount takes a reference, and in turn the mount takes a reference to the
vfsops.
- Now that mounts are reference counted, replace the overcomplicated mount
locking inherited from 4.4BSD with a recursable rwlock.

Fix some race conditions in rename.
Introduce a per-FS rename lock and new vfsops to manipulate it.
Get this lock while renaming. Also add another relookup() in do_sys_rename,
which is a hack to kludge around some of the worst deficiencies of
ufs_rename.
reviewed-by: pooka (and an earlier rev by ad)
posted on tech-kern with no objections.

specfs changes for PR kern/37717 (raidclose() is no longer called on
shutdown). There are still problems with device access and a PR will be
filed.

- Kill checkalias(). Allow multiple vnodes to reference a single device.

- Don't play dangerous tricks with block vnodes to ensure that only one
vnode can describe a block device. Instead, prohibit concurrent opens of
block devices. As a bonus remove the unreliable code that prevents
multiple file system mounts on the same device. It's no longer needed.

- Track opens by vnode and by device. Issue cdev_close() when the last open
goes away, instead of abusing vnode::v_usecount to tell if the device is
open.

Politely refuse to mount a file system which says its FAT size is
zero - our implementation can't handle it (how sensible handling
a case like that would be is a whole other debate).

fixes panic reported by Jukka Salmi on current-users

valloc -> vnalloc, vfree -> vnfree
Avoids collision with userland valloc(3).

no functional change
ad ok

Merge vmlocking2 to head.

branches: 1.55.4;
Remove cn_lwp from struct componentname. curlwp should be used
from on. The NDINIT() macro no longer takes the lwp parameter and
associates the credentials of the calling thread with the namei
structure.

branches: 1.54.2;
Remove the "struct lwp *" argument from all VFS and VOP interfaces.
The general trend is to remove it from all kernel interfaces and
this is a start. In case the calling lwp is desired, curlwp should
be used.

quick consensus on tech-kern

branches: 1.53.4;
Merge from vmlocking:

- Split vnode::v_flag into three fields, depending on field locking.
- simple_lock -> kmutex in a few places.
- Fix some simple locking problems.

Merge brelse() changes from the vmlocking branch.

Avoid stack allocation of large dirent structures in foo_readdir().

branches: 1.50.2; 1.50.4; 1.50.6; 1.50.8;
* nuke the nameidata parameter from VFS_MOUNT(). Nobody on tech-kern
knew what it was supposed to be used for and wrstuden gave a go-ahead
* while rototilling, convert file systems which went easily to
use VFS_PROTOS() instead of manually prototyping the methods

Use eopnotsupp() instead of vfs_stdsuspendctl() and retire the latter.

In sync, skip over vnodes based on if they are clean rather than
if they have pages.

branches: 1.47.2;
Make set_statvfs_info() take a parameter for the vfs name instead
of always retrieving it from mp->mnt_op->vfs_name

christos ok

Change the VFS_MOUNT() interface so that the 'data' buffer passed to the
fs code is a kernel buffer, pass though the length of the buffer as well.
Since the length of the userspace buffer isn'it (yet) passed through the mount
system call, add a field to the vfsops structure containing the default length.
Split sys_mount() for calls from compat code.
Ride one of the recent kernel version changes - old fs LKMs will load, but
sys_mount() will reject any attempt to use them.

Using POOL_INIT here makes no sense, since file systems always have
an init method. So get rid of it and #ifdef _LKM and just always
init in the init method. Give malloc types the same treatment.
Makes file systems nicer to work with in linksetless environments
and fixes a few LKM discrepancies.

branches: 1.44.4; 1.44.6;
fix the problem reported in kern/35457, "cant mount msdosfs on floppy".

Since 1.40, which introduced support for non-DEV_BSIZE media,
mounting msdos floppy returned ENOTTY.

This is because floppy driver does not support DIOCGPART or DIOCWEDGEINFO
ioctl.

Those ioctls should not be a requirement for mounting msdosfs.

This patch is made by Christian Biere.

branches: 1.43.2;
Fix typo
- * assoicated with this mount point, start over.
+ * associated with this mount point, start over.

- If the filesystem lives on a wedge, then we try to get wedge information
since DIOCGPART is going to fail. Unfortunately there is no way to get the
geometry information we need from the wedge; it would be nice for wedges
to support a geometry ioctl. The values we cannot retrieve are marked with
XXX.
- Add a lot more debugging.

New file system suspension API to replace vn_start_write and vn_finished_write.
The suspension helpers are now put into file system specific operations.
This means every file system not supporting these helpers cannot be suspended
and therefore snapshots are no longer possible.

Implemented for file systems of type ffs.

The new API is enabled on a kernel option NEWVNGATE. This option is
not enabled by default in any kernel config.

Presented and discussed on tech-kern with much input from
Bill Studenmund <wrstuden@netbsd.org> and YAMAMOTO Takashi <yamt@netbsd.org>.

Welcome to 4.99.9 (new vfs op vfs_suspendctl).

branches: 1.40.2;
Support FAT filesystems on non-DEV_BSIZE media.

Based on the patches provided in PR kern/17398 by Trevin Beattie.

__unused removal on arguments; approved by core.

Revisit mnt_vnodelist TAILQ patch. Remove all suspicious TAILQ_FOREACH()
loops where vnodes can get removed or added during the loops. This could
lead to panic's on unmount since nodes are skipped or otherwise
TAILQ_NEXT(0xdeadbeef, ...) was dereferenced.

Replace the LIST structure mp->mnt_vnodelist to a TAILQ structure since all
vnodes were synced and processed backwards. This meant that the last
accessed node was processed first and the earlierst last.

An extra benefit is the removal of the ugly hack from the Berkly days on
LFS.

In the proces, i've also replaced the various variations hand written loops
by the TAILQ_FOREACH() macro's.

- sprinkle __unused on function decls.
- fix a couple of unused bugs
- no more -Wno-unused for i386

branches: 1.35.2; 1.35.4;
add missing initializers

When mounting an MSDOS fileysystem, do not require that bytes 252 and
253 of the superblock be zero. Searching the net failed to find any
justification for checking these bytes; all available references say
that they are part of the boot code and not BOOTSIG2 and BOOTSIG3.

Modify the MSDOS 7.1 bootsector definition to have 420 bytes of boot
code and no BOOTSIG[23], rather than 418 bytes of boot code, to follow
available references and apparent Windows practice. A test build
showed that these defines are not used other than in the check removed
by this commit.

Patch tested on netbsd-3, and enabled mounting of a 4 GB CF formatted
under Windows XP and then in a digital camera. The CF was previously
unmountable.

Concept approved on tech-kern by christos@ and martin@.

Use the LWP cached credentials where sane.

Fix alignement problems for fhandle_t, exposed by gcc4.1.

While touching all vptofh/fhtovp functions, get rid of VFS_MAXFIDSIZ,
version the getfh(2) syscall and explicitly pass the size available in
the filehandle from userland.

Discussed on tech-kern, with lots of help from yamt (thanks!).

branches: 1.31.4;
integrate kauth.

branches: 1.30.2; 1.30.4; 1.30.6;
Use device_class() instead of accessing dv_class directly.

branches: 1.29.2; 1.29.4; 1.29.6;
merge ktrace-lwp.

Apply the NFS exports list rototill patch:

- Remove all NFS related stuff from file system specific code.
- Drop the vfs_checkexp hook and generalize it in the new nfs_check_export
function, thus removing redundancy from all file systems.
- Move all NFS export-related stuff from kern/vfs_subr.c to the new
file sys/nfs/nfs_export.c. The former was becoming large and its code
is always compiled, regardless of the build options. Using the latter,
the code is only compiled in when NFSSERVER is enabled. While doing this,
also make some functions in nfs_subs.c conditional to NFSSERVER.
- Add a new command in nfssvc(2), called NFSSVC_SETEXPORTSLIST, that takes a
path and a set of export entries. At the moment it can only clear the
exports list or append entries, one by one, but it is done in a way that
allows setting the whole set of entries atomically in the future (see the
comment in mountd_set_exports_list or in doc/TODO).
- Change mountd(8) to use the nfssvc(2) system call instead of mount(2) so
that it becomes file system agnostic. In fact, all this whole thing was
done to remove a 'XXX' block from this utility!
- Change the mount*, newfs and fsck* userland utilities to not deal with NFS
exports initialization; done internally by the kernel when initializing
the NFS support for each file system.
- Implement an interface for VFS (called VFS hooks) so that several kernel
subsystems can run arbitrary code upon receipt of specific VFS events.
At the moment, this only provides support for unmount and is used to
destroy NFS exports lists from the file systems being unmounted, though it
has room for extension.

Thanks go to yamt@, chs@, thorpej@, wrstuden@ and others for their comments
and advice in the development of this patch.

PR/23773: Tero Kivinen: Cannot mount msdos filesystems if filesystem size >
128GB
PR/31287: Fabien Devaux: msdosfs now can mount >128MB filesystems

Remove __P()

Let this build when MSDOSFS_DEBUG is defined (v_id disappeared a long time
ago but it was still used here).

branches: 1.24.2;
- rename variables to avoid shadowing.
- add a few const.

- Define a VFS_ATTACH() macro that places a reference to a vfsops structure
into the "vfsops" link set.
- Use VFS_ATTACH() where vfsops are declared for individual file systems.
- In vfsinit(), traverse the "vfsops" link set, rather than vfs_list_initial[].

branches: 1.22.2;
Rework the mountroot interface so that vfs_mountroot() opens the root device
and just passes it on to the file system functions. This avoids opening and
closing the device several times.

Mentioned on tech-kern some time ago, IIRC. I've been running this for a
long time.

Add the system call and VFS infrastructure for file system extended
attributes.

From FreeBSD.

set mp->mnt_stat.f_namemax on filesystem mount for use by statvfs, and use
the value for also pathconf(_PC_NAME_MAX) to keep the two in sync

set gmtoff in the structure returned by MNT_GETARGS call

Add ffs internal snapshots. Written by Marshall Kirk McKusick for FreeBSD.

- Not enabled by default. Needs kernel option FFS_SNAPSHOT.
- Change parameters of ffs_blkfree.
- Let the copy-on-write functions return an error so spec_strategy
may fail if the copy-on-write fails.
- Change genfs_*lock*() to use vp->v_vnlock instead of &vp->v_lock.
- Add flag B_METAONLY to VOP_BALLOC to return indirect block buffer.
- Add a function ffs_checkfreefile needed for snapshot creation.
- Add special handling of snapshot files:
Snapshots may not be opened for writing and the attributes are read-only.
Use the mtime as the time this snapshot was taken.
Deny mtime updates for snapshot files.
- Add function transferlockers to transfer any waiting processes from
one lock to another.
- Add vfsop VFS_SNAPSHOT to take a snapshot and make it accessible through
a vnode.
- Add snapshot support to ls, fsck_ffs and dump.

Welcome to 2.0F.

Approved by: Jason R. Thorpe <thorpej@netbsd.org>

Sysctl descriptions under vfs subtree

First pass for some caddr_t removal and changes to get rid of it where we
no longer use and/or need it

- removed casts from unionfs, deadfs and fdesc
(there are more to hunt down still)
- changed vfs_quotactl args argumet from caddr_t to void *
- changed vfs_quotactl structures/callers to reflect the api change

Compiled fine and ran for about a day. Approved/reviewed by
christos@netbsd.org and gimpy@netbsd.org.

initialize gmtoff in mount structure if mount version is < 3

Replace the statfs() family of system calls with statvfs().
Retain binary compatibility.

branches: 1.13.2;
Tango on sysctl_createv() and flags. The flags have all been renamed,
and sysctl_createv() now uses more arguments.

Dynamic sysctl.

Gone are the old kern_sysctl(), cpu_sysctl(), hw_sysctl(),
vfs_sysctl(), etc, routines, along with sysctl_int() et al. Now all
nodes are registered with the tree, and nodes can be added (or
removed) easily, and I/O to and from the tree is handled generically.

Since the nodes are registered with the tree, the mapping from name to
number (and back again) can now be discovered, instead of having to be
hard coded. Adding new nodes to the tree is likewise much simpler --
the new infrastructure handles almost all the work for simple types,
and just about anything else can be done with a small helper function.

All existing nodes are where they were before (numerically speaking),
so all existing consumers of sysctl information should notice no
difference.

PS - I'm sorry, but there's a distinct lack of documentation at the
moment. I'm working on sysctl(3/8/9) right now, and I promise to
watch out for buses.

add mnt_iflag field to struct mount for internal flags
mv MNT_GONE, MNT_UNMOUNT and MNT_WANTRDWR to this field
additonally add mnt_writeopcountupper and mnt_writeopcountlower fields
in preparation for pending write suspension support work
bump kernel version to 1.6ZD

Consistently use "bsBPB" as the struct member for the BIOS parameter block.

add -t option for gmt time offset (normally MS-DOS filesystem has timestamp
in localtime, not GMT). PR kern/22717

Allow separate masks for files and directories. Useful e.g. to turn
the execute bit off for files, but keep search permission for directories.
Change contributed in PR kern/21538 by Pavel Arnost, based on some FreeBSD
patches.
Further manpage changes, and backward-compatibility adjustments done by me.

Also fixes PR kern/16778 by Johan Danielsson, and PR kern/3400 by Rick Byers

branches: 1.7.2;
Back out the lwp/ktrace changes. They contained a lot of colateral damage,
and need to be examined and discussed more.

Undo part of the ktrace/lwp changes. In particular:
* Remove the "lwp *" argument that was added to vget(). Turns out
that nothing actually used it!
* Remove the "lwp *" arguments that were added to VFS_ROOT(), VFS_VGET(),
and VFS_FHTOVP(); all they did was pass it to vget() (which, as noted
above, didn't use it).
* Remove all of the "lwp *" arguments to internal functions that were added
just to appease the above.

Pass lwp pointers throughtout the kernel, as required, so that the lwpid can
be inserted into ktrace records. The general change has been to replace
"struct proc *" with "struct lwp *" in various function prototypes, pass
the lwp through and use l_proc to get the process pointer when needed.

Bump the kernel rev up to 1.6V

PR/1796: John Kohl: statfs misbehaves under chrooted environments.

- Under chroot it displays only the visible filesystems with appropriate paths.
- The statfs f_mntonname gets adjusted to contain the real path from root.
- While was there, fixed a bug in ext2fs, locking problems with vfs_getfsstat(),
and factored out some of the vfsop statfs() code to copy_statfs_info(). This
fixes the problem where some filesystems forgot to set fsid.
- Made coda look more like a normal fs.

Use 'void *' instead of 'caddr_t' in prototypes of VOP_IOCTL, VOP_FCNTL
and VOP_ADVLOCK, delete casts from callers (and some to copyin/out).

Add extensible malloc types, adapted from FreeBSD. This turns
malloc types into a structure, a pointer to which is passed around,
instead of an int constant. Allow the limit to be adjusted when the
malloc type is defined, or with a function call, as suggested by
Jonathan Stone.

branches: 1.1.2;
move msdosfs code from sys/msdosfs to sys/fs/msdosfs

Move the comment intended to introduce msdosfs_fsync back above
msdosfs_fsync (it has been above the wrong function since 1996).
No functional change.

branches: 1.112.6;
fix the previous to not fail to include the extension in lookups.

copy deExtension into the final 3 bytes. previously, this was found
by having them next to each other in the containing structure, but as
separate strings.

thanks to miod for pointing this out.

when calling a function that needs more bytes than we have, create a
stack variable long enough and use that instead.

found by GCC 12.

Sanitize the symbol namespace. NFC.

Move msdosfs_rename() and doscheckpath() to new file msdosfs_rename.c.

No functional change.

Factor out the lookup results from struct denode.

No functional change.

Overhaul of the EVFILT_VNODE kevent(2) filter:

- Centralize vnode kevent handling in the VOP_*() wrappers, rather than
forcing each individual file system to deal with it (except VOP_RENAME(),
because VOP_RENAME() is a mess and we currently have 2 different ways
of handling it; at least it's reasonably well-centralized in the "new"
way).
- Add support for NOTE_OPEN, NOTE_CLOSE, NOTE_CLOSE_WRITE, and NOTE_READ,
compatible with the same events in FreeBSD.
- Track which kevent notifications clients are interested in receiving
to avoid doing work for events no one cares about (avoiding, e.g.
taking locks and traversing the klist to send a NOTE_WRITE when
someone is merely watching for a file to be deleted, for example).

In support of the above:

- Add support in vnode_if.sh for specifying PRE- and POST-op handlers,
to be invoked before and after vop_pre() and vop_post(), respectively.
Basic idea from FreeBSD, but implemented differently.
- Add support in vnode_if.sh for specifying CONTEXT fields in the
vop_*_args structures. These context fields are used to convey information
between the file system VOP function and the VOP wrapper, but do not
occupy an argument slot in the VOP_*() call itself. These context fields
are initialized and subsequently interpreted by PRE- and POST-op handlers.
- Version VOP_REMOVE(), uses the a context field for the file system to report
back the resulting link count of the target vnode. Return this in tmpfs,
udf, nfs, chfs, ext2fs, lfs, and ufs.

NetBSD 9.99.92.

Abolish all the silly indirection macros for initializing vnode ops tables.

These are things of the form #define foofs_op genfs_op, or #define
foofs_op genfs_eopnotsupp, or similar. They serve no purpose besides
obfuscation, and have gotten cutpasted all over everywhere.

- Add a new vnode op: VOP_PARSEPATH.
- Move namei_getcomponent to genfs_vnops.c and call it genfs_parsepath.
- Add a parsepath entry to every vnode ops table.

VOP_PARSEPATH takes a directory vnode to be searched and a complete
following path and chooses how much of that path to consume. To begin
with, all parsepath calls are genfs_parsepath, which locates the first
'/' as always.

Note that the call doesn't take the whole struct componentname, only
the string. The other bits of struct componentname should not be
needed and there's no reason to cause potential complications by
exposing them.

branches: 1.104.6;
Introduce genfs_pathconf() and use it for the default case in all filesystems.

Add ACL support for FFS. From FreeBSD.

PR kern/54759 (vm.ubc_direct deadlock when read()/write() into mapping of itself)

- Add new flag UBC_ISMAPPED which tells ubc_uiomove() the object is mmap()ed
somewhere. Use it to decide whether to do direct-mapped copy, rather than
poking around directly in the vnode in ubc_uiomove(), which is ugly and
doesn't work for tmpfs. It would be nicer to contain all this in UVM but
the filesystem provides the needed locking here (VV_MAPPED) and to
reinvent that would suck more.

- Rename UBC_UNMAP_FLAG() to UBC_VNODE_FLAGS(). Pass in UBC_ISMAPPED where
appropriate.

Replace most uses of vp->v_usecount with a call to vrefcnt(vp), a function
that hides the details and does atomic_load_relaxed(). Signature matches
FreeBSD.

branches: 1.100.4;
UVM locking changes, proposed on tech-kern:

- Change the lock on uvm_object, vm_amap and vm_anon to be a RW lock.
- Break v_interlock and vmobjlock apart. v_interlock remains a mutex.
- Do partial PV list locking in the x86 pmap. Others to follow later.

branches: 1.99.2;
Add newly created vnodes to the namei cache. The rest of the filesystems
already did that (or they don't support writing). Discussed in tech-kern.

branches: 1.98.12;
Change VOP_REMOVE and VOP_RMDIR to preserve lock/ref on dvp.

No change to vp -- the plan is to replace the node by the
componentname in the vop parameters, and let all directory vops do
lookups internally.

Proposed on tech-kern with no objections:
https://mail-index.netbsd.org/tech-kern/2017/04/17/msg021825.html

Remove now redundant calls to fstrans_start()/fstrans_done().

branches: 1.96.2; 1.96.4;
- split a long line.
- remove extra test.
- move d_namlen setting to msdosfs_vnops.c to avoid the ifdef.

We can't depend on dp->d_namlen existing for the parts that are used in
makefs(8).

Add support to msdosfs and makefs to generate correct Unicode (UCS-2) directory
entries from UTF8 encoded file names.

Fix remaining read(fd, NULL, 1) cases.

Remove the 'cred' argument from bread(). Remove a now unused var in
ffs_snapshot.c. Update the man page accordingly.

ok hannken@

branches: 1.91.4;
Add VOP_FALLOCATE and VOP_FDISCARD to every vnode ops table I can
find.

The filesystem ones all call genfs_eopnotsupp - right now I am only
implementing the plumbing and we can implement fallocate and/or
fdiscard for files later.

The device ones call spec_fallocate (which is also genfs_eopnotsupp)
and spec_fdiscard, which dispatches to the device-level op.

The fifo ones all call vn_fifo_bypass, which also ends up being
EOPNOTSUPP.

Change msdosfs from hashlist to vcache:
- Use (dir_cluster, dir_offset, dir_generation) as key, where
dir_generation is non-zero and unique for unlinked but open nodes.
- Change deget() to return a vnode as it is unsafe to return a
referenced but unlocked denode.

branches: 1.89.2;
Change vnode operations create, mknod, mkdir and symlink to return
the resulting vnode *vpp unlocked.

Discussed on tech-kern@

Welcome to 6.99.30

Change vnode operations create, mknod, mkdir and symlink to keep the
directory node dvp locked on return.

Discussed on tech-kern@

Welcome to 6.99.29

Stop using v_mount of an unreferenced vnode -- save the mount while
the vnode has a reference.

branches: 1.86.6;
C99 section 6.7.2.3 (Tags) Note 3 states that:

A type specifier of the form

enum identifier

without an enumerator list shall only appear after the type it
specifies is complete.

which means that we cannot pass an "enum vtype" argument to
kauth_access_action() without fully specifying the type first.
Unfortunately there is a complicated include file loop which
makes that difficult, so convert this minimal function into a
macro (and capitalize it).

(ok elad@)

Move the initialization of n to after the error branch.

From Taylor R Campbell <riastradh@netbsd.org>

Change bread() and breadn() to never return a buffer on
error and modify all callers to not brelse() on error.

Welcome to 6.99.16

PR kern/46282 (6.0_BETA crash: msdosfs_bmap -> pcbmap -> bread -> bio_doread)

branches: 1.83.2;
change vflushbuf() to take the full FSYNC_* flags.
translate FSYNC_LAZY into PGO_LAZY for VOP_PUTPAGES() so that
genfs_do_io() can set the appropriate io priority for the I/O.
this is the first part of addressing PR 46325.

Add missing braces in previous commit.

Report the SF_ARCHIVED file flag if set.

Replace the remaining KAUTH_GENERIC_ISSUSER authorization calls with
something meaningful. All relevant documentation has been updated or
written.

Most of these changes were brought up in the following messages:

http://mail-index.netbsd.org/tech-kern/2012/01/18/msg012490.html
http://mail-index.netbsd.org/tech-kern/2012/01/19/msg012502.html
http://mail-index.netbsd.org/tech-kern/2012/02/17/msg012728.html

Thanks to christos, manu, njoly, and jmmv for input.

Huge thanks to pgoyette for spinning these changes through some build
cycles and ATF.

branches: 1.79.2; 1.79.4;
Add missing fstrans_done().

Should fix PR #45635 (KASSERT "fli->fli_trans_cnt == 0" failed)

branches: 1.78.2;
Even though msdosfs never truncates file names it advertises _PC_NO_TRUNC
as zero. Make it advertise one (no_trunc == true).

Names longer than NAME_MAX (255) will never pass namei() btw.

Fixes PR #43670 (msdosfs claims support for filenames longer than {NAME_MAX},
but fails)

Rename uvm_vnp_zerorange(struct vnode *, off_t, size_t) to
ubc_zerorange(struct uvm_object *, off_t, size_t, int) changing
the first argument to an uvm_object and adding a flags argument.

Modify tmpfs_reg_resize() to zero the backing store (aobj) instead
of the vnode. Ubc_purge() no longer panics when unmounting tmpfs.

Keep uvm_vnp_zerorange() until the next kernel version bump.

Welcome to 5.99.53! Merge rmind-uvmplock branch:

- Reorganize locking in UVM and provide extra serialisation for pmap(9).
New lock order: [vmpage-owner-lock] -> pmap-lock.

- Simplify locking in some pmap(9) modules by removing P->V locking.

- Use lock object on vmobjlock (and thus vnode_t::v_interlock) to share
the locks amongst UVM objects where necessary (tmpfs, layerfs, unionfs).

- Rewrite and optimise x86 TLB shootdown code, make it simpler and cleaner.
Add TLBSTATS option for x86 to collect statistics about TLB shootdowns.

- Unify /dev/mem et al in MI code and provide required locking (removes
kernel-lock on some ports). Also, avoid cache-aliasing issues.

Thanks to Andrew Doran and Joerg Sonnenberger, as their initial patches
formed the core changes of this branch.

branches: 1.75.2;
Change vflushbuf() to return an error if a synchronous write fails.

Welcome to 5.99.51.

When extending a file, either by truncating or by writing past EOF make
sure the unallocated remainder of the last page gets zeroed.

Detected by fsx.

Remove a vnode reference leak from msdosfs_rename. Release tdvp if either
doscheckpath() or relookup() fails.

Adjust test fs/vfs/t_vnops.c and remove the link count test for msdos.

Fixes PR #44661

In rename, use char[12]�for new names instead of [11]. At least
one routine called from here (unix2dosfn) expects and uses all of
a [12].

This may fix the "stack size exceeded" problem which has been
triggering in gson's test runs. (i'm not entirely sure why it
doesn't trigger in anyone else's env)

branches: 1.71.2; 1.71.4;
Remove the special refcount behavior (adding an extra reference to the
parent dir) associated with SAVESTART in relookup().

Check all call sites to make sure that SAVESTART wasn't set while
calling relookup(); if it was, adjust the refcount behavior. Remove
related references to SAVESTART.

The only code that was reaching the extra ref was msdosfs_rename,
where the refcount behavior was already fairly broken and/or gross;
repair it.

Add a dummy 4th argument to relookup to make sure code that hasn't
been inspected won't compile. (This will go away next time the
relookup semantics change, which they will.)

Abolish the SAVENAME and HASBUF flags. There is now always a buffer,
so the path in a struct componentname is now always valid during VOP
calls.

Abolish struct componentname's cn_pnbuf. Use the path buffer in the
pathbuf object passed to namei as work space instead. (For now a pnbuf
pointer appears in struct nameidata, to support certain unclean things
that haven't been fixed yet, but it will be going away in the future.)

This removes the need for the SAVENAME and HASBUF namei flags.

Remove bad cast, fix compilation with MSDOSFS_DEBUG.

Clean up vnode lock operations pass 2:

VOP_UNLOCK(vp, flags) -> VOP_UNLOCK(vp): Remove the unneeded flags argument.

Welcome to 5.99.32.

Discussed on tech-kern.

Don't double unlock fvp if source file disappears during rename.
Problem found by njoly's awesome stresstester.

Enforce RLIMIT_FSIZE before VOP_WRITE. This adds support to file
system drivers where it was missing from and fixes one buggy
implementation. The arguably weird semantics of the check are
maintained (v_size vs. va_bytes, overwrite).

Add file system suspension support (vfs_suspend(9)) to msdosfs.

Reviewed by: Antti Kantee <pooka@netbsd.org>

In genfs where available.

The only functional change is mknod now returning EOPNOTSUPP instead
of EINVAL. I make this sacrifice willingly and with a clean conscience.

Use genfs instead of homegrown stuff where possible.

branches: 1.61.2; 1.61.4;
Where possible, extract the file-system's access() routine to two internal
functions: the first checking if the operation is possible (regardless of
permissions), the second checking file-system permissions, ACLs, etc.

Mailing list reference:

http://mail-index.netbsd.org/tech-kern/2009/06/21/msg005311.html

Move the implementation of vaccess() to genfs_can_access(), in line with
the other routines of the same spirit.

Adjust file-system code to use it.

Keep vaccess() for KPI compatibility and to keep element of least
surprise. A "diagnostic" message warning that vaccess() is deprecated will
be printed when it's used (obviously, only in DIAGNOSTIC kernels).

No objections on tech-kern@:

http://mail-index.netbsd.org/tech-kern/2009/06/21/msg005310.html

Extract the open-coded authorization logic for chtimes() from various
file-systems and put it in a single function, genfs_can_chtimes().

This also makes UDF follow the same policy as all other file-systems.

Mailing list reference:

http://mail-index.netbsd.org/tech-kern/2009/04/27/msg004951.html

ANSIfy another 1261 function definitions.
The only ones left in sys are beyond by sed script!
(or in sys/dist or sys/external)
Mostly they have function pointer parameters.

branches: 1.57.2;
make this compile

merge christos-time_t

Tyop: v -> vp. Spotted by Tom's ARM builds.

Rototill all remaining file systems to use ubc_uiomove() instead
of the ubc_alloc() - uiomove() - ubc_release() dance.

In case writing past EOF, fail if extending the file fails (most
likely due to the file system being full).

Otherwise we'd fail in VOP_PUTPAGES(), which might not happen during
VOP_WRITE(), thus giving the caller the wrong impression that
writing was succesful.

branches: 1.52.4; 1.52.6; 1.52.10;
Make sure all cached buffers with valid, not yet written data have been
run through copy-on-write. Call fscow_run() with valid data where possible.

The LP_UFSCOW hack is no longer needed to protect ffs_copyonwrite() against
endless recursion.

- Add a flag B_MODIFY to bread(), breada() and breadn(). If set the caller
intends to modify the buffer returned.

- Always run copy-on-write on buffers returned from ffs_balloc().

- Add new function ffs_getblk() that gets a buffer, assigns a new blkno,
may clear the buffer and runs copy-on-write. Process possible errors
from getblk() or fscow_run(). Part of PR kern/38664.

Welcome to 4.99.63

Reviewed by: YAMAMOTO Takashi <yamt@netbsd.org>

branches: 1.51.2;
Make various bits of debug code compile again.

branches: 1.50.2;
Network protocol interrupts can now block on locks, so merge the globals
proclist_mutex and proclist_lock into a single adaptive mutex (proc_lock).
Implications:

- Inspecting process state requires thread context, so signals can no longer
be sent from a hardware interrupt handler. Signal activity must be
deferred to a soft interrupt or kthread.

- As the proc state locking is simplified, it's now safe to take exit()
and wait() out from under kernel_lock.

- The system spends less time at IPL_SCHED, and there is less lock activity.

branches: 1.49.6; 1.49.8;
Replace struct lock on vnodes with a simpler lock object built on
krwlock_t. This is a step towards removing lockmgr and simplifying
vnode locking. Discussed on tech-kern.

Remove VOP_LEASE. Discussed on tech-kern.

PR/37696: Paul Ripke: add large read / readahead support (Introduce run-length
detection to msdosfs_bmap(), from FreeBSD)

Merge vmlocking2 to head.

Use uvm_vnp_setwritesize() on file extending to instruct UVM to NOT read-in
the newly added space first. This significantly speeds up write speed for
msdosfs and making it at par with ffs wich already had this patched.

Speed increase measured on my IDE disc from 2Mb/sec to 32 Mb/sec

branches: 1.44.2; 1.44.6;
Remove the "struct lwp *" argument from all VFS and VOP interfaces.
The general trend is to remove it from all kernel interfaces and
this is a start. In case the calling lwp is desired, curlwp should
be used.

quick consensus on tech-kern

Fix a problem noticed by Reinoud: the fs would try to release an
unlocked vnode when trying to rename a directory. The fix was to
shuffle some bits around and #pray.

The rename routine actually needs a very very major wide-angle whopping:
* it takes locks out-of-order
* it deals with references from SAVESTART lookups in interesting ways
* I doubt there is any guarantee for correct operation if there
are multiple concurrent accesses
* the error branches might just as well call panic() directly

branches: 1.42.2; 1.42.4;
Merge brelse() changes from the vmlocking branch.

Avoid stack allocation of large dirent structures in foo_readdir().

branches: 1.40.4; 1.40.6; 1.40.8; 1.40.10;
Use _DIRENT_MINSIZE when determining the number of NFS cookies to allocate,
rather than hard-coding 16.

branches: 1.39.2;
hold proclist_mutex when calling psignal().

branches: 1.38.2; 1.38.4;
Kill caddr_t; there will be some MI fallout, but it will be fixed shortly.

Replace the Mach-derived boolean_t type with the C99 bool type. A
future commit will replace use of TRUE and FALSE with true and false.

branches: 1.36.2;
Consistent usage of KAUTH_GENERIC_ISSUSER.

a smorgasbord of improvements to vnode locking and path lookup:
- LOCKPARENT is no longer relevant for lookup(), relookup() or VOP_LOOKUP().
these now always return the parent vnode locked. namei() works as before.
lookup() and various other paths no longer acquire vnode locks in the
wrong order via vrele(). fixes PR 32535.
as a nice side effect, path lookup is also up to 25% faster.
- the above allows us to get rid of PDIRUNLOCK.
- also get rid of WANTPARENT (just use LOCKPARENT and unlock it).
- remove an assumption in layer_node_find() that all file systems implement
a recursive VOP_LOCK() (unionfs doesn't).
- require that all file systems supply vfs_vptofh and vfs_fhtovp routines.
fill in eopnotsupp() for file systems that don't support being exported
and remove the checks for NULL. (layerfs calls these without checking.)
- in union_lookup1(), don't change refcounts in the ISDOTDOT case, just
adjust which vnode is locked. fixes PR 33374.
- apply fixes for ufs_rename() from ufs_vnops.c rev. 1.61 to ext2fs_rename().

branches: 1.34.2;
Support FAT filesystems on non-DEV_BSIZE media.

Based on the patches provided in PR kern/17398 by Trevin Beattie.

__unused removal on arguments; approved by core.

- sprinkle __unused on function decls.
- fix a couple of unused bugs
- no more -Wno-unused for i386

Apply patch from PR kern/34583 sent by Rhialto, quoting him:

"Add a 3rd entry in the cache, which keeps the end position
from just before extending a file.
This has the desired effect of keeping the write speed constant."

And yes, that helps a lot copying large files... always at full speed
now. This closes my PR kern/30868 "Poor performance copying large files
on msdosfs".

Also remove a 2 if-statements testing the same condition, combine them.

All that from Rhialto, thank you very much.

branches: 1.30.2; 1.30.4; 1.30.6;
Use the LWP cached credentials where sane.

merge FreeBSD timecounters from branch simonb-timecounters
- struct timeval time is gone
time.tv_sec -> time_second
- struct timeval mono_time is gone
mono_time.tv_sec -> time_uptime
- access to time via
{get,}{micro,nano,bin}time()
get* versions are fast but less precise
- support NTP nanokernel implementation (NTP API 4)
- further reading:
Timecounter Paper: http://phk.freebsd.dk/pubs/timecounter.pdf
NTP Nanokernel: http://www.eecis.udel.edu/~mills/ntp/html/kern.html

branches: 1.28.2;
integrate kauth.

Coveriry CID 733: Remove dead code.

branches: 1.26.2; 1.26.4; 1.26.6;
merge yamt-uio_vmspace branch.

- use vmspace rather than proc or lwp where appropriate.
the latter is more natural to specify an address space.
(and less likely to be abused for random purposes.)
- fix a swdmover race.

branches: 1.25.2; 1.25.4;
Protect against uio_lwp being NULL from Pavel Cahyna

branches: 1.24.2;
merge ktrace-lwp.

merge yamt-readahead branch.

branches: 1.22.2;
PR/32003: Brian Buhrow: msdosfs doesn't properly zero out high cluster data
on non-FAT32 msdos filesystems.

merge yamt-vop branch. remove following VOPs.

VOP_BLKATOFF
VOP_VALLOC
VOP_BALLOC
VOP_REALLOCBLKS
VOP_VFREE
VOP_TRUNCATE
VOP_UPDATE

branches: 1.20.2;
Use nanotime() to update the time fields in filesystems. Convert the code
from macros to real functions. Original patch and review from chuq.
Note: ext2fs only keeps seconds in the on-disk inode, and msdosfs does not
have enough precision for all fields, so this is not very useful for those
two.

Make fileno calculations use ino_t variables.

Remove __P()

The maximum file size on MS-DOS filesystems is 4 GB - 1 byte, so
don't bother trying to write files bigger than this. Just return
EFBIG to caller, rather than panic()ing later.

From OpenBSD.

This closes my PR kern/30864: "panic when copying files of >4GB on msdosfs"

fileid is now a quad.

64 bit inode changes.

branches: 1.14.2; 1.14.4;
nuke trailing whitespace

branches: 1.13.2; 1.13.4;
adjust the UBC mapping code to support non-vnode uvm_objects.
this means we can no longer look at the vnode size to determine how many
pages to request in a fault, which is good since for NFS the size can change
out from under us on the server anyway. there's also a new flag UBC_UNMAP
for ubc_release(), so that the file system code can make the decision about
whether to cache mappings for files being used as executables.

set mp->mnt_stat.f_namemax on filesystem mount for use by statvfs, and use
the value for also pathconf(_PC_NAME_MAX) to keep the two in sync

caddr_t -> void * and removal of some more casts.

Remove caddr_t inlines. Approved by christos@NetBSD.org.

branches: 1.9.2; 1.9.4;
Fix xxx_strategy() to use the vnode arg instead of bp->b_vp.

Make VOP_STRATEGY(bp) a real VOP as discussed on tech-kern.

VOP_STRATEGY(bp) is replaced by one of two new functions:

- VOP_STRATEGY(vp, bp) Call the strategy routine of vp for bp.
- DEV_STRATEGY(bp) Call the d_strategy routine of bp->b_dev for bp.

DEV_STRATEGY(bp) is used only for block-to-block device situations.

add -t option for gmt time offset (normally MS-DOS filesystem has timestamp
in localtime, not GMT). PR kern/22717

Allow separate masks for files and directories. Useful e.g. to turn
the execute bit off for files, but keep search permission for directories.
Change contributed in PR kern/21538 by Pavel Arnost, based on some FreeBSD
patches.
Further manpage changes, and backward-compatibility adjustments done by me.

Also fixes PR kern/16778 by Johan Danielsson, and PR kern/3400 by Rick Byers

branches: 1.5.2;
Back out the lwp/ktrace changes. They contained a lot of colateral damage,
and need to be examined and discussed more.

Pass lwp pointers throughtout the kernel, as required, so that the lwpid can
be inserted into ktrace records. The general change has been to replace
"struct proc *" with "struct lwp *" in various function prototypes, pass
the lwp through and use l_proc to get the process pointer when needed.

Bump the kernel rev up to 1.6V

strlcpy

make 'dosdirtemplate' static const

branches: 1.1.2;
move msdosfs code from sys/msdosfs to sys/fs/msdosfs

Add support to msdosfs and makefs to generate correct Unicode (UCS-2) directory
entries from UTF8 encoded file names.

branches: 1.20.4;
Change msdosfs from hashlist to vcache:
- Use (dir_cluster, dir_offset, dir_generation) as key, where
dir_generation is non-zero and unique for unlinked but open nodes.
- Change deget() to return a vnode as it is unsafe to return a
referenced but unlocked denode.

branches: 1.19.10;
more cross-compile friendly.

expose more stuff if MAKEFS is defined for the headers, and arrange for
the source file to be compilable from userland.

Stylistic changes in comments/strings:
"FAT" and "fat" are different things, use the appropriate case.
"GEMDOS" is all caps.

branches: 1.16.6;
Only use MALLOC_DECLARE if it exists. Helps with the dirty _KERNEL
tricks fstat is using.

branches: 1.15.30; 1.15.34;
Create sysctl entries during module initialisation and destroy them
appropriately.

Many of these file systems are now ready for modularisation.

branches: 1.14.18; 1.14.22; 1.14.24; 1.14.26;
Avoid stack allocation of large dirent structures in foo_readdir().

branches: 1.13.2; 1.13.14; 1.13.16; 1.13.18;
Kill caddr_t; there will be some MI fallout, but it will be fixed shortly.