Home | History | Annotate | Download | only in ffs
History log of /src/sys/ufs/ffs/ffs_wapbl.c
RevisionDateAuthorComments
 1.50  30-Dec-2024  hannken Protect test/clear fs->fs_fmod with um_lock like it is already
protected in ffs_alloc.c.

When writing to disk protect moving superblock to buffer with um_lock.

Set/clear fs->fmod while mounting, updating a mount or unmounting
is safe as these operations run exclusive, either mounting creates
a new file system or the file system is suspended. Assert suspension
for update and unmount.

PR kern/58837 "ffs: Missing locking around fs_fmod/time"
 1.49  13-May-2024  msaitoh s/contigous/contiguous/ in comment.
 1.48  22-May-2022  andvar branches: 1.48.4;
fix various small typos, mainly in comments.
 1.47  13-May-2022  reinoud Fix typo dallocate -> deallocate
 1.46  11-Apr-2020  jdolecek remove noncompilable WAPBL_DEBUG_INODES

PR kern/49554 by Thomas Klausner
 1.45  17-Jan-2020  ad branches: 1.45.4;
VFS_VGET(), VFS_ROOT(), VFS_FHTOVP(): give them a "int lktype" argument, to
allow us to get shared locks (or no lock) on the returned vnode. Matches
FreeBSD.
 1.44  01-Jan-2019  hannken branches: 1.44.4; 1.44.6;
Add "void *extra" argument to vcache_new() so a file system may
pass more information about the file to create.

Welcome to 8.99.30
 1.43  10-Dec-2018  jdolecek make UFS_WAPBL_JLOCK_ASSERT() #ifdef DIAGNOSTIC, same as the underlying
function KASSERT(), so that it actually does something; fix code using
it to actually pass correct params, so that it compiles

remove UFS_WAPBL_JUNLOCK_ASSERT(), as that is inherently racy (it's
okay on those places if the rwlock is held by other lwp); depend
on the RW_ASSERT()/LOCKDEBUG inside rw_enter() to catch the case
with wapbl rwlock held by current lwp
 1.42  03-Sep-2018  riastradh Rename min/max -> uimin/uimax for better honesty.

These functions are defined on unsigned int. The generic name
min/max should not silently truncate to 32 bits on 64-bit systems.
This is purely a name change -- no functional change intended.

HOWEVER! Some subsystems have

#define min(a, b) ((a) < (b) ? (a) : (b))
#define max(a, b) ((a) > (b) ? (a) : (b))

even though our standard name for that is MIN/MAX. Although these
may invite multiple evaluation bugs, these do _not_ cause integer
truncation.

To avoid `fixing' these cases, I first changed the name in libkern,
and then compile-tested every file where min/max occurred in order to
confirm that it failed -- and thus confirm that nothing shadowed
min/max -- before changing it.

I have left a handful of bootloaders that are too annoying to
compile-test, and some dead code:

cobalt ews4800mips hp300 hppa ia64 luna68k vax
acorn32/if_ie.c (not included in any kernels)
macppc/if_gm.c (superseded by gem(4))

It should be easy to fix the fallout once identified -- this way of
doing things fails safe, and the goal here, after all, is to _avoid_
silent integer truncations, not introduce them.

Maybe one day we can reintroduce min/max as type-generic things that
never silently truncate. But we should avoid doing that for a while,
so that existing code has a chance to be detected by the compiler for
conversion to uimin/uimax without changing the semantics until we can
properly audit it all. (Who knows, maybe in some cases integer
truncation is actually intended!)
 1.41  28-May-2017  hannken branches: 1.41.8; 1.41.10;
No need to call vgone() on the just created in file system log vnode,
vput() is sufficient.
 1.40  22-Mar-2017  jdolecek move the ffs_sync() after wapbl_log_position() call, since that can still
create delayed writes with MNT_ASYNC when log is created
 1.39  16-Mar-2017  jdolecek need to turn off async during ffs_sync(), otherwise its bwrite() calls are
themselves turned to bdwrite(), creating dirty delayed writes

fixes panic for 'mount -o log,async ...' reported by Masanobu SAITOH
on current-users; fix help by hannken@, thank you
 1.38  10-Mar-2017  jdolecek sync any delayed writes when updating filesystem to log

Adresses PR kern/52056 by Martin Husemann, fix helped by Juergen Hannken, thanks
 1.37  10-Nov-2016  jdolecek branches: 1.37.2;
disable discard when log is enabled to preserve log consistency promise

PR kern/50725
 1.36  10-Nov-2016  jdolecek during truncate with wapbl, register deallocation for upper indirect block
before recursing into lower blocks, to make sure that it will be removed after
all its referenced blocks are removed

fixes 'ffs_blkfree_common: freeing free block' panic triggered by
ufs_truncate_retry() when just the upper indirect block registration failed,
code tried to free the lower blocks again after wapbl flush

problem found by hannken@, thank you
 1.35  02-Oct-2016  christos use __func__ and print the filesystem we are printing the message for.
 1.34  01-Oct-2016  jdolecek allocate wapbl dealloc registration structures via pool, so that there is more
flexibility with limit handling
 1.33  01-Oct-2016  jdolecek wapbl_remove_log(): add missing break; harmless, fallthrough just printed
extra debug message
 1.32  24-Sep-2016  jdolecek fix swapped KASSERT()
 1.31  24-Sep-2016  jdolecek i/o optimization for wapbl flush - only sync superblock and cgs when
they were actually changed
 1.30  28-Mar-2015  maxv branches: 1.30.2;
Remove the 'cred' argument from bread(). Remove a now unused var in
ffs_snapshot.c. Update the man page accordingly.

ok hannken@
 1.29  17-Mar-2015  hannken Change ffs to use vcache_new:
- Change ffs_valloc to return an inode number.
- Remove now obsolete UFS operations UFS_VALLOC and UFS_VFREE.
- Make ufs_makeinode private to ufs_vnops.c and pass vattr instead of mode.
 1.28  11-Jul-2014  christos branches: 1.28.4;
move the flag setting higher to avoid KASSERT (dholland)
 1.27  10-Jul-2014  christos CID 975226: hande error from UFS_WAPBL_BEGIN
 1.26  10-Jul-2014  dholland Fix unchecked UFS_WAPBL_BEGIN. Coverity 975226.
Unfortunately it looks like all we can do on error here is printf.
 1.25  25-Oct-2013  martin branches: 1.25.2;
Turn a few __unused into __diagused
 1.24  20-Oct-2013  htodd Definining needswap where needed.
 1.23  19-Oct-2013  martin Mark a potentially unused variable
 1.22  23-Jun-2013  dholland branches: 1.22.2;
Stick ffs_ in front of the following macros:
fragstoblks()
blkstofrags()
fragnum()
blknum()

to finish the job of distinguishing them from the lfs versions, which
Christos renamed the other day.

I believe this is the last of the overtly ambiguous exported symbols
from ffs... or at least, the last of the ones that conflicted with lfs.
ffs still pollutes the C namespace very broadly (as does ufs) and this
needs quite a bit more cleanup.

XXX: boo on macros with lowercase names. But I'm not tackling that just yet.
 1.21  23-Jun-2013  dholland Stick ffs_, ext2_, chfs_, filecore_, cd9660_, or mfs_ in front of
the following symbols so as to disambiguate fully. (Christos already
did the lfs ones.)

lblkno
lblktosize
lfragtosize
numfrags
blkroundup
fragroundup
 1.20  23-Jun-2013  dholland fsbtodb() -> FFS_FSBTODB(), EXT2_FSBTODB(), or MFS_FSBTODB()
dbtofsb() -> FFS_DBTOFSB() or EXT2_DBTOFSB()

(Christos already did the lfs ones a few days back)
 1.19  22-Jan-2013  dholland Stuff UFS_ in front of a few of ufs's symbols to reduce namespace
pollution. Specifically:
ROOTINO -> UFS_ROOTINO
WINO -> UFS_WINO
NXADDR -> UFS_NXADDR
NDADDR -> UFS_NDADDR
NIADDR -> UFS_NIADDR
MAXSYMLINKLEN -> UFS_MAXSYMLINKLEN
MAXSYMLINKLEN_UFS[12] -> UFS[12]_MAXSYMLINKLEN (for consistency)

Sort out ext2fs's misuse of NDADDR and NIADDR; fortunately, these have
the same values in ext2fs and ffs.

No functional change intended.
 1.18  20-Dec-2012  hannken Change bread() and breadn() to never return a buffer on
error and modify all callers to not brelse() on error.

Welcome to 6.99.16

PR kern/46282 (6.0_BETA crash: msdosfs_bmap -> pcbmap -> bread -> bio_doread)
 1.17  24-Dec-2010  mlelstv branches: 1.17.8; 1.17.18;
For update mounts the root vnode is already in use and we must not
free it. Since the mount persists even when the update fails,
this is not a problem either.
 1.16  23-Dec-2010  mlelstv mount(2) doesn't remove vnodes from the freelist in the error path,
so that they get reused with a invalid pointer to a mount structure.

As a workaround, free the vnodes used to create the in-filesystem journal
immediately.
 1.15  27-Feb-2010  mlelstv branches: 1.15.2;
Store physical block numbers in superblock that point to the journal.
Calculate position of both commit headers correctly for disks with
large sectors.
Correct calculation of circular buffer size.
 1.14  23-Feb-2010  mlelstv Replace individual queries for partition information with
new helper function.
Use this information to query physical sector sizes for WAPBL
instead of hardcoded defaults.
No longer limits physical sector sizes to 512 bytes.
 1.13  13-Sep-2009  bouyer branches: 1.13.2;
Allow tunefs to clear any type of WAPBL log, not only in-filesystem
ones. Discussed in
http://mail-index.netbsd.org/tech-kern/2009/08/17/msg005896.html
and followups.
 1.12  22-Feb-2009  ad branches: 1.12.2;
PR kern/26878 FFSv2 + softdep = livelock (no free ram)
PR kern/16942 panic with softdep and quotas
PR kern/19565 panic: softdep_write_inodeblock: indirect pointer #1 mismatch
PR kern/26274 softdep panic: allocdirect_merge: ...
PR kern/26374 Long delay before non-root users can write to softdep partitions
PR kern/28621 1.6.x "vp != NULL" panic in ffs_softdep.c:4653 while unmounting a softdep (+quota) filesystem
PR kern/29513 FFS+Softdep panic with unfsck-able file-corruption
PR kern/31544 The ffs softdep code appears to fail to write dirty bits to disk
PR kern/31981 stopping scsi disk can cause panic (softdep)
PR kern/32116 kernel panic in softdep (assertion failure)
PR kern/32532 softdep_trackbufs deadlock
PR kern/37191 softdep: locking against myself
PR kern/40474 Kernel panic after remounting raid root with softdep

Retire softdep, pass 2. As discussed and later formally announced on the
mailing lists.
 1.11  31-Jan-2009  yamt branches: 1.11.2;
0 -> NULL
 1.10  31-Jan-2009  yamt wapbl_log_position: 1 -> MNT_WAIT
 1.9  30-Nov-2008  joerg Split ffs_blkalloc into a frontend that does inode based consistency
checks and a backend that just asserts them. Use the backend in
ffs_wapbl_abort_sync_metadata instead of faking an inode.
 1.8  11-Nov-2008  joerg wapbl_replay_free needs the reply to have been stopped, so make sure
that the changes happen in the right order. Reported by veego@
 1.7  10-Nov-2008  joerg Reduce internals of WAPBL exposed to the rest of the system.
 1.6  08-Sep-2008  joerg branches: 1.6.2; 1.6.4; 1.6.6; 1.6.8; 1.6.12;
Move successful removal of unreferenced inodes under WAPBL_DEBUG to not
spam the console.

OK simon@
 1.5  05-Aug-2008  pooka zu, not zd, to print size_t
 1.4  04-Aug-2008  simonb Only allow WAPBL to operate with UFS2 style superblocks.

Problem reported by Takeshi Nakayama.
 1.3  02-Aug-2008  simonb When checking if there's enough space at the end of a partition,
compare bytes vs bytes, not sectors vs bytes.

Problem discovered and fix tested by Michael Hitch.
 1.2  31-Jul-2008  simonb Merge the simonb-wapbl branch. From the original branch commit:

Add Wasabi System's WAPBL (Write Ahead Physical Block Logging)
journaling code. Originally written by Darrin B. Jewell while
at Wasabi and updated to -current by Antti Kantee, Andy Doran,
Greg Oster and Simon Burge.

OK'd by core@, releng@.
 1.1  10-Jun-2008  simonb branches: 1.1.2; 1.1.4;
file ffs_wapbl.c was initially added on branch simonb-wapbl.
 1.1.4.2  13-Dec-2008  haad Update haad-dm branch to haad-dm-base2.
 1.1.4.1  19-Oct-2008  haad Sync with HEAD.
 1.1.2.4  28-Jul-2008  simonb Add support for creating a WAPBL log in the filesystem. Will
create an in-filesystem log on first "mount -o log" if one doesn't
exist, and will then continue to use same log in the future. See
(soon to be added) wapbl(4) for more info.

Adds a new B_CONTIG low-level allocation flag that uses hints in
"struct ffs_inode_ext" to lay out an ffs file's data contiguously.

Thanks to Greg Oster for helping with the design of this and to
Antti Kantee for code review and suggestions.
 1.1.2.3  03-Jul-2008  simonb Store the location of the journal in the superblock. Currently
nothing really uses this, other than replay checking that what is
in the superblock matches what it expects.
 1.1.2.2  12-Jun-2008  martin License police
 1.1.2.1  10-Jun-2008  simonb Initial commit of Wasabi System's WAPBL (Write Ahead Physical Block
Logging) journaling code. Originally written by Darrin B. Jewell
while at Wasabi and updated to -current by Antti Kantee, Andy Doran,
Greg Oster and Simon Burge.

Still a number of issues - look in doc/BRANCHES for "simonb-wapbl"
for more info.
 1.6.12.1  21-Apr-2010  matt sync to netbsd-5
 1.6.8.1  03-Oct-2009  snj Pull up following revision(s) (requested by bouyer in ticket #1036):
sbin/fsck_ffs/extern.h: revision 1.25 via patch
sbin/fsck_ffs/setup.c: revision 1.88 via patch
sbin/fsck_ffs/wapbl.c: revision 1.4 via patch
sbin/tunefs/tunefs.c: revision 1.41 via patch
sys/ufs/ffs/ffs_vfsops.c: revision 1.252 via patch
sys/ufs/ffs/ffs_wapbl.c: revision 1.13 via patch
Allow tunefs to clear any type of WAPBL log, not only in-filesystem
ones. Discussed in
http://mail-index.netbsd.org/tech-kern/2009/08/17/msg005896.html
and followups.
--
Do some basic checks of the WAPBL journal, to abort the boot before the
kernel refuse to mount a filesystem read-write (booting a system
multiuser with critical filesystems read-only is bad):
Add a check_wapbl() which will check some WAPBL values in the superblock,
and try to read the journal via wapbl_replay_start() if there is one.
pfatal() if one of these fail (abort boot if in preen mode,
as "CONTINUE" otherwise). In non-preen mode the bogus journal will
be cleared.
check_wapbl() is always called if the superblock supports WAPBL.
Even if FS_DOWAPBL is not there, there could be flags asking the
kernel to clear or create a log with bogus values which would cause the
kernel refuse to mount the filesystem.
Discussed in
http://mail-index.netbsd.org/tech-kern/2009/08/17/msg005896.html
and followups.
--
If the WAPBL journal can't be read (ffs_wapbl_replay_start() fails),
mount the filesystem anyway if MNT_FORCE is present.
This allows to still boot single-user a system with a corrupted
WAPBL on /, and so get a chance to run fsck to fix it.
http://mail-index.netbsd.org/tech-kern/2009/08/17/msg005896.html
and followups.
 1.6.6.2  03-Mar-2009  skrll Sync with HEAD.
 1.6.6.1  19-Jan-2009  skrll Sync with HEAD.
 1.6.4.3  17-Jan-2009  mjf Sync with HEAD.
 1.6.4.2  28-Sep-2008  mjf Sync with HEAD.
 1.6.4.1  08-Sep-2008  mjf file ffs_wapbl.c was added on branch mjf-devfs2 on 2008-09-28 10:41:06 +0000
 1.6.2.2  18-Sep-2008  wrstuden Sync with wrstuden-revivesa-base-2.
 1.6.2.1  08-Sep-2008  wrstuden file ffs_wapbl.c was added on branch wrstuden-revivesa on 2008-09-18 04:37:05 +0000
 1.11.2.1  13-May-2009  jym Sync with HEAD.

Commit is split, to avoid a "too many arguments" protocol error.
 1.12.2.4  11-Mar-2010  yamt sync with head
 1.12.2.3  16-Sep-2009  yamt sync with head
 1.12.2.2  04-May-2009  yamt sync with head.
 1.12.2.1  22-Feb-2009  yamt file ffs_wapbl.c was added on branch yamt-nfs-mp on 2009-05-04 08:14:38 +0000
 1.13.2.1  30-Apr-2010  uebayasi Sync with HEAD.
 1.15.2.1  05-Mar-2011  rmind sync with head
 1.17.18.3  03-Dec-2017  jdolecek update from HEAD
 1.17.18.2  20-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.17.18.1  25-Feb-2013  tls resync with head
 1.17.8.2  22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.17.8.1  23-Jan-2013  yamt sync with head
 1.22.2.1  18-May-2014  rmind sync with head
 1.25.2.1  10-Aug-2014  tls Rebase.
 1.28.4.4  28-Aug-2017  skrll Sync with HEAD
 1.28.4.3  05-Dec-2016  skrll Sync with HEAD
 1.28.4.2  05-Oct-2016  skrll Sync with HEAD
 1.28.4.1  06-Apr-2015  skrll Sync with HEAD
 1.30.2.4  26-Apr-2017  pgoyette Sync with HEAD
 1.30.2.3  20-Mar-2017  pgoyette Sync with HEAD
 1.30.2.2  07-Jan-2017  pgoyette Sync with HEAD. (Note that most of these changes are simply $NetBSD$
tag issues.)
 1.30.2.1  04-Nov-2016  pgoyette Sync with HEAD
 1.37.2.1  21-Apr-2017  bouyer Sync with HEAD
 1.41.10.3  21-Apr-2020  martin Sync with HEAD
 1.41.10.2  08-Apr-2020  martin Merge changes from current as of 20200406
 1.41.10.1  10-Jun-2019  christos Sync with HEAD
 1.41.8.3  18-Jan-2019  pgoyette Synch with HEAD
 1.41.8.2  26-Dec-2018  pgoyette Sync with HEAD, resolve a few conflicts
 1.41.8.1  06-Sep-2018  pgoyette Sync with HEAD

Resolve a couple of conflicts (result of the uimin/uimax changes)
 1.44.6.1  17-Jan-2020  ad Sync with head.
 1.44.4.1  07-Jan-2025  martin Pull up following revision(s) (requested by hannken in ticket #1934):

sys/ufs/ext2fs/ext2fs_vfsops.c: revision 1.228
sys/ufs/lfs/lfs_vfsops.c: revision 1.383
sys/ufs/ffs/ffs_wapbl.c: revision 1.50
sys/ufs/ffs/ffs_vfsops.c: revision 1.383 (patch)
sys/ufs/ffs/ffs_vfsops.c: revision 1.384 (patch)

Remove comment "we are always called with the filesystem marked `MPBUSY'."
above some xxx_sync() operations. These operations get called without
any exclusive lock.

This comment appeared with "add quota support" on 1990-05-02.
On 1998/02/18 MNT_MPBUSY disappeared when vfs_busy() was changed from
an exclusive lock to a shared lock.

PR kern/58837 "ffs: Missing locking around fs_fmod/time"

Protect test/clear fs->fs_fmod with um_lock like it is already
protected in ffs_alloc.c.

When writing to disk protect moving superblock to buffer with um_lock.

Set/clear fs->fmod while mounting, updating a mount or unmounting
is safe as these operations run exclusive, either mounting creates
a new file system or the file system is suspended. Assert suspension
for update and unmount.

PR kern/58837 "ffs: Missing locking around fs_fmod/time"
 1.45.4.1  20-Apr-2020  bouyer Sync with HEAD
 1.48.4.1  07-Jan-2025  martin Pull up following revision(s) (requested by hannken in ticket #1037):

sys/ufs/ext2fs/ext2fs_vfsops.c: revision 1.228
sys/ufs/lfs/lfs_vfsops.c: revision 1.383
sys/ufs/ffs/ffs_wapbl.c: revision 1.50
sys/ufs/ffs/ffs_vfsops.c: revision 1.383
sys/ufs/ffs/ffs_vfsops.c: revision 1.384

Remove comment "we are always called with the filesystem marked `MPBUSY'."
above some xxx_sync() operations. These operations get called without
any exclusive lock.

This comment appeared with "add quota support" on 1990-05-02.
On 1998/02/18 MNT_MPBUSY disappeared when vfs_busy() was changed from
an exclusive lock to a shared lock.

PR kern/58837 "ffs: Missing locking around fs_fmod/time"

Protect test/clear fs->fs_fmod with um_lock like it is already
protected in ffs_alloc.c.

When writing to disk protect moving superblock to buffer with um_lock.

Set/clear fs->fmod while mounting, updating a mount or unmounting
is safe as these operations run exclusive, either mounting creates
a new file system or the file system is suspended. Assert suspension
for update and unmount.

PR kern/58837 "ffs: Missing locking around fs_fmod/time"

RSS XML Feed