Home | History | Annotate | Download | only in ffs
History log of /src/sys/ufs/ffs/ffs_subr.c
RevisionDateAuthorComments
 1.54  07-Jan-2023  chs ufs: fixed signed/unsigned bugs affecting large file systems

Apply these commits from FreeBSD:

commit e870d1e6f97cc73308c11c40684b775bcfa906a2
Author: Kirk McKusick <mckusick@FreeBSD.org>
Date: Wed Feb 10 20:10:35 2010 +0000

This fix corrects a problem in the file system that treats large
inode numbers as negative rather than unsigned. For a default
(16K block) file system, this bug began to show up at a file system
size above about 16Tb.

To fully handle this problem, newfs must be updated to ensure that
it will never create a filesystem with more than 2^32 inodes. That
patch will be forthcoming soon.

Reported by: Scott Burns, John Kilburg, Bruce Evans
Followup by: Jeff Roberson
PR: 133980
MFC after: 2 weeks

commit 81479e688b0f643ffacd3f335b4b4bba460b769d
Author: Kirk McKusick <mckusick@FreeBSD.org>
Date: Thu Feb 11 18:14:53 2010 +0000

One last pass to get all the unsigned comparisons correct.


In additional to the changes from FreeBSD, this commit includes quite a few
related changes to appease -Wsign-compare.
 1.53  24-May-2022  andvar branches: 1.53.4;
fix various typos in comments, docs and log messages.
 1.52  21-Apr-2020  christos use %s/__func__ so that the strings can be shared.
 1.51  28-May-2019  kamil branches: 1.51.8;
Avoid unportable shift base -1 in ffs_subr.c

Cast the start variable before the modulo opration to unsigned int.

Detected with kUBSan.
 1.50  04-Jul-2018  kamil Avoid Undefined Behavior in ffs_clusteracct()

Change the type of 'bit' variable from int to unsigned int and use unsigned
values consistently.

sys/ufs/ffs/ffs_subr.c:336:10, shift exponent -1 is negative

Detected with Kernel Undefined Behavior Sanitizer.

Reported by <Harry Pantazis>
 1.49  07-May-2016  maxv branches: 1.49.16; 1.49.18;
uaf
 1.48  20-Oct-2013  htodd branches: 1.48.6;
Definining needswap where needed.
 1.47  14-Aug-2011  christos branches: 1.47.2; 1.47.12; 1.47.16;
fix sign-compare warnings
 1.46  06-Mar-2011  bouyer merge the bouyer-quota2 branch. This adds a new on-disk format
to store disk quota usage and limits, integrated with ffs
metadata. Usage is checked by fsck_ffs (no more quotacheck)
and is covered by the WAPBL journal. Enabled with kernel
option QUOTA2 (added where QUOTA was enabled in kernel config files),
turned on with tunefs(8) on a per-filesystem
basis. mount_mfs(8) can also turn quotas on.

See http://mail-index.netbsd.org/tech-kern/2011/02/19/msg010025.html
for details.
 1.45  03-Jun-2008  hannken branches: 1.45.20; 1.45.26; 1.45.28;
ufs/ffs: replace calls to getblk() with ffs_getblk(). Now all buffers
have been run through copy-on-write and async mounts work again.

Fixes PR kern/38820

Reviewed by: YAMAMOTO Takashi <yamt@netbsd.org>
 1.44  29-Jan-2007  hubertf branches: 1.44.40; 1.44.42; 1.44.44; 1.44.46;
Remove more duplicate headers.
Patch by Slava Semushin <slava.semushin@gmail.com>

Again, this was tested by comparing obj files from a pristine and a patched
source tree against an i386/ALL kernel, and also for src/sbin/fsck_ffs,
src/sbin/fsdb and src/usr.sbin/makefs. Only changes in assert() line numbers
were detected in 'objdump -d' output.
 1.43  16-Nov-2006  christos __unused removal on arguments; approved by core.
 1.42  12-Oct-2006  christos - sprinkle __unused on function decls.
- fix a couple of unused bugs
- no more -Wno-unused for i386
 1.41  14-Jan-2006  yamt branches: 1.41.18; 1.41.20;
- unify ffs_blkatoff and lfs_blkatoff.
- remove ufs_ops::uo_blkatoff.
- add directory read-ahead code. (disabled for now.)
 1.40  27-Dec-2005  chs branches: 1.40.2;
changes for making DIAGNOSTIC not change the kernel ABI:
- for structure fields that are conditionally present,
make those fields always present.
- for functions which are conditionally inline, make them never inline.
- remove some other functions which are conditionally defined but
don't actually do anything anymore.
- make a lock-debugging function conditional on only LOCKDEBUG.

as discussed on tech-kern some time back.
 1.39  11-Dec-2005  christos merge ktrace-lwp.
 1.38  02-Nov-2005  yamt merge yamt-vop branch. remove following VOPs.

VOP_BLKATOFF
VOP_VALLOC
VOP_BALLOC
VOP_REALLOCBLKS
VOP_VFREE
VOP_TRUNCATE
VOP_UPDATE
 1.37  12-Sep-2005  drochner branches: 1.37.2;
move the new ffs_itimes() to a berr place -- ffs_subr.c is shared with
userland
 1.36  12-Sep-2005  christos Use nanotime() to update the time fields in filesystems. Convert the code
from macros to real functions. Original patch and review from chuq.
Note: ext2fs only keeps seconds in the on-disk inode, and msdosfs does not
have enough precision for all fields, so this is not very useful for those
two.
 1.35  30-Aug-2005  xtraeme * Remove __P()
* Use ANSI function declarations on ext2fs and mfs
 1.34  15-Jul-2005  thorpej Use ANSI function decls.
 1.33  26-Feb-2005  perry branches: 1.33.4;
nuke trailing whitespace
 1.32  30-Dec-2003  pk branches: 1.32.8; 1.32.10;
Replace the traditional buffer memory management -- based on fixed per buffer
virtual memory reservation and a private pool of memory pages -- by a scheme
based on memory pools.

This allows better utilization of memory because buffers can now be allocated
with a granularity finer than the system's native page size (useful for
filesystems with e.g. 1k or 2k fragment sizes). It also avoids fragmentation
of virtual to physical memory mappings (due to the former fixed virtual
address reservation) resulting in better utilization of MMU resources on some
platforms. Finally, the scheme is more flexible by allowing run-time decisions
on the amount of memory to be used for buffers.

On the other hand, the effectiveness of the LRU queue for buffer recycling
may be somewhat reduced compared to the traditional method since, due to the
nature of the pool based memory allocation, the actual least recently used
buffer may release its memory to a pool different from the one needed by a
newly allocated buffer. However, this effect will kick in only if the
system is under memory pressure.
 1.31  02-Dec-2003  dbj clarify comments, especially since ffs_isfreeblock is non-intuitive:
ffs_isblock:
check if a block is available
returns true if all the correponding bits in the free map are 1
returns false if any corresponding bit in the free map is 0
ffs_isfreeblock:
check if a block is completely allocated
returns true if all the corresponding bits in the free map are 0
returns false if any corresponding bit in the free map is 1
 1.30  27-Oct-2003  lukem Overhaul how `build.sh tools' are used:

* Rename "config.h" to "nbtool_config.h" and
HAVE_CONFIG_H to HAVE_NBTOOL_CONFIG_H.
This makes in more obvious in the source when we're using
tools/compat/config.h versus "standard autoconf" config.h

* Consistently move the inclusion of nbtool_config.h to before
<sys/cdefs.h> so that the former can provide __RCSID() (et al),
and there's no need to protect those macros any more.

These changes should make it easier to "tool-ify" a program by adding:
#if HAVE_NBTOOL_CONFIG_H
#include "nbtool_config.h"
#endif
to the top of the source files (for the general case).
 1.29  07-Aug-2003  agc Move UCB-licensed code from 4-clause to 3-clause licence.

Patches provided by Joel Baker in PR 22364, verified by myself.
 1.28  02-Apr-2003  fvdl branches: 1.28.2;
Add support for UFS2. UFS2 is an enhanced FFS, adding support for
64 bit block pointers, extended attribute storage, and a few
other things.

This commit does not yet include the code to manipulate the extended
storage (for e.g. ACLs), this will be done later.

Originally written by Kirk McKusick and Network Associates Laboratories for
FreeBSD.
 1.27  25-Jan-2003  tron Use PRId64 instead of hard coding "%lld" to fix build problems under
LP64 ports.
 1.26  25-Jan-2003  tron Fix printf() format strings problems caused by "daddr_t" change.
 1.25  24-Jan-2003  fvdl Bump daddr_t to 64 bits. Replace it with int32_t in all places where
it was used on-disk, so that on-disk formats remain the same.
Remove ufs_daddr_t and ufs_lbn_t for the time being.
 1.24  01-Dec-2002  matt Add multiple inclusion protection for headers. Fix mismatched
variable declarations (missing const's) as needed.
 1.23  06-Jul-2002  fredette Fixed a printf argument type.
 1.22  10-Apr-2002  mycroft branches: 1.22.2;
Use blkstofrags() and fragstoblks(). Use &(NBBY-1) rather than %NBBY.
Switch off of fs_fragshift rather than fs_frag (generates better jump tables).
 1.21  31-Jan-2002  tv These sources are pulled into makefs(8), so we need config.h and protection
for __KERNEL_RCSID().
 1.20  09-Jan-2002  lukem Only pull in <sys/systm.h> #ifdef _KERNEL, since it's a kernel only header.
In the ! _KERNEL case, provide own prototype for panic() instead.
 1.19  30-Oct-2001  lukem add __KERNEL_RCSID()
 1.18  26-Oct-2001  lukem - pull in ufsmount.h after inode.h, because the latter pulls in
quota.h which the former needs, and this makes the usage consistent
with other files anyway
- expand the details in a few panic strings
 1.17  26-Oct-2001  lukem remove #include <ufs/ufs/quota.h> where it was just to appease
<ufs/ufs/inode.h>, since the latter now includes the former. leave the former
in source that obviously uses specific bits of it (for completeness.)
 1.16  09-Aug-2001  lukem branches: 1.16.4;
be consistent and use "u_char" instead of "unsigned char"
 1.15  30-Mar-2000  augustss branches: 1.15.6; 1.15.10;
Remove register declarations.
 1.14  15-Nov-1999  fvdl Add Kirk McKusick's soft updates code to the trunk. Not enabled by
default, as the copyright on the main file (ffs_softdep.c) is such
that is has been put into gnusrc. options SOFTDEP will pull this
in. This code also contains the trickle syncer.

Bump version number to 1.4O
 1.13  28-Jul-1998  drochner branches: 1.13.14; 1.13.16; 1.13.20;
The fragtbl[], inside[] and around[] variables are needed by "fsck",
so we can't put them inside "#ifdef _KERNEL".
Put declarations inside .c files where needed to preserve namespace.
 1.12  13-Jun-1998  kleink KNF, mostly of FFS_EI changes.
 1.11  18-Mar-1998  bouyer Add support for reading/writing FFS in non-native byte order, conditioned
to "options FFS_EI". The superblock and inodes (without blk addr) are
byteswapped at disk read/write time, other metadatas are byteswapped
when used (as they are acceeded directly in the buffer cache).
This required the addition of a "um_flags" field to struct ufsmount.
ffs_bswap.c contains superblock and inode byteswap routines also used
by userland utilities.
 1.10  01-Mar-1998  fvdl Merge with Lite2 + local changes
 1.9  12-Oct-1996  christos revert previous kprintf changes
 1.8  10-Oct-1996  christos printf -> kprintf, sprintf -> ksprintf
 1.7  20-Sep-1996  christos Make this compile cleanly from userland (fsck_ffs).
 1.6  17-Mar-1996  christos Fix printf format strings
 1.5  09-Feb-1996  christos ffs prototypes
 1.4  28-Mar-1995  jtc KERNEL -> _KERNEL
 1.3  20-Oct-1994  cgd update for new syscall args description mechanism, and deal safely
with wider types.
 1.2  29-Jun-1994  cgd New RCS ID's, take two. they're more aesthecially pleasant, and use 'NetBSD'
 1.1  08-Jun-1994  mycroft branches: 1.1.1;
Update to 4.4-Lite fs code, with local changes.
 1.1.1.2  01-Mar-1998  fvdl Import 4.4BSD-Lite2
 1.1.1.1  01-Mar-1998  fvdl Import 4.4BSD-Lite for reference
 1.13.20.2  27-Dec-1999  wrstuden Pull up to last week's -current.
 1.13.20.1  21-Dec-1999  wrstuden Initial commit of recent changes to make DEV_BSIZE go away.

Runs on i386, needs work on other arch's. Main kernel routines should be
fine, but a number of the stand programs need help.

cd, fd, ccd, wd, and sd have been updated. sd has been tested with non-512
byte block devices. vnd, raidframe, and lfs need work.

Non 2**n block support is automatic for LKM's and conditional for kernels
on "options NON_PO2_BLOCKS".
 1.13.16.1  19-Oct-1999  fvdl Bring in Kirk McKusick's FFS softdep code on a branch.
 1.13.14.1  20-Nov-2000  bouyer Update thorpej_scsipi to -current as of a month ago
 1.15.10.5  06-Sep-2002  jdolecek sync kqueue branch with HEAD
 1.15.10.4  23-Jun-2002  jdolecek catch up with -current on kqueue branch
 1.15.10.3  11-Feb-2002  jdolecek Sync w/ -current.
 1.15.10.2  10-Jan-2002  thorpej Sync kqueue branch with -current.
 1.15.10.1  25-Aug-2001  thorpej Merge Aug 24 -current into the kqueue branch.
 1.15.6.7  11-Dec-2002  thorpej Sync with HEAD.
 1.15.6.6  01-Aug-2002  nathanw Catch up to -current.
 1.15.6.5  17-Apr-2002  nathanw Catch up to -current.
 1.15.6.4  28-Feb-2002  nathanw Catch up to -current.
 1.15.6.3  11-Jan-2002  nathanw More catchup.
 1.15.6.2  14-Nov-2001  nathanw Catch up to -current.
 1.15.6.1  24-Aug-2001  nathanw Catch up with -current.
 1.16.4.1  12-Nov-2001  thorpej Sync the thorpej-mips-cache branch with -current.
 1.22.2.1  15-Jul-2002  gehenna catch up with -current.
 1.28.2.5  10-Nov-2005  skrll Sync with HEAD. Here we go again...
 1.28.2.4  04-Mar-2005  skrll Sync with HEAD.

Hi Perry!
 1.28.2.3  21-Sep-2004  skrll Fix the sync with head I botched.
 1.28.2.2  18-Sep-2004  skrll Sync with HEAD.
 1.28.2.1  03-Aug-2004  skrll Sync with HEAD
 1.32.10.1  19-Mar-2005  yamt sync with head. xen and whitespace. xen part is not finished.
 1.32.8.1  29-Apr-2005  kent sync with -current
 1.33.4.2  26-Feb-2007  yamt sync with head.
 1.33.4.1  21-Jun-2006  yamt sync with head.
 1.37.2.1  20-Oct-2005  yamt adapt ufs.
 1.40.2.1  15-Jan-2006  yamt sync with head.
 1.41.20.2  10-Dec-2006  yamt sync with head.
 1.41.20.1  22-Oct-2006  yamt sync with head
 1.41.18.2  01-Feb-2007  ad Sync with head.
 1.41.18.1  18-Nov-2006  ad Sync with head.
 1.44.46.1  23-Jun-2008  wrstuden Sync w/ -current. 34 merge conflicts to follow.
 1.44.44.1  04-May-2009  yamt sync with head.
 1.44.42.1  04-Jun-2008  yamt sync with head
 1.44.40.1  05-Jun-2008  mjf Sync with HEAD.

Also fix build.
 1.45.28.4  09-Feb-2011  bouyer Make it build without FFS_EI
 1.45.28.3  08-Feb-2011  bouyer for !_KERNEL case, always define FFS_EI.
Required for makefs, and maybe resize_ffs (it's not clear if
resize_ffs supports swapped byte order or not - swapped endian tests
are expected to fail but actually succeed :)
 1.45.28.2  08-Feb-2011  bouyer Sync with HEAD
 1.45.28.1  20-Jan-2011  bouyer Snapshot of work in progress on a modernised disk quota system:
- new quotactl syscall (versionned for backward compat), which takes
as parameter a path to a mount point, and a prop_dictionary
(in plistref format) describing commands and arguments.
For each command, status and data are returned as a prop_dictionary.
quota commands features will be added to take advantage of this,
exporting quota data or getting quota commands as plists.

- new on disk-format storage (all 64bit wide), integrated to metadata for
ffs (and playing nicely with wapbl).
Quotas are enabled on a ffs filesystem via superblock flags.
tunefs(8) can enable or disable quotas.
On a quota-enabled filesystem, fsck_ffs(8) will track per-uid/gid
block and inode usages, and will check and update quotas in Pass 6.
quota usage and limits are stored in unliked files (one for users,
one for groups)l fsck_ffs(8) will create the files if needed, or
free them if needed. This means that after enabling or disabling
quotas on a filesystem; a fsck_ffs(8) run is required.
quotacheck(8) is not needed any more, on a unclean shutdown
fsck or journal replay will take care of fixing quotas.
newfs(8) can create a ready-to-mount quota-enabled filesystem
(superblock flags are set and quota inodes are created).
Other new features or semantic changes:
- default quota datas, applied to users or groups which don't already
have a quota entry
- per-user/group grace time (instead of a filesystem global one)
- 0 really means "nothing allowed at all", not "no limit".
If you want "no limit", set the limit to UQUAD_MAX (tools will
understand "unlimited" and "-")

A quota file is structured as follow:
it starts with a header, containing a few per-filesystem values,
and the default quota limits.
Quota entries are linked together as a simple list, each entry has a
pointer (as an offset withing the file) to the next.
The header has a pointer to a list of free quota entries, and
a hash table of in-use entries. The size of the hash table depends
on the filesystem block size (header+hash table should fit in the
first block). The file is not sparse and is a multiple of
filesystem block size (when the free quota entry list is empty a new
filesystem block is allocated). quota entries to not cross
filesystem block boundaries.

In memory, the kernel keeps a cache of recently used quota entries
as a reference to the block number, and offset withing the block.
The quota entry itself is keept in the buf cache.

fsck_ffs(8), tunefs(8) and newfs(8) supports are completed (with
related atf tests :)
The kernel can update disk usage and report it via quotactl(2).

Todo: enforce quotas limits (limits are not checked by kernel yet)
update repquota, edquota and rpc.rquotad to the new world
implement compat_50_quotactl ioctl.
update quotactl(2) man page

fsck_ffs required fixes so that allocating new blocks or inodes will
properly update the superblock and cg sumaries. This was not an issue up
to now because superblock and cg sumaries check happened last, but now
allocations or frees can happen in pass 6.
 1.45.26.1  06-Jun-2011  jruoho Sync with HEAD.
 1.45.20.1  21-Apr-2011  rmind sync with head
 1.47.16.1  18-May-2014  rmind sync with head
 1.47.12.2  03-Dec-2017  jdolecek update from HEAD
 1.47.12.1  20-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.47.2.1  22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.48.6.1  29-May-2016  skrll Sync with HEAD
 1.49.18.2  21-Apr-2020  martin Sync with HEAD
 1.49.18.1  10-Jun-2019  christos Sync with HEAD
 1.49.16.1  28-Jul-2018  pgoyette Sync with HEAD
 1.51.8.1  25-Apr-2020  bouyer Sync with bouyer-xenpvh-base2 (HEAD)
 1.53.4.1  13-May-2023  martin Pull up following revision(s) (requested by chs in ticket #160):

usr.sbin/makefs/ffs/ffs_alloc.c: revision 1.31
sbin/tunefs/tunefs.c: revision 1.58
sbin/fsck_ffs/setup.c: revision 1.105
sbin/fsck_ffs/pass5.c: revision 1.56
usr.sbin/makefs/ffs.c: revision 1.74
usr.sbin/makefs/ffs/mkfs.c: revision 1.42
usr.sbin/makefs/Makefile: revision 1.40
sys/ufs/ffs/fs.h: revision 1.71
sbin/fsdb/fsdb.c: revision 1.54
sbin/resize_ffs/resize_ffs.c: revision 1.58
sbin/fsck_ffs/pass4.c: revision 1.29
usr.sbin/makefs/ffs/ffs_extern.h: revision 1.9
sbin/newfs/mkfs.c: revision 1.133
sys/ufs/ffs/ffs_alloc.c: revision 1.172
sbin/fsck_ffs/pass1b.c: revision 1.24
usr.sbin/dumpfs/dumpfs.c: revision 1.68
sys/ufs/ffs/ffs_extern.h: revision 1.88
usr.sbin/quotacheck/quotacheck.c: revision 1.51
sys/ufs/ffs/ffs_subr.c: revision 1.54
sbin/fsck_ffs/main.c: revision 1.91
sbin/fsck_ffs/pass1.c: revision 1.63

ufs: fixed signed/unsigned bugs affecting large file systems

Apply these commits from FreeBSD:
commit e870d1e6f97cc73308c11c40684b775bcfa906a2
Author: Kirk McKusick <mckusick@FreeBSD.org>
Date: Wed Feb 10 20:10:35 2010 +0000
This fix corrects a problem in the file system that treats large
inode numbers as negative rather than unsigned. For a default
(16K block) file system, this bug began to show up at a file system
size above about 16Tb.
To fully handle this problem, newfs must be updated to ensure that
it will never create a filesystem with more than 2^32 inodes. That
patch will be forthcoming soon.
Reported by: Scott Burns, John Kilburg, Bruce Evans
Followup by: Jeff Roberson
PR: 133980
MFC after: 2 weeks

commit 81479e688b0f643ffacd3f335b4b4bba460b769d
Author: Kirk McKusick <mckusick@FreeBSD.org>
Date: Thu Feb 11 18:14:53 2010 +0000
One last pass to get all the unsigned comparisons correct.

In additional to the changes from FreeBSD, this commit includes quite a few
related changes to appease -Wsign-compare.

RSS XML Feed