Home | History | Annotate | Download | only in udf
History log of /src/sys/fs/udf/udf.h
RevisionDateAuthorComments
 1.56  18-May-2024  thorpej Move the include of <sys/malloc.h> to where it's truly needed.
 1.55  27-Jun-2023  reinoud Convert UDF file system code from the depricated tsleep(9)/wakeup(9) to the MP
friendly condvar(9). No functional change intended.
 1.54  07-Aug-2022  andvar fix various typos in comments, documentation and messages.
mainly s/paramater/parameter/ and s/reduntant/redundant/.
 1.53  16-Apr-2022  andvar fix various typos in comments and log messages.
 1.52  24-May-2016  reinoud Cleanup VAT writout. To prevent issues with the sequential writing strategy
trying to write on blocks that are lost due to the synchronisation, don't just
bluntly do synchronize device caches, but split out on strategies.
 1.51  10-May-2016  reinoud Rework VAT searching on recordable media. It is now a lot more resilliant to
errors and it allows for VAT searching on crashed writeouts.

While here, make sure the node pointer is always initialised in
udf_get_node().
 1.50  24-Aug-2015  hannken Use vfs_vnode_iterator for udf_do_sync.

- Build rb_tree to get an ordered list of nodes, sync them and
clean the tree.

- Stop abusing mntvnode_lock to serialize udf_do_sync, use new
mutex ump->sync_lock instead.
 1.49  24-Aug-2015  hannken Remove dirtynodes_cv, it is only used for timed waits without any signals.

Replace the cv_timedwait with kpause.
 1.48  06-Apr-2015  hannken Change udf to vcache, keyed on udf_node->loc.loc.
 1.47  17-Sep-2014  reinoud branches: 1.47.2;
As pointed out by wiz@ prevent a possible attack or corruption that results in
an endless loop of indirect descriptors being processed.

The number of indirect descriptors followed is now maximized.
While here, also fix a use-after-free bug!
 1.46  18-Oct-2013  christos fix unused variable warnings
 1.45  27-Jul-2012  drochner branches: 1.45.2; 1.45.4;
gcc -fno-common fallout
 1.44  27-Sep-2011  christos branches: 1.44.2;
- rename UDF_MAX_NAMELEN -> UDF_MAXNAMLEN
- use NAME_MAX instead of MAXNAMLEN
 1.43  25-Sep-2010  matt Rename rb.h to rbtree.h, as it is more appropriate (c.f. ptree.h). Also
helps find code that hasn't been updated to use the new rbtree API.
 1.42  24-Sep-2010  rmind Fixes/improvements to RB-tree implementation:
1. Fix inverted node order, so that negative value from comparison operator
would represent lower (left) node, and positive - higher (right) node.
2. Add an argument (i.e. "context"), passed to comparison operators.
3. Change rb_tree_insert_node() to return a node - either inserted one or
already existing one.
4. Amend the interface to manipulate the actual object, instead of the
rb_node (in a similar way as Patricia-tree interface does).
5. Update all RB-tree users accordingly.

XXX: Perhaps rename rb.h to rbtree.h, since cleaning-up..

1-3 address the PR/43488 by Jeremy Huddleston.

Passes RB-tree regression tests.
Reviewed by: matt@, christos@
 1.41  25-Feb-2010  reinoud branches: 1.41.2;
First part of shrinking/growing metadata partition support:

- extending the metadata partition

Still to follow:
- sparsify metadata partition
- growing the metadata partition
- unsparsifying metadata partition
 1.40  24-Feb-2010  reinoud Rename metadata partition parameters to be less generic and add the flags.
 1.39  24-Feb-2010  reinoud Extract UDF metadata partition parameters as already done for VAT and sparable
partitions.
 1.38  05-Aug-2009  reinoud branches: 1.38.2;
Typo fix
 1.37  08-Jul-2009  reinoud Remove unused LIST_ENTRY()'s now replaced by the rbtree.
 1.36  07-Jul-2009  reinoud Replace the old hashtable and sorted list implemenation by a RB-tree.

Benefits are significant speed improvements on node creation/insertion while
keeping the lookup times low and still allowing sequential iteration over the
nodes.
 1.35  06-Jul-2009  reinoud Reduce the number of active ecclines; low memory systems could get into
trouble otherwise.

XXX this value should be computed eventually.
 1.34  24-Jun-2009  reinoud Commit first stage of free-space accounting. It is estimating the underlimit
of free blocks on the device and when free blocks are getting tight it tries
to readjust/recalculate that value by syncing the FS.

Second stage will be resizing the data/metadata partitions.
 1.33  23-Jun-2009  reinoud Bump the NetBSD kernel UDF id to version 0.5
 1.32  18-Jun-2009  reinoud Add RESERVE bitmask flag to DEBUG print constants adding another 0 prefix to
the constants already defined.
 1.31  18-Jun-2009  reinoud Remove now unused UDF_DIRHASH constants
 1.30  08-Feb-2009  reinoud branches: 1.30.2;
Implement session closure on sequential media when requested on mount. It will
allow CD-ROM/DVD-ROM/DB-ROM drives to read the media while still allowing them
to be appended later. It can also be seen as a way to make mountable
snapshots.
 1.29  09-Dec-2008  reinoud Fix two buf_init()'s that are not needed since they were already initialised
by getiobuf().

Also remove ancient IPL_BIO reference from the time it was called from
interrupt context instead of the current soft interrupt.
 1.28  05-Dec-2008  reinoud Two fold commit:

1) Enhance write speed significantly on RMW media like CD-RW, DVD-RW but also
on the DVD+RW and all other ECC blocked media. Significant speedups of access
to the device for say compilation on the DVD. Streaming copy is also still at
maximum speed though vast amounts of directory copy work can show side effects
that appear it to slow down but are actually logical when you consider that
most small files are embedded into the descriptors itself.

2) explicit wait for the created RMW thread to spinup
 1.27  27-Sep-2008  reinoud branches: 1.27.2; 1.27.4;
Switch UDF over to the vfs_dirhash.c dirhash code.
 1.26  06-Aug-2008  reinoud Rewrite allocation scheme's determination and implementation replacing the
old somewhat naive selection scheme that didn't allow different allocation
settings for nodes, directory information (FIDs) and data.

Also fix some curious side-effects of atime updates on RMW devices.
 1.25  29-Jul-2008  reinoud Remove comments on `need to read in'; they are read in.
 1.24  28-Jul-2008  reinoud Streamline allocation to prepare it for metadata partition. Also add
preliminary Metadata partition write support but its disabled still since
its not finished yet and not functioning correctly. All other formats are
checked and should work fine.
 1.23  27-Jul-2008  reinoud Unify naming scheme and source code beautify for better readablility.
 1.22  26-Jul-2008  reinoud Clarify the current spacemap read/write code is reading/writing physical
partition space maps. This in preperation of Metadata spacetable
reading/writeout work.
 1.21  22-Jul-2008  reinoud Limit the UDF dirhash to a configurable amount of memory. The performance
penalty even with a limited dirhash of 128kb is acceptable but it should be
set by machine memory size though.
 1.20  19-Jul-2008  reinoud Use ecma-167 constants instead of homegrown to prevent double mapping.
 1.19  19-Jul-2008  reinoud Choose which time of allocation descriptors to use on new nodes based on
the number of partition mappings.
 1.18  17-Jul-2008  reinoud Implement directory hashing to speed up directory traversals. Speed
improvements of at least 4 times in untarring and roughly 100 to 500 times
on file creation in big directories. Lookup of files was O(n*n) and is now
O(1) even for file creation. Free spaces in the directory are kept in a
seperate list for fast file creation.

The postmark benchmark gives:

UDF old:
pm>set transactions 2000
pm>set number 3000
pm>run
Creating files...Done
Performing transactions..........Done
Deleting files...Done
Time:
1593 seconds total
681 seconds of transactions (2 per second)

Files:
3956 created (2 per second)
Creation alone: 3000 files (4 per second)
Mixed with transactions: 956 files (1 per second)
990 read (1 per second)
1010 appended (1 per second)
3956 deleted (2 per second)
Deletion alone: 2912 files (9 per second)
Mixed with transactions: 1044 files (1 per second)

Data:
5.26 megabytes read (3.38 kilobytes per second)
21.93 megabytes written (14.10 kilobytes per second)
pm>



UDF new:
pm>set transactions 2000
pm>set number 3000
pm>run
Creating files...Done
Performing transactions..........Done
Deleting files...Done
Time:
19 seconds total
3 seconds of transactions (666 per second)

Files:
3956 created (208 per second)
Creation alone: 3000 files (230 per second)
Mixed with transactions: 956 files (318 per second)
990 read (330 per second)
1010 appended (336 per second)
3956 deleted (208 per second)
Deletion alone: 2912 files (970 per second)
Mixed with transactions: 1044 files (348 per second)

Data:
5.26 megabytes read (283.66 kilobytes per second)
21.93 megabytes written (1.15 megabytes per second)
 1.17  07-Jul-2008  reinoud Support for allocation extents descriptor writing to allow for arbitrary
heavily fragmented files.

Also fixing some (rare) allocation bugs and function name streamlining.

Tested on harddisc, CD-RW and CD-R i.e. all three basic backend classes.
 1.16  03-Jul-2008  reinoud branches: 1.16.2;
Add node-rebuild flag to indicate that the node was rebuild up and thus the
allocation extents have changed.
 1.15  03-Jul-2008  reinoud Create new udf_verbose class.
 1.14  28-Jun-2008  rumble Create sysctl entries during module initialisation and destroy them
appropriately.

Many of these file systems are now ready for modularisation.
 1.13  25-Jun-2008  reinoud Bump maximum number of allocation extents descriptors in a file/directory;
10 did seem far too many, but aparently ``640 kb should be enough for
everyone''.
 1.12  15-May-2008  reinoud branches: 1.12.2;
Introduce new UDF debug class and split rmw strategy debugging prints.
 1.11  14-May-2008  reinoud Import writing part of the UDF file system making optical media like CD's
and DVD's behave like floppy discs. Writing is supported upto and including
version 2.01; version 2.50 and 2.60 will follow.

Also extending the UDF implementation to support symbolic links and
hardlinks.

Added are the mmcformat(8) tool to format rewritable CD/DVD discs and
newfs_udf(8).

Limitations:
all operations can be performed on the file system though the
sheduling is currently optimised for archiving workloads.

mv(1)/rename(2) is currently only implemented for non-directories.
 1.10  10-Oct-2007  ad branches: 1.10.18; 1.10.20; 1.10.22; 1.10.24;
Merge from vmlocking:

- Split vnode::v_flag into three fields, depending on field locking.
- simple_lock -> kmutex in a few places.
- Fix some simple locking problems.
 1.9  04-Jan-2007  reinoud branches: 1.9.6; 1.9.18; 1.9.20; 1.9.22;
Implement read-only access to UDF 2.50 and UDF 2.60 discs that use metadata
partition mappings. This brings the NetBSD implementation to read all discs
upto date and will read all discs currently defined by OSTA.
 1.8  29-Sep-2006  reinoud branches: 1.8.2;
Add `vfs.udf.verbose' sysctl when running a DEBUG kernel to dynamically set
the udf_verbose variable. So when something goes wrong, it can be examined
on the spot without needing to reboot a new kernel and possibly loosing
state.
 1.7  19-Sep-2006  reinoud Make VOP_LOOKUP's O(1) instead of O(n log(n)) giving a speedup of 7500% on
a 1400 file directory.
 1.6  22-Aug-2006  reinoud branches: 1.6.2; 1.6.4; 1.6.6;
Fix pool allocation/freeing problem for once and for all correctly. Since
it was initialised quite late due to its reliance on disc data the mount
process could have stopped before initialising and thus could panic again
only now for uninitialising an not initialised pool! *sigh*
 1.5  21-Aug-2006  reinoud Make a better guess on VAT locations so discs that are semi-standard
formatted can be recognised. This also fixes the bug that prevented UDF to
mount an empty CD-R/DVD*R disc.
 1.4  02-Feb-2006  christos branches: 1.4.4; 1.4.6; 1.4.16; 1.4.18;
protect the whole file, not parts of it.
 1.3  02-Feb-2006  reinoud Fix header re-inclusion protection defines to a less generic naming
scheme.
 1.2  02-Feb-2006  reinoud Fix $NetBSD$ tags. (sorry)
 1.1  02-Feb-2006  reinoud Initial import of a UDF file system implementation for NetBSD.

The code supports read access to all media types that CD/DVD type drives
can recognize including DVD-RAM and BD- drives as well as harddisc partions
and vnd devices. UDF versions upto the latest 2.60 are to be supported
though due to lack of test media version 2.50 and 2.60 are not implemented
yet though easy to add. Both open and closed media are supported.

Write access is planned and in preparation. To facilitate this some hooks
are present in the code that are not strictly needed in a read-only
implementation but which allow writing to be added more easily.

Implemented and tested media types are CD-ROM, CD-R, CD-RW, CD-MRW,
DVD-ROM, DVD*R, DVD*RW, DVD+MRW but the same code can also read DVD-RAM,
HD-DVD and BluRay discs. Also vnd devices have been tested with several
sector sizes.

Discs created and written by UDFclient, Nero's InCD and Roxio's
DirectCD/Drag2Disc read fine.
 1.4.18.3  25-Sep-2006  riz Pull up following revision(s) (requested by reinoud in ticket #185):
sys/fs/udf/udf_subr.c: revision 1.18
sys/fs/udf/udf.h: revision 1.7
Make VOP_LOOKUP's O(1) instead of O(n log(n)) giving a speedup of 7500% on
a 1400 file directory.
 1.4.18.2  27-Aug-2006  riz Pull up following revision(s) (requested by reinoud in ticket #50):
sys/fs/udf/udf_subr.c: revision 1.14
sys/fs/udf/udf.h: revision 1.6
sys/fs/udf/udf_vfsops.c: revision 1.11
Fix pool allocation/freeing problem for once and for all correctly. Since
it was initialised quite late due to its reliance on disc data the mount
process could have stopped before initialising and thus could panic again
only now for uninitialising an not initialised pool! *sigh*
 1.4.18.1  24-Aug-2006  tron Pull up following revision(s) (requested by reinoud in ticket #46):
sys/fs/udf/udf_subr.c: revision 1.13
sys/fs/udf/udf.h: revision 1.5
Make a better guess on VAT locations so discs that are semi-standard
formatted can be recognised. This also fixes the bug that prevented UDF to
mount an empty CD-R/DVD*R disc.
 1.4.16.5  27-Oct-2007  yamt sync with head.
 1.4.16.4  26-Feb-2007  yamt sync with head.
 1.4.16.3  30-Dec-2006  yamt sync with head.
 1.4.16.2  21-Jun-2006  yamt sync with head.
 1.4.16.1  02-Feb-2006  yamt file udf.h was added on branch yamt-lazymbuf on 2006-06-21 15:09:36 +0000
 1.4.6.1  03-Sep-2006  yamt sync with head.
 1.4.4.2  18-Feb-2006  yamt sync with head.
 1.4.4.1  02-Feb-2006  yamt file udf.h was added on branch yamt-uio_vmspace on 2006-02-18 15:39:18 +0000
 1.6.6.1  22-Oct-2006  yamt sync with head
 1.6.4.2  09-Sep-2006  rpaulo sync with head
 1.6.4.1  22-Aug-2006  rpaulo file udf.h was added on branch rpaulo-netinet-merge-pcb on 2006-09-09 02:57:06 +0000
 1.6.2.2  12-Jan-2007  ad Sync with head.
 1.6.2.1  18-Nov-2006  ad Sync with head.
 1.8.2.1  04-Jan-2007  bouyer Pull up following revision(s) (requested by reinoud in ticket #332):
sys/fs/udf/udf_subr.c: revision 1.26
sys/fs/udf/udf.h: revision 1.9
sys/fs/udf/udf_vfsops.c: revision 1.19
Implement read-only access to UDF 2.50 and UDF 2.60 discs that use metadata
partition mappings. This brings the NetBSD implementation to read all discs
upto date and will read all discs currently defined by OSTA.
 1.9.22.1  14-Oct-2007  yamt sync with head.
 1.9.20.1  06-Nov-2007  matt sync with HEAD
 1.9.18.1  26-Oct-2007  joerg Sync with HEAD.

Follow the merge of pmap.c on i386 and amd64 and move
pmap_init_tmp_pgtbl into arch/x86/x86/pmap.c. Modify the ACPI wakeup
code to restore CR4 before jumping back into kernel space as the large
page option might cover that.
 1.9.6.1  05-Apr-2007  ad Compile fixes.
 1.10.24.3  10-Oct-2008  skrll Sync with HEAD.
 1.10.24.2  18-Sep-2008  wrstuden Sync with wrstuden-revivesa-base-2.
 1.10.24.1  23-Jun-2008  wrstuden Sync w/ -current. 34 merge conflicts to follow.
 1.10.22.6  09-Oct-2010  yamt sync with head
 1.10.22.5  11-Mar-2010  yamt sync with head
 1.10.22.4  19-Aug-2009  yamt sync with head.
 1.10.22.3  18-Jul-2009  yamt sync with head.
 1.10.22.2  04-May-2009  yamt sync with head.
 1.10.22.1  16-May-2008  yamt sync with head.
 1.10.20.1  18-May-2008  yamt sync with head.
 1.10.18.5  17-Jan-2009  mjf Sync with HEAD.
 1.10.18.4  05-Oct-2008  mjf Sync with HEAD.
 1.10.18.3  28-Sep-2008  mjf Sync with HEAD.
 1.10.18.2  29-Jun-2008  mjf Sync with HEAD.
 1.10.18.1  02-Jun-2008  mjf Sync with HEAD.
 1.12.2.6  31-Jul-2008  simonb Sync with head.
 1.12.2.5  28-Jul-2008  simonb Sync with head.
 1.12.2.4  22-Jul-2008  simonb Sync with head.
 1.12.2.3  18-Jul-2008  simonb Sync with head.
 1.12.2.2  03-Jul-2008  simonb Sync with head.
 1.12.2.1  27-Jun-2008  simonb Sync with head.
 1.16.2.2  13-Dec-2008  haad Update haad-dm branch to haad-dm-base2.
 1.16.2.1  19-Oct-2008  haad Sync with HEAD.
 1.27.4.4  09-Jul-2009  snj Apply patch (requested by reinoud in ticket #852):
Numerous UDF improvements:
- a much saner rmw backend
- free space management done right
- the high system time usage when encountering huge numbers of nodes
are a thing of the past due to refactoring
- various small fixes
 1.27.4.3  18-Feb-2009  snj branches: 1.27.4.3.4;
Pull up following revision(s) (requested by reinoud in ticket #447):
sys/fs/udf/udf.h: revision 1.30
sys/fs/udf/udf_readwrite.c: revision 1.10
sys/fs/udf/udf_strat_direct.c: revision 1.8
sys/fs/udf/udf_strat_rmw.c: revision 1.18
sys/fs/udf/udf_strat_sequential.c: revision 1.8
sys/fs/udf/udf_subr.c: revision 1.85
sys/fs/udf/udf_subr.h: revision 1.13
Implement session closure on sequential media when requested on mount. It will
allow CD-ROM/DVD-ROM/DB-ROM drives to read the media while still allowing them
to be appended later. It can also be seen as a way to make mountable
snapshots.
 1.27.4.2  10-Dec-2008  snj Pull up following revision(s) (requested by reinoud in ticket #174):
sys/fs/udf/udf_strat_rmw.c: revision 1.15
sys/fs/udf/udf.h: revision 1.29
Fix two buf_init()'s that are not needed since they were already initialised
by getiobuf().
Also remove ancient IPL_BIO reference from the time it was called from
interrupt context instead of the current soft interrupt.
 1.27.4.1  06-Dec-2008  snj Pull up following revision(s) (requested by reinoud in ticket #167):
sys/fs/udf/udf.h: revision 1.28
sys/fs/udf/udf_strat_rmw.c: revision 1.13
Two fold commit:
1) Enhance write speed significantly on RMW media like CD-RW, DVD-RW but also
on the DVD+RW and all other ECC blocked media. Significant speedups of access
to the device for say compilation on the DVD. Streaming copy is also still at
maximum speed though vast amounts of directory copy work can show side effects
that appear it to slow down but are actually logical when you consider that
most small files are embedded into the descriptors itself.
2) explicit wait for the created RMW thread to spinup
 1.27.4.3.4.1  21-Apr-2010  matt sync to netbsd-5
 1.27.2.2  03-Mar-2009  skrll Sync with HEAD.
 1.27.2.1  19-Jan-2009  skrll Sync with HEAD.
 1.30.2.1  23-Jul-2009  jym Sync with HEAD.
 1.38.2.2  22-Oct-2010  uebayasi Sync with HEAD (-D20101022).
 1.38.2.1  30-Apr-2010  uebayasi Sync with HEAD.
 1.41.2.1  05-Mar-2011  rmind sync with head
 1.44.2.2  22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.44.2.1  30-Oct-2012  yamt sync with head
 1.45.4.1  18-May-2014  rmind sync with head
 1.45.2.2  03-Dec-2017  jdolecek update from HEAD
 1.45.2.1  20-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.47.2.3  29-May-2016  skrll Sync with HEAD
 1.47.2.2  22-Sep-2015  skrll Sync with HEAD
 1.47.2.1  06-Apr-2015  skrll Sync with HEAD

RSS XML Feed