branches: 1.1.4; 1.1.16; 1.1.22;
Initial import of a UDF file system implementation for NetBSD.

The code supports read access to all media types that CD/DVD type drives
can recognize including DVD-RAM and BD- drives as well as harddisc partions
and vnd devices. UDF versions upto the latest 2.60 are to be supported
though due to lack of test media version 2.50 and 2.60 are not implemented
yet though easy to add. Both open and closed media are supported.

Write access is planned and in preparation. To facilitate this some hooks
are present in the code that are not strictly needed in a read-only
implementation but which allow writing to be added more easily.

Implemented and tested media types are CD-ROM, CD-R, CD-RW, CD-MRW,
DVD-ROM, DVD*R, DVD*RW, DVD+MRW but the same code can also read DVD-RAM,
HD-DVD and BluRay discs. Also vnd devices have been tested with several
sector sizes.

Discs created and written by UDFclient, Nero's InCD and Roxio's
DirectCD/Drag2Disc read fine.

Replace the variable field data[0] to data[1] to avoid undefined behaviour.

Sync with UDFclient

Avoid undefined behavior in ecma167-udf.h

Define the UDF_EXT_* symbols as unsigned values.
This prevents undefined behavior of altering the signedness bit in a part
of them. Swap the remaining symbols to unsigned values to keep the
consistency.

Required by GCC when building with Undefined Behavior Sanitizer.

branches: 1.14.52; 1.14.54;
English checked and corrected, courtisy of
Igor Sobrado <sobrado@orion.ciencias.uniovi.es> as noted in OpenBSD that has a
copy of this file.

Fix typo in comments, no code change.

Make ecma167 header file compilable by pcc

branches: 1.11.4; 1.11.8; 1.11.12;
Add metadata bitmap ICB file type

branches: 1.10.2;
Fix attribute length calculation and fix naming convention.

Elaborate file times extension times definition.

branches: 1.8.2;
Import writing part of the UDF file system making optical media like CD's
and DVD's behave like floppy discs. Writing is supported upto and including
version 2.01; version 2.50 and 2.60 will follow.

Also extending the UDF implementation to support symbolic links and
hardlinks.

Added are the mmcformat(8) tool to format rewritable CD/DVD discs and
newfs_udf(8).

Limitations:
all operations can be performed on the file system though the
sheduling is currently optimised for archiving workloads.

mv(1)/rename(2) is currently only implemented for non-directories.

branches: 1.7.6; 1.7.8; 1.7.10; 1.7.12;
Convert many of the uses of __attribute__ to equivalent
__packed, __unused and __dead macros from cdefs.h

branches: 1.6.20; 1.6.26; 1.6.28; 1.6.32;
Add UDF_ICB_FILETYPE_REALTIME type as found on UDF 2.50 discs. Also revert
a multiplication order to enforce 64 bit math.

branches: 1.5.2;
Add extended attribute numbers associated with the two defined structures
for times and device specification.

branches: 1.4.2; 1.4.4; 1.4.6;
Add shortcut for struct long_ad 's Unique ID hint that covers the lower 32
bits of the targeted files 64 bits Unique ID for recovery purposes.

Fix small though important errors in the Ecma-167 structure definitions.

branches: 1.2.4; 1.2.6; 1.2.16; 1.2.18;
Replace reinclude protection names with less generic versions.

Initial import of a UDF file system implementation for NetBSD.

The code supports read access to all media types that CD/DVD type drives
can recognize including DVD-RAM and BD- drives as well as harddisc partions
and vnd devices. UDF versions upto the latest 2.60 are to be supported
though due to lack of test media version 2.50 and 2.60 are not implemented
yet though easy to add. Both open and closed media are supported.

Write access is planned and in preparation. To facilitate this some hooks
are present in the code that are not strictly needed in a read-only
implementation but which allow writing to be added more easily.

Implemented and tested media types are CD-ROM, CD-R, CD-RW, CD-MRW,
DVD-ROM, DVD*R, DVD*RW, DVD+MRW but the same code can also read DVD-RAM,
HD-DVD and BluRay discs. Also vnd devices have been tested with several
sector sizes.

Discs created and written by UDFclient, Nero's InCD and Roxio's
DirectCD/Drag2Disc read fine.

Define filesystem attributes with vfs dependency.

Implement udf_rename() using the new genfs_rename() framework.

Fixes PR kern/47986

branches: 1.4.10; 1.4.20; 1.4.24;
Don't generate unused fs_thefs.h headers.

branches: 1.3.14;
Remove all remaining traces of UDF_READWRITE since it has been removed as
option.

branches: 1.2.4;
Import writing part of the UDF file system making optical media like CD's
and DVD's behave like floppy discs. Writing is supported upto and including
version 2.01; version 2.50 and 2.60 will follow.

Also extending the UDF implementation to support symbolic links and
hardlinks.

Added are the mmcformat(8) tool to format rewritable CD/DVD discs and
newfs_udf(8).

Limitations:
all operations can be performed on the file system though the
sheduling is currently optimised for archiving workloads.

mv(1)/rename(2) is currently only implemented for non-directories.

branches: 1.1.4; 1.1.16; 1.1.22; 1.1.72; 1.1.74; 1.1.76; 1.1.78;
Initial import of a UDF file system implementation for NetBSD.

The code supports read access to all media types that CD/DVD type drives
can recognize including DVD-RAM and BD- drives as well as harddisc partions
and vnd devices. UDF versions upto the latest 2.60 are to be supported
though due to lack of test media version 2.50 and 2.60 are not implemented
yet though easy to add. Both open and closed media are supported.

Write access is planned and in preparation. To facilitate this some hooks
are present in the code that are not strictly needed in a read-only
implementation but which allow writing to be added more easily.

Implemented and tested media types are CD-ROM, CD-R, CD-RW, CD-MRW,
DVD-ROM, DVD*R, DVD*RW, DVD+MRW but the same code can also read DVD-RAM,
HD-DVD and BluRay discs. Also vnd devices have been tested with several
sector sizes.

Discs created and written by UDFclient, Nero's InCD and Roxio's
DirectCD/Drag2Disc read fine.

Move the include of <sys/malloc.h> to where it's truly needed.

Convert UDF file system code from the depricated tsleep(9)/wakeup(9) to the MP
friendly condvar(9). No functional change intended.

fix various typos in comments, documentation and messages.
mainly s/paramater/parameter/ and s/reduntant/redundant/.

fix various typos in comments and log messages.

Cleanup VAT writout. To prevent issues with the sequential writing strategy
trying to write on blocks that are lost due to the synchronisation, don't just
bluntly do synchronize device caches, but split out on strategies.

Rework VAT searching on recordable media. It is now a lot more resilliant to
errors and it allows for VAT searching on crashed writeouts.

While here, make sure the node pointer is always initialised in
udf_get_node().

Use vfs_vnode_iterator for udf_do_sync.

- Build rb_tree to get an ordered list of nodes, sync them and
clean the tree.

- Stop abusing mntvnode_lock to serialize udf_do_sync, use new
mutex ump->sync_lock instead.

Remove dirtynodes_cv, it is only used for timed waits without any signals.

Replace the cv_timedwait with kpause.

Change udf to vcache, keyed on udf_node->loc.loc.

branches: 1.47.2;
As pointed out by wiz@ prevent a possible attack or corruption that results in
an endless loop of indirect descriptors being processed.

The number of indirect descriptors followed is now maximized.
While here, also fix a use-after-free bug!

fix unused variable warnings

branches: 1.45.2; 1.45.4;
gcc -fno-common fallout

branches: 1.44.2;
- rename UDF_MAX_NAMELEN -> UDF_MAXNAMLEN
- use NAME_MAX instead of MAXNAMLEN

Rename rb.h to rbtree.h, as it is more appropriate (c.f. ptree.h). Also
helps find code that hasn't been updated to use the new rbtree API.

Fixes/improvements to RB-tree implementation:
1. Fix inverted node order, so that negative value from comparison operator
would represent lower (left) node, and positive - higher (right) node.
2. Add an argument (i.e. "context"), passed to comparison operators.
3. Change rb_tree_insert_node() to return a node - either inserted one or
already existing one.
4. Amend the interface to manipulate the actual object, instead of the
rb_node (in a similar way as Patricia-tree interface does).
5. Update all RB-tree users accordingly.

XXX: Perhaps rename rb.h to rbtree.h, since cleaning-up..

1-3 address the PR/43488 by Jeremy Huddleston.

Passes RB-tree regression tests.
Reviewed by: matt@, christos@

branches: 1.41.2;
First part of shrinking/growing metadata partition support:

- extending the metadata partition

Still to follow:
- sparsify metadata partition
- growing the metadata partition
- unsparsifying metadata partition

Rename metadata partition parameters to be less generic and add the flags.

Extract UDF metadata partition parameters as already done for VAT and sparable
partitions.

branches: 1.38.2;
Typo fix

Remove unused LIST_ENTRY()'s now replaced by the rbtree.

Replace the old hashtable and sorted list implemenation by a RB-tree.

Benefits are significant speed improvements on node creation/insertion while
keeping the lookup times low and still allowing sequential iteration over the
nodes.

Reduce the number of active ecclines; low memory systems could get into
trouble otherwise.

XXX this value should be computed eventually.

Commit first stage of free-space accounting. It is estimating the underlimit
of free blocks on the device and when free blocks are getting tight it tries
to readjust/recalculate that value by syncing the FS.

Second stage will be resizing the data/metadata partitions.

Bump the NetBSD kernel UDF id to version 0.5

Add RESERVE bitmask flag to DEBUG print constants adding another 0 prefix to
the constants already defined.

Remove now unused UDF_DIRHASH constants

branches: 1.30.2;
Implement session closure on sequential media when requested on mount. It will
allow CD-ROM/DVD-ROM/DB-ROM drives to read the media while still allowing them
to be appended later. It can also be seen as a way to make mountable
snapshots.

Fix two buf_init()'s that are not needed since they were already initialised
by getiobuf().

Also remove ancient IPL_BIO reference from the time it was called from
interrupt context instead of the current soft interrupt.

Two fold commit:

1) Enhance write speed significantly on RMW media like CD-RW, DVD-RW but also
on the DVD+RW and all other ECC blocked media. Significant speedups of access
to the device for say compilation on the DVD. Streaming copy is also still at
maximum speed though vast amounts of directory copy work can show side effects
that appear it to slow down but are actually logical when you consider that
most small files are embedded into the descriptors itself.

2) explicit wait for the created RMW thread to spinup

branches: 1.27.2; 1.27.4;
Switch UDF over to the vfs_dirhash.c dirhash code.

Rewrite allocation scheme's determination and implementation replacing the
old somewhat naive selection scheme that didn't allow different allocation
settings for nodes, directory information (FIDs) and data.

Also fix some curious side-effects of atime updates on RMW devices.

Remove comments on `need to read in'; they are read in.

Streamline allocation to prepare it for metadata partition. Also add
preliminary Metadata partition write support but its disabled still since
its not finished yet and not functioning correctly. All other formats are
checked and should work fine.

Unify naming scheme and source code beautify for better readablility.

Clarify the current spacemap read/write code is reading/writing physical
partition space maps. This in preperation of Metadata spacetable
reading/writeout work.

Limit the UDF dirhash to a configurable amount of memory. The performance
penalty even with a limited dirhash of 128kb is acceptable but it should be
set by machine memory size though.

Use ecma-167 constants instead of homegrown to prevent double mapping.

Choose which time of allocation descriptors to use on new nodes based on
the number of partition mappings.

Implement directory hashing to speed up directory traversals. Speed
improvements of at least 4 times in untarring and roughly 100 to 500 times
on file creation in big directories. Lookup of files was O(n*n) and is now
O(1) even for file creation. Free spaces in the directory are kept in a
seperate list for fast file creation.

The postmark benchmark gives:

UDF old:
pm>set transactions 2000
pm>set number 3000
pm>run
Creating files...Done
Performing transactions..........Done
Deleting files...Done
Time:
1593 seconds total
681 seconds of transactions (2 per second)

Files:
3956 created (2 per second)
Creation alone: 3000 files (4 per second)
Mixed with transactions: 956 files (1 per second)
990 read (1 per second)
1010 appended (1 per second)
3956 deleted (2 per second)
Deletion alone: 2912 files (9 per second)
Mixed with transactions: 1044 files (1 per second)

Data:
5.26 megabytes read (3.38 kilobytes per second)
21.93 megabytes written (14.10 kilobytes per second)
pm>



UDF new:
pm>set transactions 2000
pm>set number 3000
pm>run
Creating files...Done
Performing transactions..........Done
Deleting files...Done
Time:
19 seconds total
3 seconds of transactions (666 per second)

Files:
3956 created (208 per second)
Creation alone: 3000 files (230 per second)
Mixed with transactions: 956 files (318 per second)
990 read (330 per second)
1010 appended (336 per second)
3956 deleted (208 per second)
Deletion alone: 2912 files (970 per second)
Mixed with transactions: 1044 files (348 per second)

Data:
5.26 megabytes read (283.66 kilobytes per second)
21.93 megabytes written (1.15 megabytes per second)

Support for allocation extents descriptor writing to allow for arbitrary
heavily fragmented files.

Also fixing some (rare) allocation bugs and function name streamlining.

Tested on harddisc, CD-RW and CD-R i.e. all three basic backend classes.

branches: 1.16.2;
Add node-rebuild flag to indicate that the node was rebuild up and thus the
allocation extents have changed.

Create new udf_verbose class.

Create sysctl entries during module initialisation and destroy them
appropriately.

Many of these file systems are now ready for modularisation.

Bump maximum number of allocation extents descriptors in a file/directory;
10 did seem far too many, but aparently ``640 kb should be enough for
everyone''.

branches: 1.12.2;
Introduce new UDF debug class and split rmw strategy debugging prints.

Import writing part of the UDF file system making optical media like CD's
and DVD's behave like floppy discs. Writing is supported upto and including
version 2.01; version 2.50 and 2.60 will follow.

Also extending the UDF implementation to support symbolic links and
hardlinks.

Added are the mmcformat(8) tool to format rewritable CD/DVD discs and
newfs_udf(8).

Limitations:
all operations can be performed on the file system though the
sheduling is currently optimised for archiving workloads.

mv(1)/rename(2) is currently only implemented for non-directories.

branches: 1.10.18; 1.10.20; 1.10.22; 1.10.24;
Merge from vmlocking:

- Split vnode::v_flag into three fields, depending on field locking.
- simple_lock -> kmutex in a few places.
- Fix some simple locking problems.

branches: 1.9.6; 1.9.18; 1.9.20; 1.9.22;
Implement read-only access to UDF 2.50 and UDF 2.60 discs that use metadata
partition mappings. This brings the NetBSD implementation to read all discs
upto date and will read all discs currently defined by OSTA.

branches: 1.8.2;
Add `vfs.udf.verbose' sysctl when running a DEBUG kernel to dynamically set
the udf_verbose variable. So when something goes wrong, it can be examined
on the spot without needing to reboot a new kernel and possibly loosing
state.

Make VOP_LOOKUP's O(1) instead of O(n log(n)) giving a speedup of 7500% on
a 1400 file directory.

branches: 1.6.2; 1.6.4; 1.6.6;
Fix pool allocation/freeing problem for once and for all correctly. Since
it was initialised quite late due to its reliance on disc data the mount
process could have stopped before initialising and thus could panic again
only now for uninitialising an not initialised pool! *sigh*

Make a better guess on VAT locations so discs that are semi-standard
formatted can be recognised. This also fixes the bug that prevented UDF to
mount an empty CD-R/DVD*R disc.

branches: 1.4.4; 1.4.6; 1.4.16; 1.4.18;
protect the whole file, not parts of it.

Fix header re-inclusion protection defines to a less generic naming
scheme.

Fix $NetBSD$ tags. (sorry)

Initial import of a UDF file system implementation for NetBSD.

The code supports read access to all media types that CD/DVD type drives
can recognize including DVD-RAM and BD- drives as well as harddisc partions
and vnd devices. UDF versions upto the latest 2.60 are to be supported
though due to lack of test media version 2.50 and 2.60 are not implemented
yet though easy to add. Both open and closed media are supported.

Write access is planned and in preparation. To facilitate this some hooks
are present in the code that are not strictly needed in a read-only
implementation but which allow writing to be added more easily.

Implemented and tested media types are CD-ROM, CD-R, CD-RW, CD-MRW,
DVD-ROM, DVD*R, DVD*RW, DVD+MRW but the same code can also read DVD-RAM,
HD-DVD and BluRay discs. Also vnd devices have been tested with several
sector sizes.

Discs created and written by UDFclient, Nero's InCD and Roxio's
DirectCD/Drag2Disc read fine.

fix various typos in comments.

branches: 1.47.10;
fix various small typos, mainly in comments.

Revere modification of initializer; it can lead to race conditions where two
allocation would pick the `empty' space causing a panic later on.

On freeing a virtual address in the VAT, use the correct value; this might
confuse other implementations who can reject the VAT on this.

fix typos in comments, mainly s/extention/extension/ and s/sufficent/sufficient/

fix various typos in comments and log messages.

Fix all remaining typos, mainly in comments but also in few definitions and log messages, reported by me in PR kern/54889.
Also fixed some additional typos in comments, found on review of same files or typos.

branches: 1.41.6;
PR kern/54759 (vm.ubc_direct deadlock when read()/write() into mapping of itself)

- Add new flag UBC_ISMAPPED which tells ubc_uiomove() the object is mmap()ed
somewhere. Use it to decide whether to do direct-mapped copy, rather than
poking around directly in the vnode in ubc_uiomove(), which is ugly and
doesn't work for tmpfs. It would be nicer to contain all this in UVM but
the filesystem provides the needed locking here (VV_MAPPED) and to
reinvent that would suck more.

- Rename UBC_UNMAP_FLAG() to UBC_VNODE_FLAGS(). Pass in UBC_ISMAPPED where
appropriate.

branches: 1.40.4; 1.40.10;
remove M_CANFAIL flag for malloc(9) - it was completely ignored, so had
actually no effect

branches: 1.39.14; 1.39.16;
PR/51777: David Binderman: Remove unused computation

branches: 1.38.2;
Remove dirtynodes_cv, it is only used for timed waits without any signals.

Replace the cv_timedwait with kpause.

Relax the KASSERT since an equal size is no issue

branches: 1.36.6;
used __diagused where appropriate.

fix unused variable warnings

Fix 32 bit issue in main file read-in function. On both 32 bit and 64 bit
hosts a missing cast would result in `garbage' after the 4Gbyte limit.

On growing a node that was recorded inside the (E)FE, don't forget to set the
size of the node to the NEW size. This was reported by the `t_io
udf_shrinkfile' testcase.

Fixes PR kern/47985

branches: 1.32.2; 1.32.8; 1.32.12; 1.32.14; 1.32.16; 1.32.22;
Rename uvm_vnp_zerorange(struct vnode *, off_t, size_t) to
ubc_zerorange(struct uvm_object *, off_t, size_t, int) changing
the first argument to an uvm_object and adding a flags argument.

Modify tmpfs_reg_resize() to zero the backing store (aobj) instead
of the vnode. Ubc_purge() no longer panics when unmounting tmpfs.

Keep uvm_vnp_zerorange() until the next kernel version bump.

branches: 1.31.6;
Metadata partition (v2.50+) bugfix commit :

* fix copying of the extents of the metadata node to the metadatamirror node;
it was not copying all extents.

* fix truncing metadata partition:
* fix endian conversions
* fix information length calculation so its truncated to the right length!

* allow for setting maximum extent length in extent merging. This is needed
since extents in the metadata partition files are only to be in allocation
unit sizes.

* adjust grow and shrink node to set the granularity of the maximum length of
an extent when encountering a metadatafile or metadatamirror file.

Growing the metadata partition is not yet implemented. Be so kind to report
this as unimplemented when the debug flag asks for it!

branches: 1.29.2;
Typo and style

First part of shrinking/growing metadata partition support:

- extending the metadata partition

Still to follow:
- sparsify metadata partition
- growing the metadata partition
- unsparsifying metadata partition

branches: 1.27.2;
Fix corner-case in truncing files. It could forget to free the last block.
This would result in a free-space map with one block orphaned.

Commit first stage of free-space accounting. It is estimating the underlimit
of free blocks on the device and when free blocks are getting tight it tries
to readjust/recalculate that value by syncing the FS.

Second stage will be resizing the data/metadata partitions.

Fix nitpicky spacing and debug printout

Remove unneeded (and unused) inclusion of "opt_quota.h"

Account for the logical blocks recorded by the extent in the right endian!

A descriptor's CRC lenth is 16 bit so use udf_rw16() on tag.desc_crc_len.

Fix endian related bug when using discs with a meta-data partition on
big-endian machines.

Fix possible overshoot when allocating from a space bitmap when ffs(3) returns
a too big offset for the bitmap.

branches: 1.19.2;
The great QUOTA purge of '08: get rid of those #ifdef QUOTA and
#include "opt_quota.h" which do exactly nothing. Speeds up kernel
compilation by 1.375*10^-20001 seconds. But leave the most moxious
comment in msdosfs_vfsops untouched.

branches: 1.18.2; 1.18.4;
Remove all remaining traces of UDF_READWRITE since it has been removed as
option.

Rewrite allocation scheme's determination and implementation replacing the
old somewhat naive selection scheme that didn't allow different allocation
settings for nodes, directory information (FIDs) and data.

Also fix some curious side-effects of atime updates on RMW devices.

Streamline allocation to prepare it for metadata partition. Also add
preliminary Metadata partition write support but its disabled still since
its not finished yet and not functioning correctly. All other formats are
checked and should work fine.

Clarify not implemented allocation schemes; mostly UDF 2.50
HD-DVD/BluRay-RE and UDF 2.60 for BluRay-R

Use the allocation descriptor types selected on mount when growing from an
internal to an extent recording.

Hopefully fix gcc type puned warning.

Make UDF allocation-sanity check optional even if the DEBUG flag is
specified. This should be sysctl-able when compiled with the DEBUG flag
eventually.

Support for allocation extents descriptor writing to allow for arbitrary
heavily fragmented files.

Also fixing some (rare) allocation bugs and function name streamlining.

Tested on harddisc, CD-RW and CD-R i.e. all three basic backend classes.

branches: 1.10.2;
Fix rare allocation problem when a fragmented file is truncated to a
non-zero length and the last extent was allocated.

Update node sanity check system and redo allocation recording fixing
corruptions that could take place when overwriting sparse files.

Still one rare corruption possible where blocks are accidentally marked
free, but the cause is not yet found and looking at the pattern it won't
happen in every day use.

Fix serious allocation recording bug if overwriting piece that was
allocated before. Fixes regression test on random overwriting data.

Also fix endian issues and add extra documentation to variables.

When shrinking the file, do clean the freed part in the descriptor; all
unused space need to be zero filled.

Count the length correctly when encountering an allocation extension
redirect in the sanity checking.

Fix lookup problems in very fragmented metadata partitions that use
allocation extent descriptors.

Fix extended allocation descriptor handling in translate file extent

Fix allocation extensions descriptor reading and do free them on node
destruction!

branches: 1.2.2; 1.2.4; 1.2.6;
Remove bogus KASSERT(); if we shrink the file, the number of allocated
blocsk are of course not the same... duh!

branches: 1.1.2; 1.1.4;
Import writing part of the UDF file system making optical media like CD's
and DVD's behave like floppy discs. Writing is supported upto and including
version 2.01; version 2.50 and 2.60 will follow.

Also extending the UDF implementation to support symbolic links and
hardlinks.

Added are the mmcformat(8) tool to format rewritable CD/DVD discs and
newfs_udf(8).

Limitations:
all operations can be performed on the file system though the
sheduling is currently optimised for archiving workloads.

mv(1)/rename(2) is currently only implemented for non-directories.

Remove closes 3 and 4 from my copyright.

ANSI-fy. No idea why they weren't so in the first place.

Fix *serious* bug in bswapping definitions for big endian machines.

branches: 1.5.8; 1.5.12; 1.5.16;
Import writing part of the UDF file system making optical media like CD's
and DVD's behave like floppy discs. Writing is supported upto and including
version 2.01; version 2.50 and 2.60 will follow.

Also extending the UDF implementation to support symbolic links and
hardlinks.

Added are the mmcformat(8) tool to format rewritable CD/DVD discs and
newfs_udf(8).

Limitations:
all operations can be performed on the file system though the
sheduling is currently optimised for archiving workloads.

mv(1)/rename(2) is currently only implemented for non-directories.

branches: 1.4.10; 1.4.16; 1.4.66; 1.4.68; 1.4.70; 1.4.72;
remove trailing newlines

branches: 1.3.2; 1.3.4; 1.3.6;
De __P() prototypes.

Fix header re-inclusion protection defines to a less generic naming
scheme.

Initial import of a UDF file system implementation for NetBSD.

The code supports read access to all media types that CD/DVD type drives
can recognize including DVD-RAM and BD- drives as well as harddisc partions
and vnd devices. UDF versions upto the latest 2.60 are to be supported
though due to lack of test media version 2.50 and 2.60 are not implemented
yet though easy to add. Both open and closed media are supported.

Write access is planned and in preparation. To facilitate this some hooks
are present in the code that are not strictly needed in a read-only
implementation but which allow writing to be added more easily.

Implemented and tested media types are CD-ROM, CD-R, CD-RW, CD-MRW,
DVD-ROM, DVD*R, DVD*RW, DVD+MRW but the same code can also read DVD-RAM,
HD-DVD and BluRay discs. Also vnd devices have been tested with several
sector sizes.

Discs created and written by UDFclient, Nero's InCD and Roxio's
DirectCD/Drag2Disc read fine.

Switch files copyright Reinoud Zandijk from BSD 4 clause to BSD 2 clause.

OK'd by reinoud in email (from 4 Feb 2019).

branches: 1.3.4; 1.3.16; 1.3.22; 1.3.166;
Fix header re-inclusion protection defines to a less generic naming
scheme.

Fix $NetBSD$ tags. (sorry)

Initial import of a UDF file system implementation for NetBSD.

The code supports read access to all media types that CD/DVD type drives
can recognize including DVD-RAM and BD- drives as well as harddisc partions
and vnd devices. UDF versions upto the latest 2.60 are to be supported
though due to lack of test media version 2.50 and 2.60 are not implemented
yet though easy to add. Both open and closed media are supported.

Write access is planned and in preparation. To facilitate this some hooks
are present in the code that are not strictly needed in a read-only
implementation but which allow writing to be added more easily.

Implemented and tested media types are CD-ROM, CD-R, CD-RW, CD-MRW,
DVD-ROM, DVD*R, DVD*RW, DVD+MRW but the same code can also read DVD-RAM,
HD-DVD and BluRay discs. Also vnd devices have been tested with several
sector sizes.

Discs created and written by UDFclient, Nero's InCD and Roxio's
DirectCD/Drag2Disc read fine.

Translate 254/255 compID values used for deleted entries as per UDF spec. This
allows the printing of the names of deleted file entries.

Support tools build.

branches: 1.9.12; 1.9.22; 1.9.26;
Ansify function definitions w/o arguments. Generated with sed.

ANSIfy another 1261 function definitions.
The only ones left in sys are beyond by sed script!
(or in sys/dist or sys/external)
Mostly they have function pointer parameters.

branches: 1.7.6; 1.7.8; 1.7.12; 1.7.16;
Implement wide char support for UDF by properly implementing unicode-16 to
UTF-8 like cd9660 and ntfs already do.

Import writing part of the UDF file system making optical media like CD's
and DVD's behave like floppy discs. Writing is supported upto and including
version 2.01; version 2.50 and 2.60 will follow.

Also extending the UDF implementation to support symbolic links and
hardlinks.

Added are the mmcformat(8) tool to format rewritable CD/DVD discs and
newfs_udf(8).

Limitations:
all operations can be performed on the file system though the
sheduling is currently optimised for archiving workloads.

mv(1)/rename(2) is currently only implemented for non-directories.

branches: 1.5.8; 1.5.10; 1.5.12; 1.5.14;
use __KERNEL_RCSID() instead of __RCSID()

branches: 1.4.18; 1.4.26; 1.4.28; 1.4.30;
TRUE -> true, FALSE -> false

branches: 1.3.4; 1.3.10;
The local variable `index' seems shadowing an inbuild gcc variable (???)
when compiling from userland (kernel ok) so rename the variable to Index to
avoid the confusing warning.

branches: 1.2.4;
remove ; from };

branches: 1.1.4; 1.1.6; 1.1.12;
Initial import of a UDF file system implementation for NetBSD.

The code supports read access to all media types that CD/DVD type drives
can recognize including DVD-RAM and BD- drives as well as harddisc partions
and vnd devices. UDF versions upto the latest 2.60 are to be supported
though due to lack of test media version 2.50 and 2.60 are not implemented
yet though easy to add. Both open and closed media are supported.

Write access is planned and in preparation. To facilitate this some hooks
are present in the code that are not strictly needed in a read-only
implementation but which allow writing to be added more easily.

Implemented and tested media types are CD-ROM, CD-R, CD-RW, CD-MRW,
DVD-ROM, DVD*R, DVD*RW, DVD+MRW but the same code can also read DVD-RAM,
HD-DVD and BluRay discs. Also vnd devices have been tested with several
sector sizes.

Discs created and written by UDFclient, Nero's InCD and Roxio's
DirectCD/Drag2Disc read fine.

s/dependancies/dependencies/ and s/independantly/independently/ in comments and
distrib notes.

Import writing part of the UDF file system making optical media like CD's
and DVD's behave like floppy discs. Writing is supported upto and including
version 2.01; version 2.50 and 2.60 will follow.

Also extending the UDF implementation to support symbolic links and
hardlinks.

Added are the mmcformat(8) tool to format rewritable CD/DVD discs and
newfs_udf(8).

Limitations:
all operations can be performed on the file system though the
sheduling is currently optimised for archiving workloads.

mv(1)/rename(2) is currently only implemented for non-directories.

branches: 1.3.10; 1.3.16; 1.3.66; 1.3.68; 1.3.70; 1.3.72;
remove trailing newlines

branches: 1.2.2; 1.2.4; 1.2.6;
Replace reinclude protection names with less generic versions.

Initial import of a UDF file system implementation for NetBSD.

The code supports read access to all media types that CD/DVD type drives
can recognize including DVD-RAM and BD- drives as well as harddisc partions
and vnd devices. UDF versions upto the latest 2.60 are to be supported
though due to lack of test media version 2.50 and 2.60 are not implemented
yet though easy to add. Both open and closed media are supported.

Write access is planned and in preparation. To facilitate this some hooks
are present in the code that are not strictly needed in a read-only
implementation but which allow writing to be added more easily.

Implemented and tested media types are CD-ROM, CD-R, CD-RW, CD-MRW,
DVD-ROM, DVD*R, DVD*RW, DVD+MRW but the same code can also read DVD-RAM,
HD-DVD and BluRay discs. Also vnd devices have been tested with several
sector sizes.

Discs created and written by UDFclient, Nero's InCD and Roxio's
DirectCD/Drag2Disc read fine.

fix various typos in comments and log messages.

s/checkum/checksum/ in comment.

Cleanup VAT writout. To prevent issues with the sequential writing strategy
trying to write on blocks that are lost due to the synchronisation, don't just
bluntly do synchronize device caches, but split out on strategies.

branches: 1.11.12; 1.11.30;
Welcome to 5.99.53! Merge rmind-uvmplock branch:

- Reorganize locking in UVM and provide extra serialisation for pmap(9).
New lock order: [vmpage-owner-lock] -> pmap-lock.

- Simplify locking in some pmap(9) modules by removing P->V locking.

- Use lock object on vmobjlock (and thus vnode_t::v_interlock) to share
the locks amongst UVM objects where necessary (tmpfs, layerfs, unionfs).

- Rewrite and optimise x86 TLB shootdown code, make it simpler and cleaner.
Add TLBSTATS option for x86 to collect statistics about TLB shootdowns.

- Unify /dev/mem et al in MI code and provide required locking (removes
kernel-lock on some ports). Also, avoid cache-aliasing issues.

Thanks to Andrew Doran and Joerg Sonnenberger, as their initial patches
formed the core changes of this branch.

branches: 1.10.6; 1.10.12;
Implement session closure on sequential media when requested on mount. It will
allow CD-ROM/DVD-ROM/DB-ROM drives to read the media while still allowing them
to be appended later. It can also be seen as a way to make mountable
snapshots.

The great QUOTA purge of '08: get rid of those #ifdef QUOTA and
#include "opt_quota.h" which do exactly nothing. Speeds up kernel
compilation by 1.375*10^-20001 seconds. But leave the most moxious
comment in msdosfs_vfsops untouched.

branches: 1.8.2; 1.8.4;
Remove all remaining traces of UDF_READWRITE since it has been removed as
option.

Fixup important bug in internal recorded data fixup. Although most unlikely
it could result in theory result in descriptor trashing.

On the performance side, it would try to fixup *every* descriptor even if
it wasn't an internally allocated one. Performance loss wasn't that big but
every bit helps.

Streamline allocation to prepare it for metadata partition. Also add
preliminary Metadata partition write support but its disabled still since
its not finished yet and not functioning correctly. All other formats are
checked and should work fine.

Force allocation extent descriptor CRC length to 8 for UDF versions prior
to version 2.01.

branches: 1.4.2;
Avoid panic when trying to mount a non existent block device like a
/dev/vnd0 that is not configured.

__FUNCTION__ -> __func__

If we're fixing up the internal attributes, is it then wise to check if
they are valid first? Might be on the safe side in the end but disabled for
now.

branches: 1.1.2; 1.1.4; 1.1.6; 1.1.8; 1.1.10;
Import writing part of the UDF file system making optical media like CD's
and DVD's behave like floppy discs. Writing is supported upto and including
version 2.01; version 2.50 and 2.60 will follow.

Also extending the UDF implementation to support symbolic links and
hardlinks.

Added are the mmcformat(8) tool to format rewritable CD/DVD discs and
newfs_udf(8).

Limitations:
all operations can be performed on the file system though the
sheduling is currently optimised for archiving workloads.

mv(1)/rename(2) is currently only implemented for non-directories.

Move the include of <sys/malloc.h> to where it's truly needed.

follow the steps of Andrew Doran (ad) commit and fix more s/loose/lose/ typos.
also s/beyound/beyond/ and few others along the way, mainly in comments.

Overhaul of the EVFILT_VNODE kevent(2) filter:

- Centralize vnode kevent handling in the VOP_*() wrappers, rather than
forcing each individual file system to deal with it (except VOP_RENAME(),
because VOP_RENAME() is a mess and we currently have 2 different ways
of handling it; at least it's reasonably well-centralized in the "new"
way).
- Add support for NOTE_OPEN, NOTE_CLOSE, NOTE_CLOSE_WRITE, and NOTE_READ,
compatible with the same events in FreeBSD.
- Track which kevent notifications clients are interested in receiving
to avoid doing work for events no one cares about (avoiding, e.g.
taking locks and traversing the klist to send a NOTE_WRITE when
someone is merely watching for a file to be deleted, for example).

In support of the above:

- Add support in vnode_if.sh for specifying PRE- and POST-op handlers,
to be invoked before and after vop_pre() and vop_post(), respectively.
Basic idea from FreeBSD, but implemented differently.
- Add support in vnode_if.sh for specifying CONTEXT fields in the
vop_*_args structures. These context fields are used to convey information
between the file system VOP function and the VOP wrapper, but do not
occupy an argument slot in the VOP_*() call itself. These context fields
are initialized and subsequently interpreted by PRE- and POST-op handlers.
- Version VOP_REMOVE(), uses the a context field for the file system to report
back the resulting link count of the target vnode. Return this in tmpfs,
udf, nfs, chfs, ext2fs, lfs, and ufs.

NetBSD 9.99.92.

VFS_VGET(), VFS_ROOT(), VFS_FHTOVP(): give them a "int lktype" argument, to
allow us to get shared locks (or no lock) on the returned vnode. Matches
FreeBSD.

branches: 1.12.20; 1.12.26;
Revert previous, it is actually needed. (builds are failing)

Do not uselessly include <sys/malloc.h>.

branches: 1.10.4; 1.10.8; 1.10.12;
Remove udf_node * as state variables and add the loop invariants UFS and tmpfs
carry to make it easier to debug.

First step in rewriting the genealogy case

udf_gro_genealogy() : source and destination should never be the same on call,
KASSERT it instead of checking for it.

vp is used, so don't (void) it

Rework udf_gro_rename() to be more linear instead of nesting if/else
branching.

In case there is trouble determining the number of entries in a directory,
return its not empty to be on the safe side.

Resort to the easiest way to check if a directory is deleted: the flag that
tells that there is no FID (dirent) pointing to it anymore.

Small cleanup

Remove the dirent checks and references. UDF doesn't need/use them on rename
so just to make sure, we remove all references/checks to not trigger bugs
thats are not really ours.

Implement udf_rename() using the new genfs_rename() framework.

Fixes PR kern/47986

Cleanup VAT writout. To prevent issues with the sequential writing strategy
trying to write on blocks that are lost due to the synchronisation, don't just
bluntly do synchronize device caches, but split out on strategies.

branches: 1.4.2;
Do not uselessly include <sys/malloc.h>.

branches: 1.3.24; 1.3.40;
The great QUOTA purge of '08: get rid of those #ifdef QUOTA and
#include "opt_quota.h" which do exactly nothing. Speeds up kernel
compilation by 1.375*10^-20001 seconds. But leave the most moxious
comment in msdosfs_vfsops untouched.

branches: 1.2.2; 1.2.4; 1.2.10;
Remove all remaining traces of UDF_READWRITE since it has been removed as
option.

branches: 1.1.2; 1.1.4; 1.1.6; 1.1.10; 1.1.12;
Import writing part of the UDF file system making optical media like CD's
and DVD's behave like floppy discs. Writing is supported upto and including
version 2.01; version 2.50 and 2.60 will follow.

Also extending the UDF implementation to support symbolic links and
hardlinks.

Added are the mmcformat(8) tool to format rewritable CD/DVD discs and
newfs_udf(8).

Limitations:
all operations can be performed on the file system though the
sheduling is currently optimised for archiving workloads.

mv(1)/rename(2) is currently only implemented for non-directories.

fix various typos in comments and log messages.

Convert UDF file system code from the depricated tsleep(9)/wakeup(9) to the MP
friendly condvar(9). No functional change intended.

s/adressing/addressing/

Cleanup VAT writout. To prevent issues with the sequential writing strategy
trying to write on blocks that are lost due to the synchronisation, don't just
bluntly do synchronize device caches, but split out on strategies.

Remove dubious vhold()/holdrele() from write_logvol_dscr strategies.

The udf_node already gets locked with UDF_LOCK_NODE() and udf_reclaim()
will wait for the lock to clear.

branches: 1.12.6;
used __diagused where appropriate.

fix unused variable warnings

branches: 1.10.12; 1.10.22; 1.10.26;
Retire IN_CALLBACK_ULK flagging since its not used anymore and the old
assertions on it would trigger without cause.

Remove dead code and double check if its allowed to write out non-sequential
there.

branches: 1.8.2;
Implement session closure on sequential media when requested on mount. It will
allow CD-ROM/DVD-ROM/DB-ROM drives to read the media while still allowing them
to be appended later. It can also be seen as a way to make mountable
snapshots.

The great QUOTA purge of '08: get rid of those #ifdef QUOTA and
#include "opt_quota.h" which do exactly nothing. Speeds up kernel
compilation by 1.375*10^-20001 seconds. But leave the most moxious
comment in msdosfs_vfsops untouched.

Allow for the exclusive selection of `direct' for rdonly mounts; its disabled
for now since its a lot slower than `rmw' access.

For archs that have trouble with `rmw' for whatever reason can so use it as a
scapegoat to allways mount savely rdonly though slower.

branches: 1.5.2; 1.5.4;
Remove all remaining traces of UDF_READWRITE since it has been removed as
option.

Rewrite allocation scheme's determination and implementation replacing the
old somewhat naive selection scheme that didn't allow different allocation
settings for nodes, directory information (FIDs) and data.

Also fix some curious side-effects of atime updates on RMW devices.

Streamline allocation to prepare it for metadata partition. Also add
preliminary Metadata partition write support but its disabled still since
its not finished yet and not functioning correctly. All other formats are
checked and should work fine.

Support for allocation extents descriptor writing to allow for arbitrary
heavily fragmented files.

Also fixing some (rare) allocation bugs and function name streamlining.

Tested on harddisc, CD-RW and CD-R i.e. all three basic backend classes.

branches: 1.1.2; 1.1.4; 1.1.6; 1.1.8; 1.1.10; 1.1.12;
Import writing part of the UDF file system making optical media like CD's
and DVD's behave like floppy discs. Writing is supported upto and including
version 2.01; version 2.50 and 2.60 will follow.

Also extending the UDF implementation to support symbolic links and
hardlinks.

Added are the mmcformat(8) tool to format rewritable CD/DVD discs and
newfs_udf(8).

Limitations:
all operations can be performed on the file system though the
sheduling is currently optimised for archiving workloads.

mv(1)/rename(2) is currently only implemented for non-directories.

Convert UDF file system code from the depricated tsleep(9)/wakeup(9) to the MP
friendly condvar(9). No functional change intended.

s/adressing/addressing/

s/helt/held+s/eroneously/erroneously/+s/splitted/split/+s/recommented/recommended/

Cleanup VAT writout. To prevent issues with the sequential writing strategy
trying to write on blocks that are lost due to the synchronisation, don't just
bluntly do synchronize device caches, but split out on strategies.

Remove dubious vhold()/holdrele() from write_logvol_dscr strategies.

The udf_node already gets locked with UDF_LOCK_NODE() and udf_reclaim()
will wait for the lock to clear.

diagused some variables.

remove identity assignment, found by brainy.

branches: 1.24.6;
used __diagused where appropriate.

fix unused variable warnings

branches: 1.22.12; 1.22.22; 1.22.26;
Fix alternating oddity in RMW strategy. Even when in the reading state it
would push out elements to fillup-read only when the time had come for them.
This could then trickle feed the read queue slowly, but fast enough to prevent
it from switching state.

Re-implement read-modify-write backend strategy. This version is a lot more
clean locking-wise and will consume less CPU power on needless moving-around.

Enhance dumping on buffer screwup. Also prevent races or invalid combinations
to exist for ecclines.

Retire IN_CALLBACK_ULK flagging since its not used anymore and the old
assertions on it would trigger without cause.

branches: 1.18.2;
Implement session closure on sequential media when requested on mount. It will
allow CD-ROM/DVD-ROM/DB-ROM drives to read the media while still allowing them
to be appended later. It can also be seen as a way to make mountable
snapshots.

g/c BUFQ_FOO() macros and use bufq_foo() directly.

The great QUOTA purge of '08: get rid of those #ifdef QUOTA and
#include "opt_quota.h" which do exactly nothing. Speeds up kernel
compilation by 1.375*10^-20001 seconds. But leave the most moxious
comment in msdosfs_vfsops untouched.

Fix two buf_init()'s that are not needed since they were already initialised
by getiobuf().

Also remove ancient IPL_BIO reference from the time it was called from
interrupt context instead of the current soft interrupt.

UDF_SHED_FREE state is now also handled by the state machine, so DO signal it.
Free'd up ecclines can now far more quicker be recycled.

Two fold commit:

1) Enhance write speed significantly on RMW media like CD-RW, DVD-RW but also
on the DVD+RW and all other ECC blocked media. Significant speedups of access
to the device for say compilation on the DVD. Streaming copy is also still at
maximum speed though vast amounts of directory copy work can show side effects
that appear it to slow down but are actually logical when you consider that
most small files are embedded into the descriptors itself.

2) explicit wait for the created RMW thread to spinup

Specify blobsize as the `pagingsize' of the eccline blob pool. This will relax
the requirements of the pagingsize.

Introduce an extra state to wich all pushed ecclines are pushed on. This
extra state has a holding time in wich no writeouts will be done
effectively collecting more sectors before its even sheduled for writing.

Explicitly set buffer's block numbers. Blocks were always read/written
correctly only the sheduling could go less than optimal.

branches: 1.9.2; 1.9.4;
Remove all remaining traces of UDF_READWRITE since it has been removed as
option.

Don't panic on read-errors but set flag that the packet is doomed. Its not
clear if i could read the individual sectors correctly since the entire
read packet is also one big ECC block.

Rewrite allocation scheme's determination and implementation replacing the
old somewhat naive selection scheme that didn't allow different allocation
settings for nodes, directory information (FIDs) and data.

Also fix some curious side-effects of atime updates on RMW devices.

Streamline allocation to prepare it for metadata partition. Also add
preliminary Metadata partition write support but its disabled still since
its not finished yet and not functioning correctly. All other formats are
checked and should work fine.

Support for allocation extents descriptor writing to allow for arbitrary
heavily fragmented files.

Also fixing some (rare) allocation bugs and function name streamlining.

Tested on harddisc, CD-RW and CD-R i.e. all three basic backend classes.

branches: 1.4.2; 1.4.4;
Change comments on marking buffer busy.

branches: 1.3.2; 1.3.4; 1.3.6; 1.3.8;
Introduce new UDF debug class and split rmw strategy debugging prints.

Add accidently removed lines when i pruned the code for release. Important
update.

Import writing part of the UDF file system making optical media like CD's
and DVD's behave like floppy discs. Writing is supported upto and including
version 2.01; version 2.50 and 2.60 will follow.

Also extending the UDF implementation to support symbolic links and
hardlinks.

Added are the mmcformat(8) tool to format rewritable CD/DVD discs and
newfs_udf(8).

Limitations:
all operations can be performed on the file system though the
sheduling is currently optimised for archiving workloads.

mv(1)/rename(2) is currently only implemented for non-directories.

Convert UDF file system code from the depricated tsleep(9)/wakeup(9) to the MP
friendly condvar(9). No functional change intended.

Spell approximately correctly.

Trailing whitespace

fix various typos in comments and log messages.

On switching from writing to reading explicitly synchronize the caches. It
isn't strictly needed but some devices in the wild will otherwise bluntly
ignore all reading commands resulting in a kernel and device lockup.

branches: 1.15.22;
Cleanup VAT writout. To prevent issues with the sequential writing strategy
trying to write on blocks that are lost due to the synchronisation, don't just
bluntly do synchronize device caches, but split out on strategies.

Remove dubious vhold()/holdrele() from write_logvol_dscr strategies.

The udf_node already gets locked with UDF_LOCK_NODE() and udf_reclaim()
will wait for the lock to clear.

branches: 1.13.6;
fix unused

fix unused variable warnings

branches: 1.11.8; 1.11.18; 1.11.22;
destroy mutex/cv before freeing the struct holding them, fixes
LOCKDEBUG panic

branches: 1.10.4;
Retire IN_CALLBACK_ULK flagging since its not used anymore and the old
assertions on it would trigger without cause.

Sequential writing goes before normal write. Its not likely to ever go wrong
but there could be a race otherwise.

branches: 1.8.2;
Implement session closure on sequential media when requested on mount. It will
allow CD-ROM/DVD-ROM/DB-ROM drives to read the media while still allowing them
to be appended later. It can also be seen as a way to make mountable
snapshots.

g/c BUFQ_FOO() macros and use bufq_foo() directly.

The great QUOTA purge of '08: get rid of those #ifdef QUOTA and
#include "opt_quota.h" which do exactly nothing. Speeds up kernel
compilation by 1.375*10^-20001 seconds. But leave the most moxious
comment in msdosfs_vfsops untouched.

branches: 1.5.2; 1.5.4;
Remove all remaining traces of UDF_READWRITE since it has been removed as
option.

Rewrite allocation scheme's determination and implementation replacing the
old somewhat naive selection scheme that didn't allow different allocation
settings for nodes, directory information (FIDs) and data.

Also fix some curious side-effects of atime updates on RMW devices.

Streamline allocation to prepare it for metadata partition. Also add
preliminary Metadata partition write support but its disabled still since
its not finished yet and not functioning correctly. All other formats are
checked and should work fine.

Support for allocation extents descriptor writing to allow for arbitrary
heavily fragmented files.

Also fixing some (rare) allocation bugs and function name streamlining.

Tested on harddisc, CD-RW and CD-R i.e. all three basic backend classes.

branches: 1.1.2; 1.1.4; 1.1.6; 1.1.8; 1.1.10; 1.1.12;
Import writing part of the UDF file system making optical media like CD's
and DVD's behave like floppy discs. Writing is supported upto and including
version 2.01; version 2.50 and 2.60 will follow.

Also extending the UDF implementation to support symbolic links and
hardlinks.

Added are the mmcformat(8) tool to format rewritable CD/DVD discs and
newfs_udf(8).

Limitations:
all operations can be performed on the file system though the
sheduling is currently optimised for archiving workloads.

mv(1)/rename(2) is currently only implemented for non-directories.

s/inititise/initialise/ in comment.

s/beyong/beyond/ in comment.

branches: 1.176.2;
fix various typos in comments and log messages.

s/substract/subtract/ in comments.

fix various typos in comments and messages.

Fix typos: s/trough/through/

fix various typos in comments and one output message.

fix various typos in comments, mainly origional->original,
extened->extended, incomming->incoming.

s/wich/which in comments.

Fix endian issue with UDF extended attribute handling

fix various typos in comments and output/log messages.

For bug-compatibility with Windows, the last VAT sector must be a multiple of
16/32 from the start of the track. To allow for scratches, write out at least
a 32 pieces.

fix various typos in comments, log messages and documentation.
mainly s/aparently/apparently/ and s/implmented/implemented/.

fix various typos in comments.

remove double "with" in comments and usage text. Also fix one typo.

s/implemenation/implementation/ in comment.

Fix typo: reader -> header

Fix serious issue with recordable media formatted with Win10. When closing it
after modification the VAT was written out corrupted making the disc
unreadable anymore on remount. Thanks for ig@ for spotting it in the wild!

While searching the VAT on recordable media, search the last sector too!
This fixes Win10 formatted discs from being mounted incorrectly.

s/alloction/allocation/

fix typos in word "successful".

fix typos in comments, mainly s/extention/extension/ and s/sufficent/sufficient/

s/helt/held+s/eroneously/erroneously/+s/splitted/split/+s/recommented/recommended/

fix various typos in comments and log messages.

Fix all remaining typos, mainly in comments but also in few definitions and log messages, reported by me in PR kern/54889.
Also fixed some additional typos in comments, found on review of same files or typos.

branches: 1.153.2;
more GCC 10 fixes.

mDNSResponder: another wrong return local address

dhcp: ignore a seemingly impossible stringop overflow

hpacel: avoid maybe uninitialised error that is wrong.

rsh: avoid impossible malloc(0)

udf: cast pointers through (uintptr_t) to fool invalid boundary checks

branches: 1.152.2;
Trailing whitespace

Fix spelling of preferred in comments. While here improve grammar in
one of the comments.

branches: 1.150.4;
udf_get_node(): fix typo and honor "lktype" argument.

Tighten up the locking around vp->v_iflag a little more after the recent
split of vmobjlock & v_interlock.

VFS_VGET(), VFS_ROOT(), VFS_FHTOVP(): give them a "int lktype" argument, to
allow us to get shared locks (or no lock) on the returned vnode. Matches
FreeBSD.

branches: 1.147.2;
Add newly created vnodes to the namei cache. The rest of the filesystems
already did that (or they don't support writing). Discussed in tech-kern.

branches: 1.146.2;
Fix typo in comment (s/seperate/separate/).

Add "void *extra" argument to vcache_new() so a file system may
pass more information about the file to create.

Welcome to 8.99.30

remove M_CANFAIL flag for malloc(9) - it was completely ignored, so had
actually no effect

Fix length calculation

Enhance logical volume integrity descriptor handling and provide
bug-compatibility for Linux mkudffs 1.3 images.

branches: 1.141.2;
Remove duplicate ;

add a genfs method to allow a file system to limit the range of pages
that are given to a single GOP_WRITE() call. needed by ZFS.

branches: 1.139.6; 1.139.12;
KASSERT(mutex_owned(vp->v_interlock)) in vnode iterator selector.

branches: 1.138.2; 1.138.4;
Cleanup VAT writout. To prevent issues with the sequential writing strategy
trying to write on blocks that are lost due to the synchronisation, don't just
bluntly do synchronize device caches, but split out on strategies.

Rework VAT searching on recordable media. It is now a lot more resilliant to
errors and it allows for VAT searching on crashed writeouts.

While here, make sure the node pointer is always initialised in
udf_get_node().

Fix filename creation and deleting with illegal file names. UDF has a
machanism for it but it allowed to create a file with a name it considered
illegal that then couldn't be deleted with the same name.

Fixes PR kern/50608. When confirmed, it can be closed.

Improve misleading variable name. Related to PR 50571.

XXX: also there should be real bounds-check logic in here.
XXX: if the on-disk data structure contains rubbish this code will
XXX: leak or trample arbitrary kernel memory.

add missing ,

PR/50571: David Binderman: src/sys/fs/udf/udf_subr.c:6465: obvious
performance tidyup

Use vfs_vnode_iterator for udf_do_sync.

- Build rb_tree to get an ordered list of nodes, sync them and
clean the tree.

- Stop abusing mntvnode_lock to serialize udf_do_sync, use new
mutex ump->sync_lock instead.

Remove dirtynodes_cv, it is only used for timed waits without any signals.

Replace the cv_timedwait with kpause.

Make vget always return vnode unlocked.

Convert callers who want locks to use vn_lock afterward.

Add extra argument so the compiler will report stragglers.

Change udf to vcache, keyed on udf_node->loc.loc.

fix udf_node leak. Reported by:
http://www.m00nbsd.net/ae123a9bae03f7dde5c6d654412daf5a.html#Report-4

branches: 1.127.2;
Fix bug introduced in last patch

As pointed out by wiz@ prevent a possible attack or corruption that results in
an endless loop of indirect descriptors being processed.

The number of indirect descriptors followed is now maximized.
While here, also fix a use-after-free bug!

branches: 1.125.2;
Fix debug printout of udf_write_metadata_partition_spacetable(), it referred
to an old variable.

branches: 1.124.2;
fix unused

Change vnode operations create, mknod, mkdir and symlink to return
the resulting vnode *vpp unlocked.

Discussed on tech-kern@

Welcome to 6.99.30

Don't create a variable just to assert() on it - assert on the
thing the variable got assigned.

fix unused variable warnings

Implement a real empty-directory checking for rmdir().

Should should fix the other part of PR kern/47987
Solves tests/vfs/t_vnops udf_dir_notempty

branches: 1.119.2; 1.119.4;
change vflushbuf() to take the full FSYNC_* flags.
translate FSYNC_LAZY into PGO_LAZY for VOP_PUTPAGES() so that
genfs_do_io() can set the appropriate io priority for the I/O.
this is the first part of addressing PR 46325.

branches: 1.118.4; 1.118.6;
use getdisksize()

branches: 1.117.2;
- rename UDF_MAX_NAMELEN -> UDF_MAXNAMLEN
- use NAME_MAX instead of MAXNAMLEN

Fix some typos.

Welcome to 5.99.53! Merge rmind-uvmplock branch:

- Reorganize locking in UVM and provide extra serialisation for pmap(9).
New lock order: [vmpage-owner-lock] -> pmap-lock.

- Simplify locking in some pmap(9) modules by removing P->V locking.

- Use lock object on vmobjlock (and thus vnode_t::v_interlock) to share
the locks amongst UVM objects where necessary (tmpfs, layerfs, unionfs).

- Rewrite and optimise x86 TLB shootdown code, make it simpler and cleaner.
Add TLBSTATS option for x86 to collect statistics about TLB shootdowns.

- Unify /dev/mem et al in MI code and provide required locking (removes
kernel-lock on some ports). Also, avoid cache-aliasing issues.

Thanks to Andrew Doran and Joerg Sonnenberger, as their initial patches
formed the core changes of this branch.

branches: 1.114.2;
Change vflushbuf() to return an error if a synchronous write fails.

Welcome to 5.99.51.

A not rewritable disc can be pseudo-overwritable. A pseudo-overwritable disc
can overwrite its logical volume integrity sequence so no need to make it
append-only for this class of discs.

Lookup the physical partition backing up the logical one when searching for the
metadata partition overlap for BD-R.

Fixes a kernel panic on mounting a BD-R formatted with UDF 2.60

Split out the raw to indexed partitioning code protecting against roque
implementations that use `ramdom' numbers for the physical partitions breaking
lots of implementations. Known curlpit is MicroSoft Windows 7.

Not only the partition mappings need to be protected against this but also the metadata partition files.

branches: 1.110.2; 1.110.4;
Fix the case in where a file with a latin1 file name was created. It was then
erronously entered as thelatin1 file name in the dirhash whereas the matching
routing assumes both UTF-8. This would result in a file being created but not
stat-able since the dirhash couldn't find the entry unless it was remounted.

Running with DEBUG again revealed locking errors previously undetected. The
VAT writeout was done locked while marked locked as were the readin and
writeout of the metadata partition space table.

While here, also protect the (vp) argument of the UDF_SET_SYSTEMFILE() macro.

Tested on UDF 1.50 sequential, UDF 2.01 RW and UDF 2.50 metadata RW meda.

Fixes/improvements to RB-tree implementation:
1. Fix inverted node order, so that negative value from comparison operator
would represent lower (left) node, and positive - higher (right) node.
2. Add an argument (i.e. "context"), passed to comparison operators.
3. Change rb_tree_insert_node() to return a node - either inserted one or
already existing one.
4. Amend the interface to manipulate the actual object, instead of the
rb_node (in a similar way as Patricia-tree interface does).
5. Update all RB-tree users accordingly.

XXX: Perhaps rename rb.h to rbtree.h, since cleaning-up..

1-3 address the PR/43488 by Jeremy Huddleston.

Passes RB-tree regression tests.
Reviewed by: matt@, christos@

Make holding v_interlock mandatory for callers of vget().

Announced some time ago on tech-kern.

Remove vlockmgr(). Generic vnode lock operations now use a rwlock located
in the vnode. All LK_* flags move from sys/lock.h to sys/vnode.h. Calls
to vlockmgr() in file systems get replaced with VOP_LOCK() or VOP_UNLOCK().

Welcome to 5.99.34.

Discussed on tech-kern.

Change layered file systems to always pass the locking VOP's down to the
leaf file system. Remove now unused member v_vnlock from struct vnode.
Welcome to 5.99.30

Discussed on tech-kern.

branches: 1.104.2;
First part of shrinking/growing metadata partition support:

- extending the metadata partition

Still to follow:
- sparsify metadata partition
- growing the metadata partition
- unsparsifying metadata partition

Rename metadata partition parameters to be less generic and add the flags.

Extract UDF metadata partition parameters as already done for VAT and sparable
partitions.

branches: 1.101.2;
Remove extra semicolons.

Fix typo in comments, no code change.

Issue extra synchronise caches before closing tracks or sessions. It shouldn't
be needed as the caches should already be synchronised, but better be safe.

Replace the old hashtable and sorted list implemenation by a RB-tree.

Benefits are significant speed improvements on node creation/insertion while
keeping the lookup times low and still allowing sequential iteration over the
nodes.

Make source prettier

Rewrite of udf_on_rootpath(), and vop_rename() code that calls it, after the
UFS way. The tree walking is now done the same and the code hasn't locked up
on examples that made it lockup before.

Commit first stage of free-space accounting. It is estimating the underlimit
of free blocks on the device and when free blocks are getting tight it tries
to readjust/recalculate that value by syncing the FS.

Second stage will be resizing the data/metadata partitions.

Renaming in UDF was already possible but directories could only be renamed in
the same directory.

This patch finally allows a directory to be moved between parent directories.

Enhance comment on node locking when writing out a node's descriptors.

Remove unneeded wait for SYNC being finished on node disposal of deleted node;
this could interfere and lockup.

Retire IN_CALLBACK_ULK flagging since its not used anymore and the old
assertions on it would trigger without cause.

Descriptor version is 16 bit and not 8 so use udf_rw16() to access them.
UniqueID was written in the logical volume integrity without byteswapping for
bigendian machines.

On creation of a new node, don't forget to set the length of the allocated
space in the right endian!

Forgot to use the udf_rw16() on the descriptor crc lengths of FID's!
Also fix a few dubiously formatted checks.

bzero -> memset

udf_write_filebuf: remove an write-only variable

branches: 1.85.2;
Implement session closure on sequential media when requested on mount. It will
allow CD-ROM/DVD-ROM/DB-ROM drives to read the media while still allowing them
to be appended later. It can also be seen as a way to make mountable
snapshots.

Specify BROKEN DIRECTORY ENTRY printf comes from udf
(can happen e.g. with medium error)

use M_ZERO on malloc() and remove subsequent bzero().

Implement auto-track repair on tracks marked damaged when searching for
writable tracks on session opening. Note that this an optionally implemented
feature and thus no error will be generated if it fails; the drive will most
likely autorepair it.

Fix critital panic when mounting a CD-R/DVD*R disc. The check for VV_ROOT was
not done correctly and could panic when requested to load a VAT descriptor
since the fileset was not yet fetched.

replace bitmask_snprintf(9) with snprintb(3)

Check for a node being root on node creation so when the root node is disposed
of when working deep into the directory tree it can reliably be reget and
marked correctly as the FS root.

Fixed pwd(1) lock panic and possible endless loop in other tools.

The great QUOTA purge of '08: get rid of those #ifdef QUOTA and
#include "opt_quota.h" which do exactly nothing. Speeds up kernel
compilation by 1.375*10^-20001 seconds. But leave the most moxious
comment in msdosfs_vfsops untouched.

Elaborate SYNC debug messages and rename some debug messages to a more
proper category.

Split uid/gid remapping code to be independent of eachother.

Allow for the exclusive selection of `direct' for rdonly mounts; its disabled
for now since its a lot slower than `rmw' access.

For archs that have trouble with `rmw' for whatever reason can so use it as a
scapegoat to allways mount savely rdonly though slower.

Remove old comment and remove unused calculated variable

branches: 1.73.2; 1.73.4;
Switch UDF over to the vfs_dirhash.c dirhash code.

Remove all remaining traces of UDF_READWRITE since it has been removed as
option.

Rewrite allocation scheme's determination and implementation replacing the
old somewhat naive selection scheme that didn't allow different allocation
settings for nodes, directory information (FIDs) and data.

Also fix some curious side-effects of atime updates on RMW devices.

Streamline allocation to prepare it for metadata partition. Also add
preliminary Metadata partition write support but its disabled still since
its not finished yet and not functioning correctly. All other formats are
checked and should work fine.

Delay physical partition spacebitmaps from volume-descriptor-checking until
the other supporting structures are read in.

Clarify the current spacemap read/write code is reading/writing physical
partition space maps. This in preperation of Metadata spacetable
reading/writeout work.

No use walking empty hashtables when purging.

Limit the UDF dirhash to a configurable amount of memory. The performance
penalty even with a limited dirhash of 128kb is acceptable but it should be
set by machine memory size though.

Use ecma-167 constants instead of homegrown to prevent double mapping.

Choose which time of allocation descriptors to use on new nodes based on
the number of partition mappings.

fix comment

On dirhash lookup, don't retrieve info you already have.

Remove double check

First retrieve length and hash values *before* advancing to the next entry.
On hashline collision it would check a random one next.

Implement directory hashing to speed up directory traversals. Speed
improvements of at least 4 times in untarring and roughly 100 to 500 times
on file creation in big directories. Lookup of files was O(n*n) and is now
O(1) even for file creation. Free spaces in the directory are kept in a
seperate list for fast file creation.

The postmark benchmark gives:

UDF old:
pm>set transactions 2000
pm>set number 3000
pm>run
Creating files...Done
Performing transactions..........Done
Deleting files...Done
Time:
1593 seconds total
681 seconds of transactions (2 per second)

Files:
3956 created (2 per second)
Creation alone: 3000 files (4 per second)
Mixed with transactions: 956 files (1 per second)
990 read (1 per second)
1010 appended (1 per second)
3956 deleted (2 per second)
Deletion alone: 2912 files (9 per second)
Mixed with transactions: 1044 files (1 per second)

Data:
5.26 megabytes read (3.38 kilobytes per second)
21.93 megabytes written (14.10 kilobytes per second)
pm>



UDF new:
pm>set transactions 2000
pm>set number 3000
pm>run
Creating files...Done
Performing transactions..........Done
Deleting files...Done
Time:
19 seconds total
3 seconds of transactions (666 per second)

Files:
3956 created (208 per second)
Creation alone: 3000 files (230 per second)
Mixed with transactions: 956 files (318 per second)
990 read (330 per second)
1010 appended (336 per second)
3956 deleted (208 per second)
Deletion alone: 2912 files (970 per second)
Mixed with transactions: 1044 files (348 per second)

Data:
5.26 megabytes read (283.66 kilobytes per second)
21.93 megabytes written (1.15 megabytes per second)

Dont round up nanosecond count; if its nearing wrap one could get a count
of >1 second resulting in invalid time stamps on disc.

Fix spelling in comment

On node creation, follow tmpfs and UFS by generating the uid/gid pair. This
fixes the lack of file ownership preservation on file/directory creation.

See PR kern/39124

Support for allocation extents descriptor writing to allow for arbitrary
heavily fragmented files.

Also fixing some (rare) allocation bugs and function name streamlining.

Tested on harddisc, CD-RW and CD-R i.e. all three basic backend classes.

branches: 1.54.2;
Disable read/write mounting for metadatapartition since attempts to delete
files now corrupt the image due to the lack of allocation scheme support.

Allow metadata partitioned discs to be mounted read-write even though its
allocation scheme is not implemented yet. Creating new files or deleting
anything is thus not possible.

Fix allocation extensions descriptor reading and do free them on node
destruction!

Support the creation of extended attributes for file creation in the
kernel and check and update them when found.

Note that it won't *insert* an extended attribute yet if it hasn't found it
while updating the birthtime.

Cleanup file access,modification,attribute and birthtime setting of files
and make the modification time when set never older than the creation time.

branches: 1.49.2;
fix argument type.

Implement wide char support for UDF by properly implementing unicode-16 to
UTF-8 like cd9660 and ntfs already do.

Cleaner gcc initialisation detection failure notice.

Fix `uninitialised variable' warning issued by the sh3 compiler. gcc can't
detect that the `last_node' variable will be set before used since it can't
parse the semantics of `TAILQ_EMPTY()' that is used as a guard first.

Thanks for H?rvard for finding and reporting it :)

Import writing part of the UDF file system making optical media like CD's
and DVD's behave like floppy discs. Writing is supported upto and including
version 2.01; version 2.50 and 2.60 will follow.

Also extending the UDF implementation to support symbolic links and
hardlinks.

Added are the mmcformat(8) tool to format rewritable CD/DVD discs and
newfs_udf(8).

Limitations:
all operations can be performed on the file system though the
sheduling is currently optimised for archiving workloads.

mv(1)/rename(2) is currently only implemented for non-directories.

branches: 1.44.6; 1.44.8; 1.44.10; 1.44.12;
Merge vmlocking2 to head.

use __KERNEL_RCSID() instead of __RCSID()

branches: 1.42.2; 1.42.4; 1.42.6;
Pullup fixes from UDF RW support project. This commit fixes the
mmc_discinfo conceptional bug and also vastly improves VAT searching.

Remove the "struct lwp *" argument from all VFS and VOP interfaces.
The general trend is to remove it from all kernel interfaces and
this is a start. In case the calling lwp is desired, curlwp should
be used.

quick consensus on tech-kern

branches: 1.40.2;
Fix panic when the VAT couldn't be found. It would refer an stale
udf_node->vnode.

branches: 1.39.2;
Merge from vmlocking:

- Split vnode::v_flag into three fields, depending on field locking.
- simple_lock -> kmutex in a few places.
- Fix some simple locking problems.

Merge brelse() changes from the vmlocking branch.

Avoid stack allocation of large dirent structures in foo_readdir().

branches: 1.36.4; 1.36.6; 1.36.8; 1.36.10;
It's not a good idea for device drivers to modify b_flags, as they don't
need to understand the locking around that field. Instead of setting
B_ERROR, set b_error instead. b_error is 'owned' by whoever completes
the I/O request.

Initialize size and/or writesize when creating a vnode.

branches: 1.34.2;
fix typos

Some rogue implementations use random physical partion numbers just to
break other implementations so lookup the physical number instead of
indexing it. Choosing random numbers here is legal according to the specs,
but not a logical choice and most likely done as a wierd kind of copy
protection.

Rogue implementation found to use this
*Microsoft CDIMAGE UDF

branches: 1.32.4; 1.32.6;
Call genfs_node_destroy() where appropriate.

branches: 1.31.2;
Fix important directory lookup bug that could result in an endless loop. It
fixes the case where a directory lookup is done in a directory has never
been visted/listed; the search optimalisation that searches the directory
from where it left behind the last time would never reach the initial
offset of zero since it would always have at least processed one entry.

Use the generic clock_ymdhms_to_secs() instead of home-brewn version to
translate UDF timestamps to timespec.

Forgot one place for UDF_ICB_FILETYPE_RANDOMACCESS.

Add UDF_ICB_FILETYPE_REALTIME type as found on UDF 2.50 discs. Also revert
a multiplication order to enforce 64 bit math.

Fix compilation issue when specifying different flags or compiling with a
different compiler. Thanks to Jared for noting.

Implement read-only access to UDF 2.50 and UDF 2.60 discs that use metadata
partition mappings. This brings the NetBSD implementation to read all discs
upto date and will read all discs currently defined by OSTA.

Spell "separate" correctly. From Zafer Aydogan.

Spell "schedule" correctly. From Zafer Aydogan.

branches: 1.23.2;
__unused removal on arguments; approved by core.

- sprinkle __unused on function decls.
- fix a couple of unused bugs
- no more -Wno-unused for i386

Fix panic on unmounting VAT file systems. The VAT table was allocated and
freed on the wrong malloc type.

Fix memory leaks on mounting and dismounting UDF volumes.

Fix lookup bug that could result in not finding the last entry. The check
for the cycling was too early. Also fix potential endless loop when the
last entry was deleted.

Make VOP_LOOKUP's O(1) instead of O(n log(n)) giving a speedup of 7500% on
a 1400 file directory.

Fix serious file-corruption issue on sparable partitions.

Fix panic when the sparable table didn't read in correctly.

branches: 1.15.2; 1.15.4; 1.15.6;
Fix copying too much space from a small buffer. The chances of it actually
happening are small but corrupt media could trigger it.

Fix pool allocation/freeing problem for once and for all correctly. Since
it was initialised quite late due to its reliance on disc data the mount
process could have stopped before initialising and thus could panic again
only now for uninitialising an not initialised pool! *sigh*

Make a better guess on VAT locations so discs that are semi-standard
formatted can be recognised. This also fixes the bug that prevented UDF to
mount an empty CD-R/DVD*R disc.

Fix comment; it stated a mimimum of 128 translations duh! a maximum it is!

branches: 1.11.2;
Fixing a bad case of walking/writing out of allocated space.... the
allocated space was 2048 bytes, but when adding 1024 to the variable
`unix_name' to split the allocated space in half it effectively starts just
OUTSIDE the allocated space. This ought to fix memory corruption bugs when
using UDF.

This is a routine to revisit one day.

branches: 1.10.2;
Don't allocate too much stuff on the stack.

branches: 1.9.2;
stack polics: Don't allocate 4K on the stack.
remove ; from };

branches: 1.8.2;
kauth fallout

integrate kauth.

s/allready/already/

branches: 1.5.2; 1.5.4; 1.5.6;
merge yamt-uio_vmspace branch.

- use vmspace rather than proc or lwp where appropriate.
the latter is more natural to specify an address space.
(and less likely to be abused for random purposes.)
- fix a swdmover race.

branches: 1.4.2;
VND and harddiscs have their own defect free space management.

Change a comment into a TODO and fix up case where we could return an
illegal allready-freed memory space.

branches: 1.2.2;
Fix $NetBSD$ tags. (sorry)

Initial import of a UDF file system implementation for NetBSD.

The code supports read access to all media types that CD/DVD type drives
can recognize including DVD-RAM and BD- drives as well as harddisc partions
and vnd devices. UDF versions upto the latest 2.60 are to be supported
though due to lack of test media version 2.50 and 2.60 are not implemented
yet though easy to add. Both open and closed media are supported.

Write access is planned and in preparation. To facilitate this some hooks
are present in the code that are not strictly needed in a read-only
implementation but which allow writing to be added more easily.

Implemented and tested media types are CD-ROM, CD-R, CD-RW, CD-MRW,
DVD-ROM, DVD*R, DVD*RW, DVD+MRW but the same code can also read DVD-RAM,
HD-DVD and BluRay discs. Also vnd devices have been tested with several
sector sizes.

Discs created and written by UDFclient, Nero's InCD and Roxio's
DirectCD/Drag2Disc read fine.

VFS_VGET(), VFS_ROOT(), VFS_FHTOVP(): give them a "int lktype" argument, to
allow us to get shared locks (or no lock) on the returned vnode. Matches
FreeBSD.

branches: 1.20.18; 1.20.24;
Cleanup VAT writout. To prevent issues with the sequential writing strategy
trying to write on blocks that are lost due to the synchronisation, don't just
bluntly do synchronize device caches, but split out on strategies.

branches: 1.19.8;
Implement a real empty-directory checking for rmdir().

Should should fix the other part of PR kern/47987
Solves tests/vfs/t_vnops udf_dir_notempty

branches: 1.18.10; 1.18.20; 1.18.24;
First part of shrinking/growing metadata partition support:

- extending the metadata partition

Still to follow:
- sparsify metadata partition
- growing the metadata partition
- unsparsifying metadata partition

branches: 1.17.2;
Replace the old hashtable and sorted list implemenation by a RB-tree.

Benefits are significant speed improvements on node creation/insertion while
keeping the lookup times low and still allowing sequential iteration over the
nodes.

Rewrite of udf_on_rootpath(), and vop_rename() code that calls it, after the
UFS way. The tree walking is now done the same and the code hasn't locked up
on examples that made it lockup before.

Commit first stage of free-space accounting. It is estimating the underlimit
of free blocks on the device and when free blocks are getting tight it tries
to readjust/recalculate that value by syncing the FS.

Second stage will be resizing the data/metadata partitions.

Renaming in UDF was already possible but directories could only be renamed in
the same directory.

This patch finally allows a directory to be moved between parent directories.

branches: 1.13.2;
Implement session closure on sequential media when requested on mount. It will
allow CD-ROM/DVD-ROM/DB-ROM drives to read the media while still allowing them
to be appended later. It can also be seen as a way to make mountable
snapshots.

branches: 1.12.2; 1.12.4;
Rewrite allocation scheme's determination and implementation replacing the
old somewhat naive selection scheme that didn't allow different allocation
settings for nodes, directory information (FIDs) and data.

Also fix some curious side-effects of atime updates on RMW devices.

Streamline allocation to prepare it for metadata partition. Also add
preliminary Metadata partition write support but its disabled still since
its not finished yet and not functioning correctly. All other formats are
checked and should work fine.

Forgot to rename those two references to write_space_bitmap too.

Implement directory hashing to speed up directory traversals. Speed
improvements of at least 4 times in untarring and roughly 100 to 500 times
on file creation in big directories. Lookup of files was O(n*n) and is now
O(1) even for file creation. Free spaces in the directory are kept in a
seperate list for fast file creation.

The postmark benchmark gives:

UDF old:
pm>set transactions 2000
pm>set number 3000
pm>run
Creating files...Done
Performing transactions..........Done
Deleting files...Done
Time:
1593 seconds total
681 seconds of transactions (2 per second)

Files:
3956 created (2 per second)
Creation alone: 3000 files (4 per second)
Mixed with transactions: 956 files (1 per second)
990 read (1 per second)
1010 appended (1 per second)
3956 deleted (2 per second)
Deletion alone: 2912 files (9 per second)
Mixed with transactions: 1044 files (1 per second)

Data:
5.26 megabytes read (3.38 kilobytes per second)
21.93 megabytes written (14.10 kilobytes per second)
pm>



UDF new:
pm>set transactions 2000
pm>set number 3000
pm>run
Creating files...Done
Performing transactions..........Done
Deleting files...Done
Time:
19 seconds total
3 seconds of transactions (666 per second)

Files:
3956 created (208 per second)
Creation alone: 3000 files (230 per second)
Mixed with transactions: 956 files (318 per second)
990 read (330 per second)
1010 appended (336 per second)
3956 deleted (208 per second)
Deletion alone: 2912 files (970 per second)
Mixed with transactions: 1044 files (348 per second)

Data:
5.26 megabytes read (283.66 kilobytes per second)
21.93 megabytes written (1.15 megabytes per second)

Support for allocation extents descriptor writing to allow for arbitrary
heavily fragmented files.

Also fixing some (rare) allocation bugs and function name streamlining.

Tested on harddisc, CD-RW and CD-R i.e. all three basic backend classes.

branches: 1.7.2;
Cleanup file access,modification,attribute and birthtime setting of files
and make the modification time when set never older than the creation time.

branches: 1.6.2;
Implement wide char support for UDF by properly implementing unicode-16 to
UTF-8 like cd9660 and ntfs already do.

Import writing part of the UDF file system making optical media like CD's
and DVD's behave like floppy discs. Writing is supported upto and including
version 2.01; version 2.50 and 2.60 will follow.

Also extending the UDF implementation to support symbolic links and
hardlinks.

Added are the mmcformat(8) tool to format rewritable CD/DVD discs and
newfs_udf(8).

Limitations:
all operations can be performed on the file system though the
sheduling is currently optimised for archiving workloads.

mv(1)/rename(2) is currently only implemented for non-directories.

branches: 1.4.10; 1.4.16; 1.4.66; 1.4.68; 1.4.70; 1.4.72;
remove trailing newlines

branches: 1.3.2; 1.3.4; 1.3.6;
Fix header re-inclusion protection defines to a less generic naming
scheme.

Fix $NetBSD$ tags. (sorry)

Initial import of a UDF file system implementation for NetBSD.

The code supports read access to all media types that CD/DVD type drives
can recognize including DVD-RAM and BD- drives as well as harddisc partions
and vnd devices. UDF versions upto the latest 2.60 are to be supported
though due to lack of test media version 2.50 and 2.60 are not implemented
yet though easy to add. Both open and closed media are supported.

Write access is planned and in preparation. To facilitate this some hooks
are present in the code that are not strictly needed in a read-only
implementation but which allow writing to be added more easily.

Implemented and tested media types are CD-ROM, CD-R, CD-RW, CD-MRW,
DVD-ROM, DVD*R, DVD*RW, DVD+MRW but the same code can also read DVD-RAM,
HD-DVD and BluRay discs. Also vnd devices have been tested with several
sector sizes.

Discs created and written by UDFclient, Nero's InCD and Roxio's
DirectCD/Drag2Disc read fine.

Assert that udf_get_node()/udf_loadvnode() returns a valid root node when it doesn't return an error.

Lock devvp for vinvalbuf().

fix few typos for word "previous(ly)" in comments.

Make sysctl udf.verbose dependent on UDF_DEBUG instead of DEBUG

fix typos in word "successful".

Fix all remaining typos, mainly in comments but also in few definitions and log messages, reported by me in PR kern/54889.
Also fixed some additional typos in comments, found on review of same files or typos.

branches: 1.80.6;
Move comment related to the sysctl_createv() in SYSCTL_SETUP() from the old
place to the new place too.

Replace most uses of vp->v_usecount with a call to vrefcnt(vp), a function
that hides the details and does atomic_load_relaxed(). Signature matches
FreeBSD.

branches: 1.78.2;
Use the module subsystem's ability to process SYSCTL_SETUP() entries to
automate installation of sysctl nodes.

Note that there are still a number of device and pseudo-device modules
that create entries tied to individual device units, rather than to the
module itself. These are not changed.

VFS_VGET(), VFS_ROOT(), VFS_FHTOVP(): give them a "int lktype" argument, to
allow us to get shared locks (or no lock) on the returned vnode. Matches
FreeBSD.

branches: 1.76.6; 1.76.10; 1.76.12;
No need to call vflush from failing udf_mount(). If the system nodes
really have to disappear we should change vrele() to vrecycle() here.

branches: 1.75.4; 1.75.6;
KASSERT(mutex_owned(vp->v_interlock)) in vnode iterator selector.

Add generic genfs_suspendctl() and use it for all file systems.
Layered file systems need work.

branches: 1.73.2; 1.73.4;
KNF. Remove extra spaces. No functional change.

It is meaningless to check the root node if there was an error.

Use vfs_vnode_iterator for udf_do_sync.

- Build rb_tree to get an ordered list of nodes, sync them and
clean the tree.

- Stop abusing mntvnode_lock to serialize udf_do_sync, use new
mutex ump->sync_lock instead.

Use vfs_vnode_iterator for udf_unmount_sanity_check.

Remove dirtynodes_cv, it is only used for timed waits without any signals.

Replace the cv_timedwait with kpause.

Change udf to vcache, keyed on udf_node->loc.loc.

branches: 1.67.4;
An (un)privileged user can easily make the kernel dereference a NULL
pointer.

The kernel allows 'data' to be NULL; it's the fs's responsibility to
ensure that it isn't NULL (if the fs actually needs data).

ok christos@

branches: 1.66.2;
Change all vfsops to use C99 designated initializers.

No functional changes intended.

Ensure that the top level sysctl nodes (kern, vfs, net, ...) exist before
the sysctl link sets are processed, and remove redundancy.

Shaves >13kB off of an amd64 GENERIC, not to mention >1k duplicate
lines of code.

Replace macro v_specmountpoint with two functions spec_node_getmountedfs()
and spec_node_setmountedfs() to manage the file system mounted on a device.
Assert the device is a block device.

Welcome to 6.99.24

Discussed on tech-kern@ some time ago.

Reviewed by: David Holland <dholland@netbsd.org>

branches: 1.63.2; 1.63.4;
Replace the remaining KAUTH_GENERIC_ISSUSER authorization calls with
something meaningful. All relevant documentation has been updated or
written.

Most of these changes were brought up in the following messages:

http://mail-index.netbsd.org/tech-kern/2012/01/18/msg012490.html
http://mail-index.netbsd.org/tech-kern/2012/01/19/msg012502.html
http://mail-index.netbsd.org/tech-kern/2012/02/17/msg012728.html

Thanks to christos, manu, njoly, and jmmv for input.

Huge thanks to pgoyette for spinning these changes through some build
cycles and ATF.

branches: 1.62.4; 1.62.6; 1.62.10; 1.62.12;
VOP_OPEN() needs a locked vnode. All these copy-and-pasted xxxfs_mount()
implementations need more review.

branches: 1.61.2;
- rename UDF_MAX_NAMELEN -> UDF_MAXNAMLEN
- use NAME_MAX instead of MAXNAMLEN

Clean up vnode lock operations pass 2:

VOP_UNLOCK(vp, flags) -> VOP_UNLOCK(vp): Remove the unneeded flags argument.

Welcome to 5.99.32.

Discussed on tech-kern.

branches: 1.59.2; 1.59.4;
Replace the old hashtable and sorted list implemenation by a RB-tree.

Benefits are significant speed improvements on node creation/insertion while
keeping the lookup times low and still allowing sequential iteration over the
nodes.

Convert 67 namei call sites to use namei_simple, in these functions:

check_console, veriexecclose, veriexec_delete, veriexec_file_add,
emul_find_root, coff_load_shlib (sh3 version), coff_load_shlib,
compat_20_sys_statfs, compat_20_netbsd32_statfs,
ELFNAME2(netbsd32,probe_noteless), darwin_sys_statfs,
ibcs2_sys_statfs, ibcs2_sys_statvfs, linux_sys_uselib,
osf1_sys_statfs, sunos_sys_statfs, sunos32_sys_statfs,
ultrix_sys_statfs, do_sys_mount, fss_create_files (3 of 4),
adosfs_mount, cd9660_mount, coda_ioctl, coda_mount, ext2fs_mount,
ffs_mount, filecore_mount, hfs_mount, lfs_mount, msdosfs_mount,
ntfs_mount, sysvbfs_mount, udf_mount, union_mount, sys_chflags,
sys_lchflags, sys_chmod, sys_lchmod, sys_chown, sys_lchown,
sys___posix_chown, sys___posix_lchown, sys_link, do_sys_pstatvfs,
sys_quotactl, sys_revoke, sys_truncate, do_sys_utimes, sys_extattrctl,
sys_extattr_set_file, sys_extattr_set_link, sys_extattr_get_file,
sys_extattr_get_link, sys_extattr_delete_file,
sys_extattr_delete_link, sys_extattr_list_file, sys_extattr_list_link,
sys_setxattr, sys_lsetxattr, sys_getxattr, sys_lgetxattr,
sys_listxattr, sys_llistxattr, sys_removexattr, sys_lremovexattr

All have been scrutinized (several times, in fact) and compile-tested,
but not all have been explicitly tested in action.

XXX: While I haven't (intentionally) changed the use or nonuse of
XXX: TRYEMULROOT in any of these places, I'm not convinced all the
XXX: uses are correct; an audit might be desirable.

Commit first stage of free-space accounting. It is estimating the underlimit
of free blocks on the device and when free blocks are getting tight it tries
to readjust/recalculate that value by syncing the FS.

Second stage will be resizing the data/metadata partitions.

Add genfs_can_mount() and use it to prevent some more code duplication of
the security checks when mounting a device (VOP_ACCESS() + kauth(9) call)).

Proposed with no objections on tech-kern@:

http://mail-index.netbsd.org/tech-kern/2009/04/20/msg004859.html

The vnode is always expected to be locked, so no locking is done outside
the file-system code.

branches: 1.55.2;
If mounting for read-write dont allow a session number to be specified since
its currently still misbehaving. If supported later, it would allow one or a
series of sessions on a sequential recordable media to be ignored as if they
never were created.

Also fix a small comment: its not the direct but the bootstrap disc strategy
that we close down.

Check for a node being root on node creation so when the root node is disposed
of when working deep into the directory tree it can reliably be reget and
marked correctly as the FS root.

Fixed pwd(1) lock panic and possible endless loop in other tools.

The great QUOTA purge of '08: get rid of those #ifdef QUOTA and
#include "opt_quota.h" which do exactly nothing. Speeds up kernel
compilation by 1.375*10^-20001 seconds. But leave the most moxious
comment in msdosfs_vfsops untouched.

branches: 1.52.2;
Fix to prevent a closed sequential media to be opened for read/write.

branches: 1.51.2;
Switch UDF over to the vfs_dirhash.c dirhash code.

Remove all remaining traces of UDF_READWRITE since it has been removed as
option.

forgot to destroy an additional mutex and an condition variable.

Remove the need for the kernel option UDF_READWRITE to be able to write to
discs since writing is mature enough.

Rewrite allocation scheme's determination and implementation replacing the
old somewhat naive selection scheme that didn't allow different allocation
settings for nodes, directory information (FIDs) and data.

Also fix some curious side-effects of atime updates on RMW devices.

Streamline allocation to prepare it for metadata partition. Also add
preliminary Metadata partition write support but its disabled still since
its not finished yet and not functioning correctly. All other formats are
checked and should work fine.

Unify naming scheme and source code beautify for better readablility.

Forgot to rename those two references to write_space_bitmap too.

Limit the UDF dirhash to a configurable amount of memory. The performance
penalty even with a limited dirhash of 128kb is acceptable but it should be
set by machine memory size though.

Limit sectorsize to max. 4096 bytes. This prevents a panic when declaring
the node pool since standard pools are limited upto 4096 aparently.
Pagesize limit even?

Instead of bombing out when trying to mount a disc RW when UDF_READWRITE is
not defined, downgrade it to a read-only mount.

Implement directory hashing to speed up directory traversals. Speed
improvements of at least 4 times in untarring and roughly 100 to 500 times
on file creation in big directories. Lookup of files was O(n*n) and is now
O(1) even for file creation. Free spaces in the directory are kept in a
seperate list for fast file creation.

The postmark benchmark gives:

UDF old:
pm>set transactions 2000
pm>set number 3000
pm>run
Creating files...Done
Performing transactions..........Done
Deleting files...Done
Time:
1593 seconds total
681 seconds of transactions (2 per second)

Files:
3956 created (2 per second)
Creation alone: 3000 files (4 per second)
Mixed with transactions: 956 files (1 per second)
990 read (1 per second)
1010 appended (1 per second)
3956 deleted (2 per second)
Deletion alone: 2912 files (9 per second)
Mixed with transactions: 1044 files (1 per second)

Data:
5.26 megabytes read (3.38 kilobytes per second)
21.93 megabytes written (14.10 kilobytes per second)
pm>



UDF new:
pm>set transactions 2000
pm>set number 3000
pm>run
Creating files...Done
Performing transactions..........Done
Deleting files...Done
Time:
19 seconds total
3 seconds of transactions (666 per second)

Files:
3956 created (208 per second)
Creation alone: 3000 files (230 per second)
Mixed with transactions: 956 files (318 per second)
990 read (330 per second)
1010 appended (336 per second)
3956 deleted (208 per second)
Deletion alone: 2912 files (970 per second)
Mixed with transactions: 1044 files (348 per second)

Data:
5.26 megabytes read (283.66 kilobytes per second)
21.93 megabytes written (1.15 megabytes per second)

branches: 1.39.2;
Create sysctl entries during module initialisation and destroy them
appropriately.

Many of these file systems are now ready for modularisation.

branches: 1.38.2;
Import writing part of the UDF file system making optical media like CD's
and DVD's behave like floppy discs. Writing is supported upto and including
version 2.01; version 2.50 and 2.60 will follow.

Also extending the UDF implementation to support symbolic links and
hardlinks.

Added are the mmcformat(8) tool to format rewritable CD/DVD discs and
newfs_udf(8).

Limitations:
all operations can be performed on the file system though the
sheduling is currently optimised for archiving workloads.

mv(1)/rename(2) is currently only implemented for non-directories.

Convert file systems to dynamically attach with the new module interface.
Make VFS hooks dynamic while we're here and say farewell to VFS_ATTACH and
VFS_HOOKS_ATTACH linksets.

As a consequence, most of the file systems can now be loaded as new style
modules.

Quick sanity check by ad@.

branches: 1.36.2;
PR kern/38057 ffs makes assuptions about devvp file system
PR kern/33406 softdeps get stuck in endless loop

Introduce VFS_FSYNC() and call it when syncing a block device, if it
has a mounted file system.

branches: 1.35.6; 1.35.8; 1.35.10;
Fix some race conditions in rename.
Introduce a per-FS rename lock and new vfsops to manipulate it.
Get this lock while renaming. Also add another relookup() in do_sys_rename,
which is a hack to kludge around some of the worst deficiencies of
ufs_rename.
reviewed-by: pooka (and an earlier rev by ad)
posted on tech-kern with no objections.

specfs changes for PR kern/37717 (raidclose() is no longer called on
shutdown). There are still problems with device access and a PR will be
filed.

- Kill checkalias(). Allow multiple vnodes to reference a single device.

- Don't play dangerous tricks with block vnodes to ensure that only one
vnode can describe a block device. Instead, prohibit concurrent opens of
block devices. As a bonus remove the unreliable code that prevents
multiple file system mounts on the same device. It's no longer needed.

- Track opens by vnode and by device. Issue cdev_close() when the last open
goes away, instead of abusing vnode::v_usecount to tell if the device is
open.

use __KERNEL_RCSID() instead of __RCSID()

branches: 1.32.2; 1.32.4;
Remove cn_lwp from struct componentname. curlwp should be used
from on. The NDINIT() macro no longer takes the lwp parameter and
associates the credentials of the calling thread with the namei
structure.

branches: 1.31.2;
Remove the "struct lwp *" argument from all VFS and VOP interfaces.
The general trend is to remove it from all kernel interfaces and
this is a start. In case the calling lwp is desired, curlwp should
be used.

quick consensus on tech-kern