add the ability to only output paths included in a spec file when creating
from a directory (-c)

Implement FreeBSD's -f spec1 -f spec2 spec comparision function. This
is more useful than -C and diff because it highlights the keys that have
changed value so for example, you can see that the size of a file has
not changed but the contents have. (brooks)

branches: 1.32.6; 1.32.12;
Enable WARNS=4 by default, except for:
cpuctl dumplfs hprop ipf iprop-log kadmin kcm kdc kdigest
kimpersonate kstash ktutil makefs ndbootd ntp pppd quot
racoon racoonctl rtadvd sntp sup tcpdchk tcpdmatch tcpdump
traceroute traceroute6 user veriexecgen wsmoused zic
(Mostly third-party applications)

branches: 1.31.12;
warns=4

branches: 1.30.4;
Don't depend on lubutil if we're building host tools. Fix tools build on
non-NetBSD hosts.

Complete the move of string_to_flags() and flags_to_string() from the
bin/ls sources to libutil:
o Bump libutil minor version number
o Fix uses to include <util.h> to pick up the function definitions
o Fix most uses of flags_to_string() to release the now-malloc()ed result

The "#ifdef UNUSED_BLOCK" in stat_flags.c wasn't so unused after
all -- it was used by the mtree source. So, instead do an "#ifdef
MTREE", and define that one when building mtree.

WARNS=3 seems to work for me

Now that <bsd.prog.mk> DTRT if HOSTPROG is defined (i.e, it is a no-op),
there's no need to special-case .include-ing it.

tweaks for fparseln(3) move from libutil to libc:
- remove #include <util.h> if nothing else needed it
- remove LDFLAGS+=-lutil if nothing else needed it

use NETBSDSRCDIR as appropriate

Add hooks to make mtree compilable from src/tools.

Add '-N dbdir', to specify that user & group lookups (via pwcache(3)) are to
be done from dbdir/master.passwd and dbdir/group, using simple text file
parsing routines.

with local crc.c, don't need usr.bin/cksum in .PATH any more

User interface changes from (or inspired by) FreeBSD:
- Add -L to walk the tree `logically', by following symbolic links in
the heirarchy.
- Add -P to walk the tree `physically'. This is the current behaviour,
and the default.
- Add "-X excludes-file" to give mtree the ability to exclude files and
directories from its traversal. excludes-file contains fnmatch(3)
patterns to exclude from the walk.
- Add "md5digest" synonym for "md5".
- Add "rmd160" keyword for RMD-160 message digest, and "rmd160digest" synonym.
- Add "sha1" keyword for SHA-1 message digest, and "sha1digest" synonym.
- Don't try to compare() other attributes if the type doesn't match;
it's nothing but trouble, and no use anyway.
- In -c, only emit "/set" records if something has changed since the
previous one.

User interface changes by me:
- Check a device's parameters before checking uid/gid/mode.
- If updating (-u), modify the following to match the specification:
- Device type (retaining existing ownership).
- Symlink target.

Fixes from (or inspired by) FreeBSD:
- Use p->ftslevel instead of own code to keep track of the level ourself.
The previous code got majorly confused if fts(3) couldn't descend
into a subdir, resulting in leaf nodes getting attached to the wrong
directory.
XXX: This new method is much much more robust, even though it's not 100%
perfect; it might result in a couple of following entries in the spec
to be incorrectly tagged as missing.
- Pass a useful pathname to rlink(), so that logical (-L) traversal
doesn't confuse symlink checking.
- Consistently use MAXPATHLEN+1 sized buffers for pathnames, so that
there's room for the NUL.
- Use mtree_err() and strerror(p->fts_errno) to report errors during
the fts(3) walk.

Fixes by me:
- Remove now-unused `const char *name' argument from compare().
- Change crc_total from an int to a u_int32_t, to match usr.bin/cksum/crc.c.
- Remove trailing whitespace.
- Remove unnecessary (void) casts on functions.
- Reorder entries in the getopt() switch.
- Replace strtoq() with strtoll(), and use strtoul() appropriately.
- Renumber F_ flags to be in alphabetical order.

complete WARNS=2 cleanup (with a bit of ickiness for the fts_open argv issue)

Make mtree a host tool and use it when appropriate.

Add "device" keyword, which allows the device number to be specified,
to be later checked and possibly created. Uses parsing and encoding
routines from mknod(8).

replace home-grown parser with fparseln()

code.c is no longer used, we use strsvis instead. Removed prototypes
from extern.h

Encode file names using strvis(3) when creating a specification.
Escape white space characters and `#' characters. Do the inverse
when reading specs.

This is a temporary fix until we have strzouvis(3) (or some such)
which is an improved version of strvis (taking an additional list
of characters to encode).

Fixes PR 5006 and PR 7663

branches: 1.13.2; 1.13.6;
add support for 4.4BSD file flags. we take a `flags=xxx' parameter,
using the same style of naming as chflags, ls & xinstall use, plus an
additional `none' to turn off a default setting.

use CPPFLAGS instead of CFLAGS

branches: 1.11.2;
merge lite-2

Back out the .PATH.c changes. The .depend problem (and others)
will be fixed using the new .NOPATH make feature instead.

Use .PATH.c: ...

add NetBSD id's

clean up import, fix a few bugs, etc.

specify man pages the new way.

don't forget the man page. from Luke Mewburn <lm@rmit.edu.au>

fix oops

latest mtree from uunet. doesn't compile, but that'll be fixed soon.
(want to have 'pseudo-virgin' sources, for when fixed fnmatch stuff
is released...)

Add RCS identifiers.

branches: 1.1.1;
Initial revision

code.c is no longer used, we use strsvis instead. Removed prototypes
from extern.h

branches: 1.4.2;
Add __RCSID()

Add RCS tag

prototype encode()/decode()

Encode file names using strvis(3) when creating a specification.
Escape white space characters and `#' characters. Do the inverse
when reading specs.

This is a temporary fix until we have strzouvis(3) (or some such)
which is an improved version of strvis (taking an additional list
of characters to encode).

Fixes PR 5006 and PR 7663

PR/59826: Jose Luis Duran: Fix FreeBSD 9 compatibility output

usr.sbin/mtree: Fix typos.

Submitted by: Jose Luis Duran
PR bin/59824

PR/58875: Jose Luis Duran: Produce consistent checksums in verification
by scanning directories in the same order as usual. While here, fix some
incorrect types.

branches: 1.60.8;
Deal with the case when a device gets recreated (eg when mtree is called by
MAKEDEV) and the stat buffer contains information about the original device
before checking/setting the owner/group/mode of the new device file.

mtree: use POSIX type uint32_t instead of u_int32_t

From Brooks Davis:
When comparing a directory to an mtree file NetBSD mtree has output like
this:

xxx: size (4, 5)
md5digest (0x6de9439834c9147569741d3c9c9fc010, 0xa5d119ab8edcda0ef7f381da8=
f52f92f)
=2E/yyy missing

FreeBSD's historical behavior is this:

xxx changed
size expected 4 found 5
md5digest expected 6de9439834c9147569741d3c9c9fc010 found a5d119ab8edcda0e=
f7f381da8f52f92f
=2E/yyy missing

FreeBSD's mergemaster -U command depends on the latter behavior so
the following patch alters the freebsd9 mode of mtree to output
that format. I don't think programs should rely on this output one way
or another since it's clearly meant for humans, but this seems like the
easiest short term fix.

Our <sys/param.h> ends up calling header files that define intmax_t. This
should not be the case (but >sys/param.h> is not a standard header so all bets
are off). FreeBSD's does not, so explicitly include <stdint.h> to get it.

use intmax_t instead of long long. fixes wrong cast for time_t.

account for differences in digest names

explicitly include <sys/stat.h>

more const
check if we support SHA384 before using
(brooks)

branches: 1.52.8; 1.52.14;
fix dev_t formats.

The hashing routines (MD5File() and co.) can do dynamic allocation, so
take advantage of that instead of an ugly hard-coded MAXHASHLEN limit
that needs updating.

I think this was suggested by cube@ years ago.

Fix mtree to release the returned strings from flags_to_string()
following the move of it from bin/ls to libutil and the change of
it to return malloc()ed memory.

kill crypto/rmd160.h and crypto/sha2.h, and instead make symlinks to
/usr/include from /usr/include/sys. This makes all the one way hash
header handling identical.

Use crypto/rmd160.h.

Use MAXHASHLEN here too.

Add SHA2 support in mtree.

branches: 1.45.2;
Implement -M to "merge" entries, even with different types.
This can be used in /etc/security to allow special.local to override
types of entries in special (e.g, replacing a dir with a link).

Rename Wflag to mtree_Wflag, to be more "external namespace" friendly.

Completely rework how tools/compat is done. Purge all uses/references to
_NETBSD_SOURCE as this makes cross building from older/newer versions of
NetBSD harder, not easier (and also makes the resulting tools 'different')

Wrap all required code with the inclusion of nbtool_config.h, attempt to
only use POSIX code in all places (or when reasonable test w. configure and
provide definitions: ala u_int, etc).

Reviewed by lukem. Tested on FreeBSD 4.9, Redhat Linux ES3, NetBSD 1.6.2 x86
NetBSD current (x86 and amd64) and Solaris 9.

Fixes PR's: PR#17762 PR#25944

branches: 1.43.2;
Move UCB-licensed code from 4-clause to 3-clause licence.

Patches provided by Joel Baker in PR 22366, verified by myself.

Remove convoluted code to work around missing lch{flags,mod,own}();
instead, rely upon libnbcompat to provide a workaround (which returns
an error if the target is a symlink).
Should fix another MacOS X build issue due to missing lchown(2).

- Check HAVE_LCHFLAGS for lchflags(2) support, rather than assuming
that HAVE_STRUCT_STAT_ST_FLAGS implies this.
- Set HAVE_LCHFLAGS for native builds
- Clean up {CLEAR,SET,CHANGE}FLAGS macros, and only provide if
HAVE_STRUCT_STAT_ST_FLAGS is set.

(Fixes compilation as a tool on MacOS X, noted by Allen Briggs.)

Funtion appropriately in the absence of a lchmod call on the host.

- use lch*() instead of ch*() to change permissions, so that links in
the specfile don't confuse things. doesn't affect the default
(physical) operation, and shouldn't affect logical either.
- slightly rework how miss() is implemented so that duplicated code is
reduced and links may optionally have a mode, uname and/or gname.

Protect __RCSID and __COPYRIGHT from being invoked if not defined.

Add hooks to make mtree compilable from src/tools.

provide the ability to remove support for generating or checking various
digests by #define-ing NO_MD5, NO_RMD160, or NO_SHA1. the keywords are
still parsed.

Generally don't need the extra byte in MAXPATHLEN sized buffers, because
pathnames are guaranteed by POSIX to be MAXPATHLEN-1 bytes long.
Kindly pointed out by Warner Losh in private email.

User interface changes from (or inspired by) FreeBSD:
- Add -L to walk the tree `logically', by following symbolic links in
the heirarchy.
- Add -P to walk the tree `physically'. This is the current behaviour,
and the default.
- Add "-X excludes-file" to give mtree the ability to exclude files and
directories from its traversal. excludes-file contains fnmatch(3)
patterns to exclude from the walk.
- Add "md5digest" synonym for "md5".
- Add "rmd160" keyword for RMD-160 message digest, and "rmd160digest" synonym.
- Add "sha1" keyword for SHA-1 message digest, and "sha1digest" synonym.
- Don't try to compare() other attributes if the type doesn't match;
it's nothing but trouble, and no use anyway.
- In -c, only emit "/set" records if something has changed since the
previous one.

User interface changes by me:
- Check a device's parameters before checking uid/gid/mode.
- If updating (-u), modify the following to match the specification:
- Device type (retaining existing ownership).
- Symlink target.

Fixes from (or inspired by) FreeBSD:
- Use p->ftslevel instead of own code to keep track of the level ourself.
The previous code got majorly confused if fts(3) couldn't descend
into a subdir, resulting in leaf nodes getting attached to the wrong
directory.
XXX: This new method is much much more robust, even though it's not 100%
perfect; it might result in a couple of following entries in the spec
to be incorrectly tagged as missing.
- Pass a useful pathname to rlink(), so that logical (-L) traversal
doesn't confuse symlink checking.
- Consistently use MAXPATHLEN+1 sized buffers for pathnames, so that
there's room for the NUL.
- Use mtree_err() and strerror(p->fts_errno) to report errors during
the fts(3) walk.

Fixes by me:
- Remove now-unused `const char *name' argument from compare().
- Change crc_total from an int to a u_int32_t, to match usr.bin/cksum/crc.c.
- Remove trailing whitespace.
- Remove unnecessary (void) casts on functions.
- Reorder entries in the getopt() switch.
- Replace strtoq() with strtoll(), and use strtoul() appropriately.
- Renumber F_ flags to be in alphabetical order.

Add -W: don't attempt to change the perms/time/flags on existing entries, nor
set them when creating new directories. (AKA the ``don't whack'' option).
Concept suggested by Todd Vierling.

- enforce the top-most entry must be "." and must be a directory; avoids
various potential problems when intermixing full and relative paths
- changes to make it much easier to use spec() - the specfile parser - in
other programs (via .PATHing spec.c and misc.c):
- move excludetags, includetags and keys from mtree.c to misc.c
- implement mtree_err() using vwarnx() instead of assuming name is
"mtree"
- move inotype() and nodetype() from compare.c to misc.c
- add nodetoinode(), to convert from an mtree F_* type to a
mode_t S_IF* type
- clean up #include use; don't assume "mtree.h" pulls in some
standard includes
- change spec() to take a FILE * arg (where the specfile is read from)

need semicolon between goto label and closing brace

make this mostly WARNS=2 safe

Add "device" keyword, which allows the device number to be specified,
to be later checked and possibly created. Uses parsing and encoding
routines from mknod(8).

features:
- add a new keyword - "tags" - which is a comma separated list of tags
associated with that file.
- add "-E tags"; exclude files in -D output with matching tags
- add "-I tags"; only include files in -D output with matching tags
(default is to list all)

fixes:
- move all extern variable references from individual files into extern.h
- `quote' some more user-specified strings in error messages
- man page: list an options arguments in the description, and sort xrefs.

- implement -D - dump the specfile out in a manner that's parseable
by various tools
- implement -R key - remove the given key(s) from the list of keys to print
(now it's possible with "-k type -R type" to remove all keys :)
- rename ftype() to nodetype(), and make it public
- use pwcache(3) functions instead of get{gr,pw}{nam,uid}(3)
- add dump_nodes(), which provides the guts for -D.

Add a -l flag which does "loose" permissions checks, i.e. a mode 444
matches a check for mode 644 (though obviously not the reverse). This
can be used by the nightly security run, making the output1 more useful
by having it contain fewer spurious permissions violations.

Note that I did not make -l work if you have a sgid/suid/sticky bit
set. I don't know how you could cause security trouble with more
stringent settings and a suid file, but I don't want to find out the
hard way.

- add parsetype() (a la parsekey()) to parse the "type=xxx" arg. this improves
the error detection for invalid types, as well as shrinking binary size
- implement inotype() using ftype(), rather than duplicating the switch
- change "char *" arguments to "const char *"

ANSIfy, KNF, in preperation for more work.

use %ll_ instead of the less standard %q_

branches: 1.22.8;
fix thinko in mtime stuff.

portability fixes:
- don't use timespecs in struct stat on non 4.4BSD systems
- don't assume that tv_sec is a time_t

- add -m and -i to change file flags more usefully.
- fix bugs in fileflags checking

all of the above from dr. lex wennmacher <wennmach@geo.Uni-Koeln.DE>
with some error fixes from egcs via myself.

Appease gcc-2.8.1, make printf formats more portable.

Add an `md5' keyword to mtree, which takes as an argument the MD5 cryptographic
checksum of the file in question, optionally preceeded by `0x'.

This has been tested, and does work, AFAICT. There is currently no md5 equivalent
of the '-s' flag for use with checksum.

add support for 4.4BSD file flags. we take a `flags=xxx' parameter,
using the same style of naming as chflags, ls & xinstall use, plus an
additional `none' to turn off a default setting.

Use u_int32_t rather than Sys-V u_long.
Rename err() to mtree_err(), to avoid conflict with libc.

Add {} pairs to shut up egcs.

Make this compile again on NetBSD/alpha; cast off_t variable to long long
so that it matches printf format `%qd'.

branches: 1.13.2;
WARNSify, getopt returns -1 not EOF

fix printf() formats, make explicit comparisons to NULL (gcc -Wall)

Add a `-t' option to update time stamps.

branches: 1.10.4;
Rename struct timespec fields to conform to POSIX.1b

Insert parentheses to get correct precedence when comparing time-stamps
(from Arne Juul; PR#1660).

add NetBSD id's

clean up import, fix a few bugs, etc.

off_t exp.

make things a bit more consistent

latest mtree from uunet. doesn't compile, but that'll be fixed soon.
(want to have 'pseudo-virgin' sources, for when fixed fnmatch stuff
is released...)

update for fts library change

Add RCS identifiers.

branches: 1.1.1;
Initial revision

PR/58875: Jose Luis Duran: Produce consistent checksums in verification
by scanning directories in the same order as usual. While here, fix some
incorrect types.

branches: 1.10.8;
mtree: use POSIX type uint32_t instead of u_int32_t

remove dup extern (brooks)

branches: 1.8.42; 1.8.48;
appease gcc -Wuninitialized

ANSIfy function declaration, remove register declarations, and some
minor KNFing.

Completely rework how tools/compat is done. Purge all uses/references to
_NETBSD_SOURCE as this makes cross building from older/newer versions of
NetBSD harder, not easier (and also makes the resulting tools 'different')

Wrap all required code with the inclusion of nbtool_config.h, attempt to
only use POSIX code in all places (or when reasonable test w. configure and
provide definitions: ala u_int, etc).

Reviewed by lukem. Tested on FreeBSD 4.9, Redhat Linux ES3, NetBSD 1.6.2 x86
NetBSD current (x86 and amd64) and Solaris 9.

Fixes PR's: PR#17762 PR#25944

branches: 1.5.2;
Move UCB-licensed code from 4-clause to 3-clause licence.

Patches provided by Joel Baker in PR 22366, verified by myself.

Protect __RCSID and __COPYRIGHT from being invoked if not defined.

Add hooks to make mtree compilable from src/tools.

need <stdio.h> and <time.h> for mtree.h

Add a separate version of the CRC routine that listens to sflag.

PR/59838: Jose Louis Duran: Check for F_TYPE before printing the type

PR/58875: Jose Luis Duran: Produce consistent checksums in verification
by scanning directories in the same order as usual. While here, fix some
incorrect types.

branches: 1.78.2;
For the NetBSD tools build we provide our own fts.h not FreeBSD's so the
comparator should have the NetBSD signature.

PR/57735: Jose Louis Duran: check if fts_read(3) had an error.
(FreeBSD commit https://github.com/freebsd/freebsd-src/commit/\
2dfa4b66b3d0caaaae6ce2df476b5615f8415a19)

branches: 1.76.10;
Adjust test, we only want to return the structure member if the result is not NULL.

Heads up by Sascha Wildner @ DragonFly BSD.

branches: 1.75.2; 1.75.4;
factor out common code from Poul-Henning Kamp.

use uintmax_t for nlink_t from FreeBSD

branches: 1.73.2; 1.73.16;
Pass a file pointer to write data instead of writing to stdout.
No functional change.

branches: 1.72.2;
Our <sys/param.h> ends up calling header files that define intmax_t. This
should not be the case (but >sys/param.h> is not a standard header so all bets
are off). FreeBSD's does not, so explicitly include <stdint.h> to get it.

Restore old behavior of not printing sizes for directories, since it varies
depending on filesystem implementation. (Brooks Davis)

use intmax_t instead of long long. fixes wrong cast for time_t.

add the ability to only output paths included in a spec file when creating
from a directory (-c)

Implement the "flavor" output discussed in tech-userlevel@, by Brooks Davis

make us compatible with FreeBSD (for -n) (Brooks Davies)

-n is incorrectly supressing the ".." lines. (Brooks Davis)

Allow uid and uname or gid and gname to both be printed while
preserving the behavior of falling back to [ug]id when the name is not
resolvable. (brooks)

clarify seconds.09nanoseconds format (brooks)

add -j flag to indent directory blocks (from brooks)

add -n flag to disable comments (brook)

our fts compare routine is less constified.

more const
check if we support SHA384 before using
(brooks)

branches: 1.59.2;
resource leak (Coverity issues 274383 and 274384)

branches: 1.58.6;
Make "mtree -C" sort its output.

As the input is read from a specfile into a tree of linked lists,
keep each linked list sorted. The sort order is the same as that
already used by "mtree -c": directories sort after non-directories, but
otherwise names are sorted in the order used by strcmp().

branches: 1.57.2;
When {MD5,RMD160,SHA1,SHA256_,SHA384_,SHA512_}File fail, use errno
to print what's actually wrong

fix dev_t formats.

PR/37482: Malcolm Herbert: mtree does not check for getlogin() returning null
in spec create

Renamed dsort to dcmp, since it doesn't sort, it only compares.

Even on non-BSD systems, the "-R all" option should disable printing of
the "time" attribute. Noticed by Malcolm Herbert on tech-pkg.

cast to long explicitly for the compat build

branches: 1.51.4;
The hashing routines (MD5File() and co.) can do dynamic allocation, so
take advantage of that instead of an ugly hard-coded MAXHASHLEN limit
that needs updating.

I think this was suggested by cube@ years ago.

kill crypto/rmd160.h and crypto/sha2.h, and instead make symlinks to
/usr/include from /usr/include/sys. This makes all the one way hash
header handling identical.

Use crypto/rmd160.h.

Use MAXHASHLEN here too.

Add SHA2 support in mtree.

branches: 1.46.2;
strsvis(3) encode & decode the "link=" value (symlink target).

Completely rework how tools/compat is done. Purge all uses/references to
_NETBSD_SOURCE as this makes cross building from older/newer versions of
NetBSD harder, not easier (and also makes the resulting tools 'different')

Wrap all required code with the inclusion of nbtool_config.h, attempt to
only use POSIX code in all places (or when reasonable test w. configure and
provide definitions: ala u_int, etc).

Reviewed by lukem. Tested on FreeBSD 4.9, Redhat Linux ES3, NetBSD 1.6.2 x86
NetBSD current (x86 and amd64) and Solaris 9.

Fixes PR's: PR#17762 PR#25944

branches: 1.44.2;
Overhaul how `build.sh tools' are used:

* Rename "config.h" to "nbtool_config.h" and
HAVE_CONFIG_H to HAVE_NBTOOL_CONFIG_H.
This makes in more obvious in the source when we're using
tools/compat/config.h versus "standard autoconf" config.h

* Consistently move the inclusion of nbtool_config.h to before
<sys/cdefs.h> so that the former can provide __RCSID() (et al),
and there's no need to protect those macros any more.

These changes should make it easier to "tool-ify" a program by adding:
#if HAVE_NBTOOL_CONFIG_H
#include "nbtool_config.h"
#endif
to the top of the source files (for the general case).

Move UCB-licensed code from 4-clause to 3-clause licence.

Patches provided by Joel Baker in PR 22366, verified by myself.

Add -C, which dumps the specification with the full path name first,
and then all the other fields. (I.e, like -D except with the name
first instead of last).
Consistenly strsvis(3) encode path names (even for -C and -D).

avoid non-constant initializer errors by changing hack to workaround
gcc warning to a slightly different hack.

Remove extraneous \n's in {err,warn}{,x} that used to be printfs.

Protect __RCSID and __COPYRIGHT from being invoked if not defined.

Add hooks to make mtree compilable from src/tools.

provide the ability to remove support for generating or checking various
digests by #define-ing NO_MD5, NO_RMD160, or NO_SHA1. the keywords are
still parsed.

User interface changes from (or inspired by) FreeBSD:
- Add -L to walk the tree `logically', by following symbolic links in
the heirarchy.
- Add -P to walk the tree `physically'. This is the current behaviour,
and the default.
- Add "-X excludes-file" to give mtree the ability to exclude files and
directories from its traversal. excludes-file contains fnmatch(3)
patterns to exclude from the walk.
- Add "md5digest" synonym for "md5".
- Add "rmd160" keyword for RMD-160 message digest, and "rmd160digest" synonym.
- Add "sha1" keyword for SHA-1 message digest, and "sha1digest" synonym.
- Don't try to compare() other attributes if the type doesn't match;
it's nothing but trouble, and no use anyway.
- In -c, only emit "/set" records if something has changed since the
previous one.

User interface changes by me:
- Check a device's parameters before checking uid/gid/mode.
- If updating (-u), modify the following to match the specification:
- Device type (retaining existing ownership).
- Symlink target.

Fixes from (or inspired by) FreeBSD:
- Use p->ftslevel instead of own code to keep track of the level ourself.
The previous code got majorly confused if fts(3) couldn't descend
into a subdir, resulting in leaf nodes getting attached to the wrong
directory.
XXX: This new method is much much more robust, even though it's not 100%
perfect; it might result in a couple of following entries in the spec
to be incorrectly tagged as missing.
- Pass a useful pathname to rlink(), so that logical (-L) traversal
doesn't confuse symlink checking.
- Consistently use MAXPATHLEN+1 sized buffers for pathnames, so that
there's room for the NUL.
- Use mtree_err() and strerror(p->fts_errno) to report errors during
the fts(3) walk.

Fixes by me:
- Remove now-unused `const char *name' argument from compare().
- Change crc_total from an int to a u_int32_t, to match usr.bin/cksum/crc.c.
- Remove trailing whitespace.
- Remove unnecessary (void) casts on functions.
- Reorder entries in the getopt() switch.
- Replace strtoq() with strtoll(), and use strtoul() appropriately.
- Renumber F_ flags to be in alphabetical order.

- enforce the top-most entry must be "." and must be a directory; avoids
various potential problems when intermixing full and relative paths
- changes to make it much easier to use spec() - the specfile parser - in
other programs (via .PATHing spec.c and misc.c):
- move excludetags, includetags and keys from mtree.c to misc.c
- implement mtree_err() using vwarnx() instead of assuming name is
"mtree"
- move inotype() and nodetype() from compare.c to misc.c
- add nodetoinode(), to convert from an mtree F_* type to a
mode_t S_IF* type
- clean up #include use; don't assume "mtree.h" pulls in some
standard includes
- change spec() to take a FILE * arg (where the specfile is read from)

complete WARNS=2 cleanup (with a bit of ickiness for the fts_open argv issue)

make this mostly WARNS=2 safe

Add "device" keyword, which allows the device number to be specified,
to be later checked and possibly created. Uses parsing and encoding
routines from mknod(8).

features:
- add a new keyword - "tags" - which is a comma separated list of tags
associated with that file.
- add "-E tags"; exclude files in -D output with matching tags
- add "-I tags"; only include files in -D output with matching tags
(default is to list all)

fixes:
- move all extern variable references from individual files into extern.h
- `quote' some more user-specified strings in error messages
- man page: list an options arguments in the description, and sort xrefs.

- implement -D - dump the specfile out in a manner that's parseable
by various tools
- implement -R key - remove the given key(s) from the list of keys to print
(now it's possible with "-k type -R type" to remove all keys :)
- rename ftype() to nodetype(), and make it public
- use pwcache(3) functions instead of get{gr,pw}{nam,uid}(3)
- add dump_nodes(), which provides the guts for -D.

ANSIfy, KNF, in preperation for more work.

Remove all the "#ifdef __APPLE__" code, to make this actually
build again on Darwin. The code in question is outdated.

OK'd by Wilfredo Sanchez wsanchez@MIT.EDU.

LP64 format string cleanup. Problem reported by garbled.

More format string cleanups by sommerfeld.

branches: 1.25.4;
Now that we have strsvis, use it.

Encode file names using strvis(3) when creating a specification.
Escape white space characters and `#' characters. Do the inverse
when reading specs.

This is a temporary fix until we have strzouvis(3) (or some such)
which is an improved version of strvis (taking an additional list
of characters to encode).

Fixes PR 5006 and PR 7663

branches: 1.23.4;
portability fixes:
- don't use timespecs in struct stat on non 4.4BSD systems
- don't assume that tv_sec is a time_t

Actually generate MD5 checksums, as reported in PR 7928.
Approved by Dr. Lex Wennmacher <wennmach@geo.Uni-Koeln.DE>

branches: 1.21.2;
- add -m and -i to change file flags more usefully.
- fix bugs in fileflags checking

all of the above from dr. lex wennmacher <wennmach@geo.Uni-Koeln.DE>
with some error fixes from egcs via myself.

Appease gcc-2.8.1, make printf formats more portable.

Namespace protection against MAXUID etc.

add support for 4.4BSD file flags. we take a `flags=xxx' parameter,
using the same style of naming as chflags, ls & xinstall use, plus an
additional `none' to turn off a default setting.

Use u_int32_t rather than Sys-V u_long.
Rename err() to mtree_err(), to avoid conflict with libc.

Add braces to satisfy egcs.

Add {} pairs to shut up egcs.

- use an array MAXHOSTNAMELEN+1 size to hold hostnames
- ensure hostname from gethostname() is nul-terminated in all cases
- minor KNF
- use MAXHOSTNAMELEN over various other values/defines
- be safe will buffers that hold hostnames

WARNSify, getopt returns -1 not EOF

fix printf() formats, make explicit comparisons to NULL (gcc -Wall)

Only record sizes for regular files.

branches: 1.10.4;
Rename struct timespec fields to conform to POSIX.1b

add NetBSD id's

clean up import, fix a few bugs, etc.

off_t exp.

make things a bit more consistent

latest mtree from uunet. doesn't compile, but that'll be fixed soon.
(want to have 'pseudo-virgin' sources, for when fixed fnmatch stuff
is released...)

Use *->fts_errno instead of errno after fts_read(). The fts manpage
indicates that the fts_errno will be set to the correct value, but there
are no guarentees about errno.

update for fts library change

Add RCS identifiers.

branches: 1.1.1;
Initial revision

Completely rework how tools/compat is done. Purge all uses/references to
_NETBSD_SOURCE as this makes cross building from older/newer versions of
NetBSD harder, not easier (and also makes the resulting tools 'different')

Wrap all required code with the inclusion of nbtool_config.h, attempt to
only use POSIX code in all places (or when reasonable test w. configure and
provide definitions: ala u_int, etc).

Reviewed by lukem. Tested on FreeBSD 4.9, Redhat Linux ES3, NetBSD 1.6.2 x86
NetBSD current (x86 and amd64) and Solaris 9.

Fixes PR's: PR#17762 PR#25944

branches: 1.12.2;
#include <util.h>
this is needed by fparseln on macos/darwin and is harmless on netbsd

tweaks for fparseln(3) move from libutil to libc:
- remove #include <util.h> if nothing else needed it
- remove LDFLAGS+=-lutil if nothing else needed it

Don't use __RCSID unless it's defined.

Use __RCSID().

Roll in fixes to permit cross-compiling from non-NetBSD hosts. This
round has been tested on Solaris/x86 and Linux hosts.

* Add host tools cap_mkdb, ctags, m4, uudecode.
* Protect __RCSID() and __COPYRIGHT() better.
* Reduce the number of places that need to include "config.h", to keep
sources closer to their "vanilla" versions.
* Add more compat #defines and autoconf-checked functions.

Make almost all tools compile and run properly on non-NetBSD hosts. (In
particular, most tools now run correctly on Solaris 7.)

add <util.h> back in for native builds.

Add hooks to make mtree compilable from src/tools.

Ugg...Turn LIST_FOREACH into a while loop around LIST_NEXT...

The *_FOREACH macros got added post 1.5 and were pulled up into 1.5.2. This
makes it impossible to compile mtree local on a 1.5-release box as a cross
toolchain bit for bootstrapping to -current. (and why I never saw this problem
on my 1.5.2 machine)

This whole thing is solved via a compat lib for the tools for all netbsd bits
and/or trimming down a version of mtree to just the bits needed for building
the snapshots.

replace <sys/time.h> with <time.h> (for consistency)

- KNF & whitespace cleanup
- use fparseln() instead of fgetln()

branches: 1.1.1;
Initial revision

PR/58875: Jose Luis Duran: Produce consistent checksums in verification
by scanning directories in the same order as usual. While here, fix some
incorrect types.

branches: 1.40.8;
mtree: use POSIX type uint32_t instead of u_int32_t

Pass a file pointer to write data instead of writing to stdout.
No functional change.

branches: 1.38.6;
add the ability to only output paths included in a spec file when creating
from a directory (-c)

Implement the "flavor" output discussed in tech-userlevel@, by Brooks Davis

Implement FreeBSD's -f spec1 -f spec2 spec comparision function. This
is more useful than -C and diff because it highlights the keys that have
changed value so for example, you can see that the size of a file has
not changed but the contents have. (brooks)

add -j flag to indent directory blocks (from brooks)

add -q flag to silence warnings about symlinks to existing dirs (from OpenBSD
via brooks)

add -n flag to disable comments (brook)

branches: 1.32.2; 1.32.8;
Use __dead and __printflike

The recent addition of sorting to "mtree -C" broke some third party
scripts. Now make sorting optional, controled by the "-S" flag.

branches: 1.30.20;
Complete the move of string_to_flags() and flags_to_string() from the
bin/ls sources to libutil:
o Bump libutil minor version number
o Fix uses to include <util.h> to pick up the function definitions
o Fix most uses of flags_to_string() to release the now-malloc()ed result

Implement -M to "merge" entries, even with different types.
This can be used in /etc/security to allow special.local to override
types of entries in special (e.g, replacing a dir with a link).

Rename Wflag to mtree_Wflag, to be more "external namespace" friendly.

Overhaul how `build.sh tools' are used:

* Rename "config.h" to "nbtool_config.h" and
HAVE_CONFIG_H to HAVE_NBTOOL_CONFIG_H.
This makes in more obvious in the source when we're using
tools/compat/config.h versus "standard autoconf" config.h

* Consistently move the inclusion of nbtool_config.h to before
<sys/cdefs.h> so that the former can provide __RCSID() (et al),
and there's no need to protect those macros any more.

These changes should make it easier to "tool-ify" a program by adding:
#if HAVE_NBTOOL_CONFIG_H
#include "nbtool_config.h"
#endif
to the top of the source files (for the general case).

Move UCB-licensed code from 4-clause to 3-clause licence.

Patches provided by Joel Baker in PR 22366, verified by myself.

Add -C, which dumps the specification with the full path name first,
and then all the other fields. (I.e, like -D except with the name
first instead of last).
Consistenly strsvis(3) encode path names (even for -C and -D).

Remove convoluted code to work around missing lch{flags,mod,own}();
instead, rely upon libnbcompat to provide a workaround (which returns
an error if the target is a symlink).
Should fix another MacOS X build issue due to missing lchown(2).

- Check HAVE_LCHFLAGS for lchflags(2) support, rather than assuming
that HAVE_STRUCT_STAT_ST_FLAGS implies this.
- Set HAVE_LCHFLAGS for native builds
- Clean up {CLEAR,SET,CHANGE}FLAGS macros, and only provide if
HAVE_STRUCT_STAT_ST_FLAGS is set.

(Fixes compilation as a tool on MacOS X, noted by Allen Briggs.)

Funtion appropriately in the absence of a lchmod call on the host.

Roll in fixes to permit cross-compiling from non-NetBSD hosts. This
round has been tested on Solaris/x86 and Linux hosts.

* Add host tools cap_mkdb, ctags, m4, uudecode.
* Protect __RCSID() and __COPYRIGHT() better.
* Reduce the number of places that need to include "config.h", to keep
sources closer to their "vanilla" versions.
* Add more compat #defines and autoconf-checked functions.

Make almost all tools compile and run properly on non-NetBSD hosts. (In
particular, most tools now run correctly on Solaris 7.)

Add hooks to make mtree compilable from src/tools.

User interface changes from (or inspired by) FreeBSD:
- Add -L to walk the tree `logically', by following symbolic links in
the heirarchy.
- Add -P to walk the tree `physically'. This is the current behaviour,
and the default.
- Add "-X excludes-file" to give mtree the ability to exclude files and
directories from its traversal. excludes-file contains fnmatch(3)
patterns to exclude from the walk.
- Add "md5digest" synonym for "md5".
- Add "rmd160" keyword for RMD-160 message digest, and "rmd160digest" synonym.
- Add "sha1" keyword for SHA-1 message digest, and "sha1digest" synonym.
- Don't try to compare() other attributes if the type doesn't match;
it's nothing but trouble, and no use anyway.
- In -c, only emit "/set" records if something has changed since the
previous one.

User interface changes by me:
- Check a device's parameters before checking uid/gid/mode.
- If updating (-u), modify the following to match the specification:
- Device type (retaining existing ownership).
- Symlink target.

Fixes from (or inspired by) FreeBSD:
- Use p->ftslevel instead of own code to keep track of the level ourself.
The previous code got majorly confused if fts(3) couldn't descend
into a subdir, resulting in leaf nodes getting attached to the wrong
directory.
XXX: This new method is much much more robust, even though it's not 100%
perfect; it might result in a couple of following entries in the spec
to be incorrectly tagged as missing.
- Pass a useful pathname to rlink(), so that logical (-L) traversal
doesn't confuse symlink checking.
- Consistently use MAXPATHLEN+1 sized buffers for pathnames, so that
there's room for the NUL.
- Use mtree_err() and strerror(p->fts_errno) to report errors during
the fts(3) walk.

Fixes by me:
- Remove now-unused `const char *name' argument from compare().
- Change crc_total from an int to a u_int32_t, to match usr.bin/cksum/crc.c.
- Remove trailing whitespace.
- Remove unnecessary (void) casts on functions.
- Reorder entries in the getopt() switch.
- Replace strtoq() with strtoll(), and use strtoul() appropriately.
- Renumber F_ flags to be in alphabetical order.

- move "Wflag" global from mtree.c to spec.c, and reinstate the functionality
of tv's previous code, which skips uname or gname parsing if -W is enabled.
- rename "lineno" to "mtree_lineno", to reduce possibility of name
clashes in code that yanks in spec.c & misc.c (unlikely, but you never know)

Add -W: don't attempt to change the perms/time/flags on existing entries, nor
set them when creating new directories. (AKA the ``don't whack'' option).
Concept suggested by Todd Vierling.

- add lineno field to NODE
- move inotype() prototype into mtree.h from extern.h

- enforce the top-most entry must be "." and must be a directory; avoids
various potential problems when intermixing full and relative paths
- changes to make it much easier to use spec() - the specfile parser - in
other programs (via .PATHing spec.c and misc.c):
- move excludetags, includetags and keys from mtree.c to misc.c
- implement mtree_err() using vwarnx() instead of assuming name is
"mtree"
- move inotype() and nodetype() from compare.c to misc.c
- add nodetoinode(), to convert from an mtree F_* type to a
mode_t S_IF* type
- clean up #include use; don't assume "mtree.h" pulls in some
standard includes
- change spec() to take a FILE * arg (where the specfile is read from)

cleanup tags support;
- create a typedef struct slist_t to hold stuff in (avoid stringlist(3)
because it's less portable)
- move the tag manipulate stuff into misc.c
- add matchtags(), which returns 0 if the node is explicitly excluded,
or not included if an include list is given; or 1 otherwise
- in -D, change tags support from files-only to all-non-directories

(xxx; still considering implementing tags support for -c and !-D)

features:
- add a new keyword - "tags" - which is a comma separated list of tags
associated with that file.
- add "-E tags"; exclude files in -D output with matching tags
- add "-I tags"; only include files in -D output with matching tags
(default is to list all)

fixes:
- move all extern variable references from individual files into extern.h
- `quote' some more user-specified strings in error messages
- man page: list an options arguments in the description, and sort xrefs.

- implement -D - dump the specfile out in a manner that's parseable
by various tools
- implement -R key - remove the given key(s) from the list of keys to print
(now it's possible with "-k type -R type" to remove all keys :)
- rename ftype() to nodetype(), and make it public
- use pwcache(3) functions instead of get{gr,pw}{nam,uid}(3)
- add dump_nodes(), which provides the guts for -D.

- add parsetype() (a la parsekey()) to parse the "type=xxx" arg. this improves
the error detection for invalid types, as well as shrinking binary size
- implement inotype() using ftype(), rather than duplicating the switch
- change "char *" arguments to "const char *"

ANSIfy, KNF, in preperation for more work.

Remove all the "#ifdef __APPLE__" code, to make this actually
build again on Darwin. The code in question is outdated.

OK'd by Wilfredo Sanchez wsanchez@MIT.EDU.

More format string cleanups by sommerfeld.

branches: 1.7.4;
code.c is no longer used, we use strsvis instead. Removed prototypes
from extern.h

prototype encode()/decode()

branches: 1.5.2; 1.5.6;
add support for 4.4BSD file flags. we take a `flags=xxx' parameter,
using the same style of naming as chflags, ls & xinstall use, plus an
additional `none' to turn off a default setting.

Use u_int32_t rather than Sys-V u_long.
Rename err() to mtree_err(), to avoid conflict with libc.

add NetBSD id's

clean up import, fix a few bugs, etc.

branches: 1.1.1;
latest mtree from uunet. doesn't compile, but that'll be fixed soon.
(want to have 'pseudo-virgin' sources, for when fixed fnmatch stuff
is released...)

fix pasto; thanks John-Mark Gurney...

say why we can't use the password or the group databases.

skip comment lines on password and group databases (Brooks Davis)

branches: 1.7.4; 1.7.26;
Remove clause 3 and 4 from TNF licenses

branches: 1.6.4;
use __RCSID()

branches: 1.5.16;
Completely rework how tools/compat is done. Purge all uses/references to
_NETBSD_SOURCE as this makes cross building from older/newer versions of
NetBSD harder, not easier (and also makes the resulting tools 'different')

Wrap all required code with the inclusion of nbtool_config.h, attempt to
only use POSIX code in all places (or when reasonable test w. configure and
provide definitions: ala u_int, etc).

Reviewed by lukem. Tested on FreeBSD 4.9, Redhat Linux ES3, NetBSD 1.6.2 x86
NetBSD current (x86 and amd64) and Solaris 9.

Fixes PR's: PR#17762 PR#25944

branches: 1.4.2;
Move UCB-licensed code from 4-clause to 3-clause licence.

Patches provided by Joel Baker in PR 22366, verified by myself.

Make almost all tools compile and run properly on non-NetBSD hosts. (In
particular, most tools now run correctly on Solaris 7.)

don't bother initialising pwfile[] to _PATH_MASTERPASSWD and grfile[]
to _PATH_GROUP, because the simple_get{pw,gr}*() functions should not
be called by anything before those path arrays are setup by setup_getid().

Add '-N dbdir', to specify that user & group lookups (via pwcache(3)) are to
be done from dbdir/master.passwd and dbdir/group, using simple text file
parsing routines.

PR/58875: Jose Luis Duran: Produce consistent checksums in verification
by scanning directories in the same order as usual. While here, fix some
incorrect types.

branches: 1.34.42;
move flavor to misc.c because pax needs it.

- two entries for rmd160digest in misc.c (Brooks Davis)

Implement FreeBSD's "nochange" keyworld which causes the existance of a
filesystem object to be checked but none of its attiributes.(brooks)

account for differences in digest names

branches: 1.30.8; 1.30.14;
fix -Wsign-compare issues

c99 initializers

PR/34818: rad at twig dot com: Recognize solaris door files.

The list of keywords must be sorted because it is search with bsearch(3).
Correcting this makes mtree(8) work again under Solaris.

Add SHA2 support in mtree.

branches: 1.25.2;
Completely rework how tools/compat is done. Purge all uses/references to
_NETBSD_SOURCE as this makes cross building from older/newer versions of
NetBSD harder, not easier (and also makes the resulting tools 'different')

Wrap all required code with the inclusion of nbtool_config.h, attempt to
only use POSIX code in all places (or when reasonable test w. configure and
provide definitions: ala u_int, etc).

Reviewed by lukem. Tested on FreeBSD 4.9, Redhat Linux ES3, NetBSD 1.6.2 x86
NetBSD current (x86 and amd64) and Solaris 9.

Fixes PR's: PR#17762 PR#25944

branches: 1.24.2;
Move UCB-licensed code from 4-clause to 3-clause licence.

Patches provided by Joel Baker in PR 22366, verified by myself.

inotype() only needs to understand S_ types, not the F_ types

Protect __RCSID and __COPYRIGHT from being invoked if not defined.

Add hooks to make mtree compilable from src/tools.

User interface changes from (or inspired by) FreeBSD:
- Add -L to walk the tree `logically', by following symbolic links in
the heirarchy.
- Add -P to walk the tree `physically'. This is the current behaviour,
and the default.
- Add "-X excludes-file" to give mtree the ability to exclude files and
directories from its traversal. excludes-file contains fnmatch(3)
patterns to exclude from the walk.
- Add "md5digest" synonym for "md5".
- Add "rmd160" keyword for RMD-160 message digest, and "rmd160digest" synonym.
- Add "sha1" keyword for SHA-1 message digest, and "sha1digest" synonym.
- Don't try to compare() other attributes if the type doesn't match;
it's nothing but trouble, and no use anyway.
- In -c, only emit "/set" records if something has changed since the
previous one.

User interface changes by me:
- Check a device's parameters before checking uid/gid/mode.
- If updating (-u), modify the following to match the specification:
- Device type (retaining existing ownership).
- Symlink target.

Fixes from (or inspired by) FreeBSD:
- Use p->ftslevel instead of own code to keep track of the level ourself.
The previous code got majorly confused if fts(3) couldn't descend
into a subdir, resulting in leaf nodes getting attached to the wrong
directory.
XXX: This new method is much much more robust, even though it's not 100%
perfect; it might result in a couple of following entries in the spec
to be incorrectly tagged as missing.
- Pass a useful pathname to rlink(), so that logical (-L) traversal
doesn't confuse symlink checking.
- Consistently use MAXPATHLEN+1 sized buffers for pathnames, so that
there's room for the NUL.
- Use mtree_err() and strerror(p->fts_errno) to report errors during
the fts(3) walk.

Fixes by me:
- Remove now-unused `const char *name' argument from compare().
- Change crc_total from an int to a u_int32_t, to match usr.bin/cksum/crc.c.
- Remove trailing whitespace.
- Remove unnecessary (void) casts on functions.
- Reorder entries in the getopt() switch.
- Replace strtoq() with strtoll(), and use strtoul() appropriately.
- Renumber F_ flags to be in alphabetical order.

- move "Wflag" global from mtree.c to spec.c, and reinstate the functionality
of tv's previous code, which skips uname or gname parsing if -W is enabled.
- rename "lineno" to "mtree_lineno", to reduce possibility of name
clashes in code that yanks in spec.c & misc.c (unlikely, but you never know)

- enforce the top-most entry must be "." and must be a directory; avoids
various potential problems when intermixing full and relative paths
- changes to make it much easier to use spec() - the specfile parser - in
other programs (via .PATHing spec.c and misc.c):
- move excludetags, includetags and keys from mtree.c to misc.c
- implement mtree_err() using vwarnx() instead of assuming name is
"mtree"
- move inotype() and nodetype() from compare.c to misc.c
- add nodetoinode(), to convert from an mtree F_* type to a
mode_t S_IF* type
- clean up #include use; don't assume "mtree.h" pulls in some
standard includes
- change spec() to take a FILE * arg (where the specfile is read from)

make this mostly WARNS=2 safe

Add "device" keyword, which allows the device number to be specified,
to be later checked and possibly created. Uses parsing and encoding
routines from mknod(8).

- support "all" as a valid keyword for -k, -K and -R.
- parsekey(): support "all" as a keyword which returns a value with all bits
set that pertain to a valid keyword.
- dump_nodes():
- only print out a keyword if it's requested AND it's set in the node
- UNAME falls back to UID if the user name can't be determined
- GNAME falls back to GID if the group name can't be determined
- rework man page:
- use Sy instead of Cm
- use Pq, Dq, Ql, etc instead of ``...''

cleanup tags support;
- create a typedef struct slist_t to hold stuff in (avoid stringlist(3)
because it's less portable)
- move the tag manipulate stuff into misc.c
- add matchtags(), which returns 0 if the node is explicitly excluded,
or not included if an include list is given; or 1 otherwise
- in -D, change tags support from files-only to all-non-directories

(xxx; still considering implementing tags support for -c and !-D)

features:
- add a new keyword - "tags" - which is a comma separated list of tags
associated with that file.
- add "-E tags"; exclude files in -D output with matching tags
- add "-I tags"; only include files in -D output with matching tags
(default is to list all)

fixes:
- move all extern variable references from individual files into extern.h
- `quote' some more user-specified strings in error messages
- man page: list an options arguments in the description, and sort xrefs.

Fix printf format.

replace home-grown parser with fparseln()

- add parsetype() (a la parsekey()) to parse the "type=xxx" arg. this improves
the error detection for invalid types, as well as shrinking binary size
- implement inotype() using ftype(), rather than duplicating the switch
- change "char *" arguments to "const char *"

ANSIfy, KNF, in preperation for more work.

Add an `md5' keyword to mtree, which takes as an argument the MD5 cryptographic
checksum of the file in question, optionally preceeded by `0x'.

This has been tested, and does work, AFAICT. There is currently no md5 equivalent
of the '-s' flag for use with checksum.

add support for 4.4BSD file flags. we take a `flags=xxx' parameter,
using the same style of naming as chflags, ls & xinstall use, plus an
additional `none' to turn off a default setting.

Use u_int32_t rather than Sys-V u_long.
Rename err() to mtree_err(), to avoid conflict with libc.

WARNSify, getopt returns -1 not EOF

add an 'optional' keyword, which allows files to be in the
specification, but _not_ in the file hierarchy. contrib'd by:
Ed Symanzik <zik@enoch.cl.msu.edu>. Man page adjustment to match, and
minor man page cleanup, by me.

add NetBSD id's

clean up import, fix a few bugs, etc.

branches: 1.1.1;
latest mtree from uunet. doesn't compile, but that'll be fixed soon.
(want to have 'pseudo-virgin' sources, for when fixed fnmatch stuff
is released...)

Clean up deleted files.

Add RCS indentifiers.

branches: 1.1.1;
Initial revision

PR/57735: Jose Louis Duran: Add missing xref to mtree.5
(FreeBSD commit https://github.com/freebsd/freebsd-src/commit/\
da232c04ab9eba3d3a7fc73881daff847071e9fd)

mtree(8): a few markup fixes

Didn't touch the section that describes the spec file format. We have
the format documented here, in mtree(8), and we also have the format
description in mtree(5) that comes from libarchive.

Use more markup. Bump date for previous.

Describe correctly how whitespace is formatted as the docs did not
match the source.

Use \(en.

Fix naming inconsistency; PR 53769 from silas.

branches: 1.72.10; 1.72.12;
Fix typo.

branches: 1.71.2; 1.71.4;
Pull into this year.

attempt to reset file flags when -rr

fix man page.

add the ability to only output paths included in a spec file when creating
from a directory (-c)

Typo fix.

Add missing letter and use x.y versions for Fx/Nx.

Implement the "flavor" output discussed in tech-userlevel@, by Brooks Davis

- No mention of -t in the usage message or the manpage SYNOPSIS (Brooks Davis)

Fix typos; remove trailing whitespace; add missing options to SYNOPSIS.

Implement FreeBSD's "nochange" keyworld which causes the existance of a
filesystem object to be checked but none of its attiributes.(brooks)

Implement FreeBSD's "nochange" keyworld which causes the existance of a
filesystem object to be checked but none of its attributes. (brooks)

Implement FreeBSD's -f spec1 -f spec2 spec comparision function. This
is more useful than -C and diff because it highlights the keys that have
changed value so for example, you can see that the size of a file has
not changed but the contents have. (brooks)

clarify seconds.09nanoseconds format (brooks)

add -j flag to indent directory blocks (from brooks)

add -q flag to silence warnings about symlinks to existing dirs (from OpenBSD
via brooks)

add -n flag to disable comments (brook)

account for differences in digest names

- Sort order of options mentioned in text;
- improve wording;
- remove unneeded macros (fixes mandoc(1) warnings);
- in the `EXAMPLES' section, for `-d', mention the use of `-U'
besides that of `-u' (as the former is a variant on the latter).

From Bug Hunting.

branches: 1.53.6; 1.53.12;
Fix date.

Document -t (modify mtime). Bump date.

* Rewrite intro to mention all three things mtree can do (check directory
tree against spec; create spec; modify spec).
* Consistently use "options" to refer to command line options. "flags"
is used to refer to file flags.
* Some other clarifications and changes for consistency.

Sort options and option descriptions.
Remove an incomplete sentence ("It will") -- probably pasto.
Give the tagged lists more space for the first column.
Remove unnecessary Bk/Ek pairs.

The recent addition of sorting to "mtree -C" broke some third party
scripts. Now make sorting optional, controled by the "-S" flag.

Don't use .Xo/.Xc. Replace use of .Sm off / .Sm on with .Ns.

branches: 1.47.2;
New sentence, new line.

-c: make it clear where the file hierarchy starts
-I, -E: they also work with -C, not just with -D

Move TNF licenses to 2 clause form

branches: 1.44.16;
describe how to suppress printing of "type" keyword when -k is used.
bump date.

i.e. without a space. PR 33328 by Igor Sobrado.

SHA2 support has been pulled up to NetBSD 3.0, update the history part.

Add SHA2 support in mtree.

Reference cksum(1) only.

branches: 1.39.2;
Implement -M to "merge" entries, even with different types.
This can be used in /etc/security to allow special.local to override
types of entries in special (e.g, replacing a dir with a link).

Rename Wflag to mtree_Wflag, to be more "external namespace" friendly.

Move UCB-licensed code from 4-clause to 3-clause licence.

Patches provided by Joel Baker in PR 22366, verified by myself.

Quote some characters.

grammar tweak.

.Nm does not need a dummy argument ("") before punctuation or
for correct formatting of the SYNOPSIS any longer.

File types belong under 'type', not 'uname'.

Add -C, which dumps the specification with the full path name first,
and then all the other fields. (I.e, like -D except with the name
first instead of last).
Consistenly strsvis(3) encode path names (even for -C and -D).

correct description of -D

support multiple entries for "." and full paths if the types match; the last
listed entry's settings override the existing settings for that node.

Support `/unset all', to unset all of the global values set with `/set ...'

Add '-N dbdir', to specify that user & group lookups (via pwcache(3)) are to
be done from dbdir/master.passwd and dbdir/group, using simple text file
parsing routines.

Whitespace nits.

User interface changes from (or inspired by) FreeBSD:
- Add -L to walk the tree `logically', by following symbolic links in
the heirarchy.
- Add -P to walk the tree `physically'. This is the current behaviour,
and the default.
- Add "-X excludes-file" to give mtree the ability to exclude files and
directories from its traversal. excludes-file contains fnmatch(3)
patterns to exclude from the walk.
- Add "md5digest" synonym for "md5".
- Add "rmd160" keyword for RMD-160 message digest, and "rmd160digest" synonym.
- Add "sha1" keyword for SHA-1 message digest, and "sha1digest" synonym.
- Don't try to compare() other attributes if the type doesn't match;
it's nothing but trouble, and no use anyway.
- In -c, only emit "/set" records if something has changed since the
previous one.

User interface changes by me:
- Check a device's parameters before checking uid/gid/mode.
- If updating (-u), modify the following to match the specification:
- Device type (retaining existing ownership).
- Symlink target.

Fixes from (or inspired by) FreeBSD:
- Use p->ftslevel instead of own code to keep track of the level ourself.
The previous code got majorly confused if fts(3) couldn't descend
into a subdir, resulting in leaf nodes getting attached to the wrong
directory.
XXX: This new method is much much more robust, even though it's not 100%
perfect; it might result in a couple of following entries in the spec
to be incorrectly tagged as missing.
- Pass a useful pathname to rlink(), so that logical (-L) traversal
doesn't confuse symlink checking.
- Consistently use MAXPATHLEN+1 sized buffers for pathnames, so that
there's room for the NUL.
- Use mtree_err() and strerror(p->fts_errno) to report errors during
the fts(3) walk.

Fixes by me:
- Remove now-unused `const char *name' argument from compare().
- Change crc_total from an int to a u_int32_t, to match usr.bin/cksum/crc.c.
- Remove trailing whitespace.
- Remove unnecessary (void) casts on functions.
- Reorder entries in the getopt() switch.
- Replace strtoq() with strtoll(), and use strtoul() appropriately.
- Renumber F_ flags to be in alphabetical order.

Add -W: don't attempt to change the perms/time/flags on existing entries, nor
set them when creating new directories. (AKA the ``don't whack'' option).
Concept suggested by Todd Vierling.

document requirement about first node being a directory called "."

improve enumeration of the different specfile entry types

Add "device" keyword, which allows the device number to be specified,
to be later checked and possibly created. Uses parsing and encoding
routines from mknod(8).

- use "full path" instead of "absolute path"; perry thinks it's less confusing
- clarify the `all' keyword support in -k, -K and -R. should solve [misc/14186]
- use ".sp" instead of ".Pp" (from FreeBSD).

* Implement support for absolute paths in the specfile. An absolute path
is one that contains a `/' character that is not the first character.
All parent directories referenced in the path must exist.
* Add copyright for all the work I've done.
* Document the history of various features added since 4.3-Reno.

- support "all" as a valid keyword for -k, -K and -R.
- parsekey(): support "all" as a keyword which returns a value with all bits
set that pertain to a valid keyword.
- dump_nodes():
- only print out a keyword if it's requested AND it's set in the node
- UNAME falls back to UID if the user name can't be determined
- GNAME falls back to GID if the group name can't be determined
- rework man page:
- use Sy instead of Cm
- use Pq, Dq, Ql, etc instead of ``...''

cleanup tags support;
- create a typedef struct slist_t to hold stuff in (avoid stringlist(3)
because it's less portable)
- move the tag manipulate stuff into misc.c
- add matchtags(), which returns 0 if the node is explicitly excluded,
or not included if an include list is given; or 1 otherwise
- in -D, change tags support from files-only to all-non-directories

(xxx; still considering implementing tags support for -c and !-D)

Whitespace fixes, sort sections.

features:
- add a new keyword - "tags" - which is a comma separated list of tags
associated with that file.
- add "-E tags"; exclude files in -D output with matching tags
- add "-I tags"; only include files in -D output with matching tags
(default is to list all)

fixes:
- move all extern variable references from individual files into extern.h
- `quote' some more user-specified strings in error messages
- man page: list an options arguments in the description, and sort xrefs.

- implement -D - dump the specfile out in a manner that's parseable
by various tools
- implement -R key - remove the given key(s) from the list of keys to print
(now it's possible with "-k type -R type" to remove all keys :)
- rename ftype() to nodetype(), and make it public
- use pwcache(3) functions instead of get{gr,pw}{nam,uid}(3)
- add dump_nodes(), which provides the guts for -D.

Add a -l flag which does "loose" permissions checks, i.e. a mode 444
matches a check for mode 644 (though obviously not the reverse). This
can be used by the nightly security run, making the output1 more useful
by having it contain fewer spurious permissions violations.

Note that I did not make -l work if you have a sgid/suid/sticky bit
set. I don't know how you could cause security trouble with more
stringent settings and a suid file, but I don't want to find out the
hard way.

use .Xr

mtree uses strsvis, not strvis.

branches: 1.12.4;
Encode file names using strvis(3) when creating a specification.
Escape white space characters and `#' characters. Do the inverse
when reading specs.

This is a temporary fix until we have strzouvis(3) (or some such)
which is an improved version of strvis (taking an additional list
of characters to encode).

Fixes PR 5006 and PR 7663

branches: 1.11.2; 1.11.6;
Clean up SYNOPSIS formatting.

- add -m and -i to change file flags more usefully.
- fix bugs in fileflags checking

all of the above from dr. lex wennmacher <wennmach@geo.Uni-Koeln.DE>
with some error fixes from egcs via myself.

Add an `md5' keyword to mtree, which takes as an argument the MD5 cryptographic
checksum of the file in question, optionally preceeded by `0x'.

This has been tested, and does work, AFAICT. There is currently no md5 equivalent
of the '-s' flag for use with checksum.

add support for 4.4BSD file flags. we take a `flags=xxx' parameter,
using the same style of naming as chflags, ls & xinstall use, plus an
additional `none' to turn off a default setting.

WARNSify, getopt returns -1 not EOF

Add -U flag, checking for mismatch. This is the same as the old -u flag,
except that an error which was corrected is deemed not to be an error.

Necessary for use in the FreeBSD ports system.

xref chown(8) not chown(1), kill extra comma

add an 'optional' keyword, which allows files to be in the
specification, but _not_ in the file hierarchy. contrib'd by:
Ed Symanzik <zik@enoch.cl.msu.edu>. Man page adjustment to match, and
minor man page cleanup, by me.

add NetBSD id's

clean up import, fix a few bugs, etc.

branches: 1.1.1;
latest mtree from uunet. doesn't compile, but that'll be fixed soon.
(want to have 'pseudo-virgin' sources, for when fixed fnmatch stuff
is released...)

PR/59838: Jose Louis Duran: Save the R mask and apply it at the end so the
order of -k -K -R does not matter.

PR/58875: Jose Luis Duran: Produce consistent checksums in verification
by scanning directories in the same order as usual. While here, fix some
incorrect types.

branches: 1.50.28;
attempt to reset file flags when -rr

Pass a file pointer to write data instead of writing to stdout.
No functional change.

branches: 1.48.4;
move flags only used in main() inside main(), from Ed Schouten

add the ability to only output paths included in a spec file when creating
from a directory (-c)

move flavor to misc.c because pax needs it.

joerg@: just use unsigned int please

Implement the "flavor" output discussed in tech-userlevel@, by Brooks Davis

- No "j" in the getopt string. (Brooks Davis)

Sync usage with man page.

Implement FreeBSD's -f spec1 -f spec2 spec comparision function. This
is more useful than -C and diff because it highlights the keys that have
changed value so for example, you can see that the size of a file has
not changed but the contents have. (brooks)

add -j flag to indent directory blocks (from brooks)

add -q flag to silence warnings about symlinks to existing dirs (from OpenBSD
via brooks)

add -n flag to disable comments (brook)

branches: 1.37.2; 1.37.8;
Use __dead and __printflike

Sync usage with man page.

The recent addition of sorting to "mtree -C" broke some third party
scripts. Now make sorting optional, controled by the "-S" flag.

branches: 1.34.6;
Remove the \n and tabs from the __COPYRIGHT() strings.
Tweak to use a consistent format.

branches: 1.33.12;
Fix a typo in a boolean test, which used & instead of &&.

Implement -M to "merge" entries, even with different types.
This can be used in /etc/security to allow special.local to override
types of entries in special (e.g, replacing a dir with a link).

Rename Wflag to mtree_Wflag, to be more "external namespace" friendly.

Completely rework how tools/compat is done. Purge all uses/references to
_NETBSD_SOURCE as this makes cross building from older/newer versions of
NetBSD harder, not easier (and also makes the resulting tools 'different')

Wrap all required code with the inclusion of nbtool_config.h, attempt to
only use POSIX code in all places (or when reasonable test w. configure and
provide definitions: ala u_int, etc).

Reviewed by lukem. Tested on FreeBSD 4.9, Redhat Linux ES3, NetBSD 1.6.2 x86
NetBSD current (x86 and amd64) and Solaris 9.

Fixes PR's: PR#17762 PR#25944

branches: 1.30.2;
Move UCB-licensed code from 4-clause to 3-clause licence.

Patches provided by Joel Baker in PR 22366, verified by myself.

Add -C, which dumps the specification with the full path name first,
and then all the other fields. (I.e, like -D except with the name
first instead of last).
Consistenly strsvis(3) encode path names (even for -C and -D).

Protect __RCSID and __COPYRIGHT from being invoked if not defined.

Add hooks to make mtree compilable from src/tools.

Add '-N dbdir', to specify that user & group lookups (via pwcache(3)) are to
be done from dbdir/master.passwd and dbdir/group, using simple text file
parsing routines.

Generally don't need the extra byte in MAXPATHLEN sized buffers, because
pathnames are guaranteed by POSIX to be MAXPATHLEN-1 bytes long.
Kindly pointed out by Warner Losh in private email.

User interface changes from (or inspired by) FreeBSD:
- Add -L to walk the tree `logically', by following symbolic links in
the heirarchy.
- Add -P to walk the tree `physically'. This is the current behaviour,
and the default.
- Add "-X excludes-file" to give mtree the ability to exclude files and
directories from its traversal. excludes-file contains fnmatch(3)
patterns to exclude from the walk.
- Add "md5digest" synonym for "md5".
- Add "rmd160" keyword for RMD-160 message digest, and "rmd160digest" synonym.
- Add "sha1" keyword for SHA-1 message digest, and "sha1digest" synonym.
- Don't try to compare() other attributes if the type doesn't match;
it's nothing but trouble, and no use anyway.
- In -c, only emit "/set" records if something has changed since the
previous one.

User interface changes by me:
- Check a device's parameters before checking uid/gid/mode.
- If updating (-u), modify the following to match the specification:
- Device type (retaining existing ownership).
- Symlink target.

Fixes from (or inspired by) FreeBSD:
- Use p->ftslevel instead of own code to keep track of the level ourself.
The previous code got majorly confused if fts(3) couldn't descend
into a subdir, resulting in leaf nodes getting attached to the wrong
directory.
XXX: This new method is much much more robust, even though it's not 100%
perfect; it might result in a couple of following entries in the spec
to be incorrectly tagged as missing.
- Pass a useful pathname to rlink(), so that logical (-L) traversal
doesn't confuse symlink checking.
- Consistently use MAXPATHLEN+1 sized buffers for pathnames, so that
there's room for the NUL.
- Use mtree_err() and strerror(p->fts_errno) to report errors during
the fts(3) walk.

Fixes by me:
- Remove now-unused `const char *name' argument from compare().
- Change crc_total from an int to a u_int32_t, to match usr.bin/cksum/crc.c.
- Remove trailing whitespace.
- Remove unnecessary (void) casts on functions.
- Reorder entries in the getopt() switch.
- Replace strtoq() with strtoll(), and use strtoul() appropriately.
- Renumber F_ flags to be in alphabetical order.

- move "Wflag" global from mtree.c to spec.c, and reinstate the functionality
of tv's previous code, which skips uname or gname parsing if -W is enabled.
- rename "lineno" to "mtree_lineno", to reduce possibility of name
clashes in code that yanks in spec.c & misc.c (unlikely, but you never know)

Add -W: don't attempt to change the perms/time/flags on existing entries, nor
set them when creating new directories. (AKA the ``don't whack'' option).
Concept suggested by Todd Vierling.

- enforce the top-most entry must be "." and must be a directory; avoids
various potential problems when intermixing full and relative paths
- changes to make it much easier to use spec() - the specfile parser - in
other programs (via .PATHing spec.c and misc.c):
- move excludetags, includetags and keys from mtree.c to misc.c
- implement mtree_err() using vwarnx() instead of assuming name is
"mtree"
- move inotype() and nodetype() from compare.c to misc.c
- add nodetoinode(), to convert from an mtree F_* type to a
mode_t S_IF* type
- clean up #include use; don't assume "mtree.h" pulls in some
standard includes
- change spec() to take a FILE * arg (where the specfile is read from)

cleanup tags support;
- create a typedef struct slist_t to hold stuff in (avoid stringlist(3)
because it's less portable)
- move the tag manipulate stuff into misc.c
- add matchtags(), which returns 0 if the node is explicitly excluded,
or not included if an include list is given; or 1 otherwise
- in -D, change tags support from files-only to all-non-directories

(xxx; still considering implementing tags support for -c and !-D)

features:
- add a new keyword - "tags" - which is a comma separated list of tags
associated with that file.
- add "-E tags"; exclude files in -D output with matching tags
- add "-I tags"; only include files in -D output with matching tags
(default is to list all)

fixes:
- move all extern variable references from individual files into extern.h
- `quote' some more user-specified strings in error messages
- man page: list an options arguments in the description, and sort xrefs.

- implement -D - dump the specfile out in a manner that's parseable
by various tools
- implement -R key - remove the given key(s) from the list of keys to print
(now it's possible with "-k type -R type" to remove all keys :)
- rename ftype() to nodetype(), and make it public
- use pwcache(3) functions instead of get{gr,pw}{nam,uid}(3)
- add dump_nodes(), which provides the guts for -D.

Add a -l flag which does "loose" permissions checks, i.e. a mode 444
matches a check for mode 644 (though obviously not the reverse). This
can be used by the nightly security run, making the output1 more useful
by having it contain fewer spurious permissions violations.

Note that I did not make -l work if you have a sgid/suid/sticky bit
set. I don't know how you could cause security trouble with more
stringent settings and a suid file, but I don't want to find out the
hard way.

use setprogname

ANSIfy, KNF, in preperation for more work.

branches: 1.14.10;
- add -m and -i to change file flags more usefully.
- fix bugs in fileflags checking

all of the above from dr. lex wennmacher <wennmach@geo.Uni-Koeln.DE>
with some error fixes from egcs via myself.

getcwd() is not equivalent of getwd() if you want to print an error message.

Use getcwd(3) instead of getwd(3).

add support for 4.4BSD file flags. we take a `flags=xxx' parameter,
using the same style of naming as chflags, ls & xinstall use, plus an
additional `none' to turn off a default setting.

Use u_int32_t rather than Sys-V u_long.
Rename err() to mtree_err(), to avoid conflict with libc.

WARNSify, getopt returns -1 not EOF

Add -U flag, checking for mismatch. This is the same as the old -u flag,
except that an error which was corrected is deemed not to be an error.

Necessary for use in the FreeBSD ports system.

Add a missing "break;", pointed out by Chris Demetriou.

Add a `-t' option to update time stamps.

branches: 1.5.6;
add NetBSD id's

clean up import, fix a few bugs, etc.

latest mtree from uunet. doesn't compile, but that'll be fixed soon.
(want to have 'pseudo-virgin' sources, for when fixed fnmatch stuff
is released...)

Add RCS identifiers.

branches: 1.1.1;
Initial revision

Fix typo in comment.

Implement FreeBSD's "nochange" keyworld which causes the existance of a
filesystem object to be checked but none of its attiributes.(brooks)

Implement FreeBSD's -f spec1 -f spec2 spec comparision function. This
is more useful than -C and diff because it highlights the keys that have
changed value so for example, you can see that the size of a file has
not changed but the contents have. (brooks)

account for differences in digest names

branches: 1.27.6; 1.27.12;
Tags are internally stored with leading and trailing commas, for ease
of using strstr(3) to check them against inclusion or exclusion lists.
Don't output the leading or trailing commas.

branches: 1.26.20;
The hashing routines (MD5File() and co.) can do dynamic allocation, so
take advantage of that instead of an ugly hard-coded MAXHASHLEN limit
that needs updating.

I think this was suggested by cube@ years ago.

PR/34818: rad at twig dot com: Recognize solaris door files.

Use MAXHASHLEN here too.

Add SHA2 support in mtree.

branches: 1.22.2;
Implement -M to "merge" entries, even with different types.
This can be used in /etc/security to allow special.local to override
types of entries in special (e.g, replacing a dir with a link).

Rename Wflag to mtree_Wflag, to be more "external namespace" friendly.

Move UCB-licensed code from 4-clause to 3-clause licence.

Patches provided by Joel Baker in PR 22366, verified by myself.

Add -C, which dumps the specification with the full path name first,
and then all the other fields. (I.e, like -D except with the name
first instead of last).
Consistenly strsvis(3) encode path names (even for -C and -D).

Add '-N dbdir', to specify that user & group lookups (via pwcache(3)) are to
be done from dbdir/master.passwd and dbdir/group, using simple text file
parsing routines.

User interface changes from (or inspired by) FreeBSD:
- Add -L to walk the tree `logically', by following symbolic links in
the heirarchy.
- Add -P to walk the tree `physically'. This is the current behaviour,
and the default.
- Add "-X excludes-file" to give mtree the ability to exclude files and
directories from its traversal. excludes-file contains fnmatch(3)
patterns to exclude from the walk.
- Add "md5digest" synonym for "md5".
- Add "rmd160" keyword for RMD-160 message digest, and "rmd160digest" synonym.
- Add "sha1" keyword for SHA-1 message digest, and "sha1digest" synonym.
- Don't try to compare() other attributes if the type doesn't match;
it's nothing but trouble, and no use anyway.
- In -c, only emit "/set" records if something has changed since the
previous one.

User interface changes by me:
- Check a device's parameters before checking uid/gid/mode.
- If updating (-u), modify the following to match the specification:
- Device type (retaining existing ownership).
- Symlink target.

Fixes from (or inspired by) FreeBSD:
- Use p->ftslevel instead of own code to keep track of the level ourself.
The previous code got majorly confused if fts(3) couldn't descend
into a subdir, resulting in leaf nodes getting attached to the wrong
directory.
XXX: This new method is much much more robust, even though it's not 100%
perfect; it might result in a couple of following entries in the spec
to be incorrectly tagged as missing.
- Pass a useful pathname to rlink(), so that logical (-L) traversal
doesn't confuse symlink checking.
- Consistently use MAXPATHLEN+1 sized buffers for pathnames, so that
there's room for the NUL.
- Use mtree_err() and strerror(p->fts_errno) to report errors during
the fts(3) walk.

Fixes by me:
- Remove now-unused `const char *name' argument from compare().
- Change crc_total from an int to a u_int32_t, to match usr.bin/cksum/crc.c.
- Remove trailing whitespace.
- Remove unnecessary (void) casts on functions.
- Reorder entries in the getopt() switch.
- Replace strtoq() with strtoll(), and use strtoul() appropriately.
- Renumber F_ flags to be in alphabetical order.

- add lineno field to NODE
- move inotype() prototype into mtree.h from extern.h

- enforce the top-most entry must be "." and must be a directory; avoids
various potential problems when intermixing full and relative paths
- changes to make it much easier to use spec() - the specfile parser - in
other programs (via .PATHing spec.c and misc.c):
- move excludetags, includetags and keys from mtree.c to misc.c
- implement mtree_err() using vwarnx() instead of assuming name is
"mtree"
- move inotype() and nodetype() from compare.c to misc.c
- add nodetoinode(), to convert from an mtree F_* type to a
mode_t S_IF* type
- clean up #include use; don't assume "mtree.h" pulls in some
standard includes
- change spec() to take a FILE * arg (where the specfile is read from)

Add "device" keyword, which allows the device number to be specified,
to be later checked and possibly created. Uses parsing and encoding
routines from mknod(8).

cleanup tags support;
- create a typedef struct slist_t to hold stuff in (avoid stringlist(3)
because it's less portable)
- move the tag manipulate stuff into misc.c
- add matchtags(), which returns 0 if the node is explicitly excluded,
or not included if an include list is given; or 1 otherwise
- in -D, change tags support from files-only to all-non-directories

(xxx; still considering implementing tags support for -c and !-D)

features:
- add a new keyword - "tags" - which is a comma separated list of tags
associated with that file.
- add "-E tags"; exclude files in -D output with matching tags
- add "-I tags"; only include files in -D output with matching tags
(default is to list all)

fixes:
- move all extern variable references from individual files into extern.h
- `quote' some more user-specified strings in error messages
- man page: list an options arguments in the description, and sort xrefs.

add F_TYPE to KEYDEFAULT.
whilst -c always prints the type (even if removed with -R),
-D wasn't printing it in the default config

- add -m and -i to change file flags more usefully.
- fix bugs in fileflags checking

all of the above from dr. lex wennmacher <wennmach@geo.Uni-Koeln.DE>
with some error fixes from egcs via myself.

Add an `md5' keyword to mtree, which takes as an argument the MD5 cryptographic
checksum of the file in question, optionally preceeded by `0x'.

This has been tested, and does work, AFAICT. There is currently no md5 equivalent
of the '-s' flag for use with checksum.

add support for 4.4BSD file flags. we take a `flags=xxx' parameter,
using the same style of naming as chflags, ls & xinstall use, plus an
additional `none' to turn off a default setting.

Use u_int32_t rather than Sys-V u_long.
Rename err() to mtree_err(), to avoid conflict with libc.

add an 'optional' keyword, which allows files to be in the
specification, but _not_ in the file hierarchy. contrib'd by:
Ed Symanzik <zik@enoch.cl.msu.edu>. Man page adjustment to match, and
minor man page cleanup, by me.

add NetBSD id's

clean up import, fix a few bugs, etc.

#definery fun...

latest mtree from uunet. doesn't compile, but that'll be fixed soon.
(want to have 'pseudo-virgin' sources, for when fixed fnmatch stuff
is released...)

Add RCS identifiers.

branches: 1.1.1;
Initial revision

Don't modify strings stored in hash, otherwise filling up of
directory hierarchy stops if the same hash value occurs in
directory and leaf.

Fix the issue that
"./usr/share/locale/pl/LC_MESSAGES/gettext-tools.mo" isn't included
in sets file in the case of MKUNPRIVED=no.

branches: 1.2.4; 1.2.10; 1.2.12; 1.2.14; 1.2.22; 1.2.26;
fill up the directory hierarchy so that leaf files are not ignored if
specified by themselves.

add the ability to only output paths included in a spec file when creating
from a directory (-c)

PR/58886: Jose Louis Duran: Fix freed memory access leading to SEGV,
given the following spec file (mt -f <specfile>):
------
/set type=dir
.
dup
..
dup
child_entry type=file
------
This happens because when calling:
addchild(last, centry);
if it is the same entry, centry will be freed and then
last = centry;
will point to freed memory. Make addchild return the entry that is supposed
to be used instead.

usr.sbin/mtree: Fix typos.

Submitted by: Jose Luis Duran
PR bin/59824

PR/58875: Jose Luis Duran: Produce consistent checksums in verification
by scanning directories in the same order as usual. While here, fix some
incorrect types.

Provide a more detail message about mismatched entries.

branches: 1.90.16;
use uintmax_t for nlink_t from FreeBSD

Pass a file pointer to write data instead of writing to stdout.
No functional change.

branches: 1.88.2;
Our <sys/param.h> ends up calling header files that define intmax_t. This
should not be the case (but >sys/param.h> is not a standard header so all bets
are off). FreeBSD's does not, so explicitly include <stdint.h> to get it.

Default a bare "." with no type to F_DIR. For FreeBSD compatibility (Brooks
Davis)

use intmax_t instead of long long. fixes wrong cast for time_t.

Implement the "flavor" output discussed in tech-userlevel@, by Brooks Davis

disable encoding of globbing characters for now.

Encode literal global characters in files. Otherwise the results of
updating a directory using the resulting spec will be surprising.

See http://svnweb.freebsd.org/base/head/usr.sbin/mtree/test/test00.sh
for test cases.
(brooks)

clarify seconds.09nanoseconds format (brooks)

account for differences in digest names

branches: 1.80.2;
Add __printflike attribution to use vprintf and friends with an argument
as format string.

branches: 1.79.4;
Suppress a superfluous whitespace after "time=..." in the output.

branches: 1.78.2;
Fix errors in previous: "optional" and "ignore" keywords also need the
appendfield treatment.

Do not print a trailing space on each line with "mtree -C".

Fix the "mtree -M" problem reported in PR 42031 by Geoff Wing.
The cause of the problem was that part of the code assumed that
nodecmp() on two nodes with the same name would return 0, but in
fact nodecmp() would return -1 or +1 if one of the nodes was a
directory and the other was not. The fix is to separate the notion
of whether or not a duplicate name was found from the notion of
where the new node should appear in the list.

When an mtree spec file omits a parent directory, "missing directory in
specification", instead of "no such file or directory".

The recent addition of sorting to "mtree -C" broke some third party
scripts. Now make sorting optional, controled by the "-S" flag.

fix another missing "prev" link

Fix broken "prev" pointer in one case.

Watch one's "p"s and "q"s and fix cast aspersions on alpha ("warning: field
precision should have type 'int', but argument 2 has type 'long int'")

Tags are internally stored with leading and trailing commas, for ease
of using strstr(3) to check them against inclusion or exclusion lists.
Don't output the leading or trailing commas.

Make "mtree -C" sort its output.

As the input is read from a specfile into a tree of linked lists,
keep each linked list sorted. The sort order is the same as that
already used by "mtree -c": directories sort after non-directories, but
otherwise names are sorted in the order used by strcmp().

branches: 1.68.2;
fix -Wsign-compare issues

fix time_t and dev_t formats

ensure mtree_err gets a format string

Remove clause 3 and 4 from TNF licenses

branches: 1.64.10;
Complete the move of string_to_flags() and flags_to_string() from the
bin/ls sources to libutil:
o Bump libutil minor version number
o Fix uses to include <util.h> to pick up the function definitions
o Fix most uses of flags_to_string() to release the now-malloc()ed result

Apply patch from Peter Bex to fix PR bin/31922.

If parameter don't have values, then 'continue' after setting the flag bit.
Fixes coverty CID 776 and removes the need for a 'val = NULL' added to
appease gcc's undefined variable tests.
This parsing code sucks though, it doesn't correctly check for the '='
between a parameter and its value, whitespace can be used instead.

fix setmode error handling.

Add SHA2 support in mtree.

branches: 1.59.2;
In replacenode(), don't strunvis() a NULL pointer. (Fixes bug in previous.)

strsvis(3) encode & decode the "link=" value (symlink target).

Implement -M to "merge" entries, even with different types.
This can be used in /etc/security to allow special.local to override
types of entries in special (e.g, replacing a dir with a link).

Rename Wflag to mtree_Wflag, to be more "external namespace" friendly.

Completely rework how tools/compat is done. Purge all uses/references to
_NETBSD_SOURCE as this makes cross building from older/newer versions of
NetBSD harder, not easier (and also makes the resulting tools 'different')

Wrap all required code with the inclusion of nbtool_config.h, attempt to
only use POSIX code in all places (or when reasonable test w. configure and
provide definitions: ala u_int, etc).

Reviewed by lukem. Tested on FreeBSD 4.9, Redhat Linux ES3, NetBSD 1.6.2 x86
NetBSD current (x86 and amd64) and Solaris 9.

Fixes PR's: PR#17762 PR#25944

foliage from pack_dev change :-)

branches: 1.54.2;
Handle new-style error return from mknod/pack_dev.c

#include <util.h>
this is needed by fparseln on macos/darwin and is harmless on netbsd

Include filename in:
'existing entry for `sha1.h', type `file' does not match type `link'

realloc pedant

Move UCB-licensed code from 4-clause to 3-clause licence.

Patches provided by Joel Baker in PR 22366, verified by myself.

Add -C, which dumps the specification with the full path name first,
and then all the other fields. (I.e, like -D except with the name
first instead of last).
Consistenly strsvis(3) encode path names (even for -C and -D).

tweaks for fparseln(3) move from libutil to libc:
- remove #include <util.h> if nothing else needed it
- remove LDFLAGS+=-lutil if nothing else needed it

support multiple entries for "." and full paths if the types match; the last
listed entry's settings override the existing settings for that node.

Support `/unset all', to unset all of the global values set with `/set ...'

Roll in fixes to permit cross-compiling from non-NetBSD hosts. This
round has been tested on Solaris/x86 and Linux hosts.

* Add host tools cap_mkdb, ctags, m4, uudecode.
* Protect __RCSID() and __COPYRIGHT() better.
* Reduce the number of places that need to include "config.h", to keep
sources closer to their "vanilla" versions.
* Add more compat #defines and autoconf-checked functions.

Make almost all tools compile and run properly on non-NetBSD hosts. (In
particular, most tools now run correctly on Solaris 7.)

add <util.h> back in for native builds.

Add hooks to make mtree compilable from src/tools.

Generally don't need the extra byte in MAXPATHLEN sized buffers, because
pathnames are guaranteed by POSIX to be MAXPATHLEN-1 bytes long.
Kindly pointed out by Warner Losh in private email.

User interface changes from (or inspired by) FreeBSD:
- Add -L to walk the tree `logically', by following symbolic links in
the heirarchy.
- Add -P to walk the tree `physically'. This is the current behaviour,
and the default.
- Add "-X excludes-file" to give mtree the ability to exclude files and
directories from its traversal. excludes-file contains fnmatch(3)
patterns to exclude from the walk.
- Add "md5digest" synonym for "md5".
- Add "rmd160" keyword for RMD-160 message digest, and "rmd160digest" synonym.
- Add "sha1" keyword for SHA-1 message digest, and "sha1digest" synonym.
- Don't try to compare() other attributes if the type doesn't match;
it's nothing but trouble, and no use anyway.
- In -c, only emit "/set" records if something has changed since the
previous one.

User interface changes by me:
- Check a device's parameters before checking uid/gid/mode.
- If updating (-u), modify the following to match the specification:
- Device type (retaining existing ownership).
- Symlink target.

Fixes from (or inspired by) FreeBSD:
- Use p->ftslevel instead of own code to keep track of the level ourself.
The previous code got majorly confused if fts(3) couldn't descend
into a subdir, resulting in leaf nodes getting attached to the wrong
directory.
XXX: This new method is much much more robust, even though it's not 100%
perfect; it might result in a couple of following entries in the spec
to be incorrectly tagged as missing.
- Pass a useful pathname to rlink(), so that logical (-L) traversal
doesn't confuse symlink checking.
- Consistently use MAXPATHLEN+1 sized buffers for pathnames, so that
there's room for the NUL.
- Use mtree_err() and strerror(p->fts_errno) to report errors during
the fts(3) walk.

Fixes by me:
- Remove now-unused `const char *name' argument from compare().
- Change crc_total from an int to a u_int32_t, to match usr.bin/cksum/crc.c.
- Remove trailing whitespace.
- Remove unnecessary (void) casts on functions.
- Reorder entries in the getopt() switch.
- Replace strtoq() with strtoll(), and use strtoul() appropriately.
- Renumber F_ flags to be in alphabetical order.

- move "Wflag" global from mtree.c to spec.c, and reinstate the functionality
of tv's previous code, which skips uname or gname parsing if -W is enabled.
- rename "lineno" to "mtree_lineno", to reduce possibility of name
clashes in code that yanks in spec.c & misc.c (unlikely, but you never know)

Revert previous until it's fixed a more generic way ... I didn't realize
this was pulled in via reachover from elsewhere.

Don't try to do uid or gid lookups from a supplied spec file if -W is given;
the numerical IDs won't be used anyway.

fix potential sparc compiler warning (if DEBUG was defined, that is...)

- add lineno field to NODE
- move inotype() prototype into mtree.h from extern.h

- enforce the top-most entry must be "." and must be a directory; avoids
various potential problems when intermixing full and relative paths
- changes to make it much easier to use spec() - the specfile parser - in
other programs (via .PATHing spec.c and misc.c):
- move excludetags, includetags and keys from mtree.c to misc.c
- implement mtree_err() using vwarnx() instead of assuming name is
"mtree"
- move inotype() and nodetype() from compare.c to misc.c
- add nodetoinode(), to convert from an mtree F_* type to a
mode_t S_IF* type
- clean up #include use; don't assume "mtree.h" pulls in some
standard includes
- change spec() to take a FILE * arg (where the specfile is read from)

convert from strtok() to strsep()

Add "device" keyword, which allows the device number to be specified,
to be later checked and possibly created. Uses parsing and encoding
routines from mknod(8).

* Implement support for absolute paths in the specfile. An absolute path
is one that contains a `/' character that is not the first character.
All parent directories referenced in the path must exist.
* Add copyright for all the work I've done.
* Document the history of various features added since 4.3-Reno.

- support "all" as a valid keyword for -k, -K and -R.
- parsekey(): support "all" as a keyword which returns a value with all bits
set that pertain to a valid keyword.
- dump_nodes():
- only print out a keyword if it's requested AND it's set in the node
- UNAME falls back to UID if the user name can't be determined
- GNAME falls back to GID if the group name can't be determined
- rework man page:
- use Sy instead of Cm
- use Pq, Dq, Ql, etc instead of ``...''

dump_nodes() fixes:
- in F_GID, print the gid not the uid
- support F_GNAME

cleanup tags support;
- create a typedef struct slist_t to hold stuff in (avoid stringlist(3)
because it's less portable)
- move the tag manipulate stuff into misc.c
- add matchtags(), which returns 0 if the node is explicitly excluded,
or not included if an include list is given; or 1 otherwise
- in -D, change tags support from files-only to all-non-directories

(xxx; still considering implementing tags support for -c and !-D)

- skip F_OPT entries (recently added diagnostic detected this)
- put F_TAG parsing in the correct alphabetical position

features:
- add a new keyword - "tags" - which is a comma separated list of tags
associated with that file.
- add "-E tags"; exclude files in -D output with matching tags
- add "-I tags"; only include files in -D output with matching tags
(default is to list all)

fixes:
- move all extern variable references from individual files into extern.h
- `quote' some more user-specified strings in error messages
- man page: list an options arguments in the description, and sort xrefs.

- implement -D - dump the specfile out in a manner that's parseable
by various tools
- implement -R key - remove the given key(s) from the list of keys to print
(now it's possible with "-k type -R type" to remove all keys :)
- rename ftype() to nodetype(), and make it public
- use pwcache(3) functions instead of get{gr,pw}{nam,uid}(3)
- add dump_nodes(), which provides the guts for -D.

replace home-grown parser with fparseln()

- add parsetype() (a la parsekey()) to parse the "type=xxx" arg. this improves
the error detection for invalid types, as well as shrinking binary size
- implement inotype() using ftype(), rather than duplicating the switch
- change "char *" arguments to "const char *"

improve readability of error messages by quoting the string that we
did not understand. Avoids lossage like "unknown group named".

ANSIfy, KNF, in preperation for more work.

Remove all the "#ifdef __APPLE__" code, to make this actually
build again on Darwin. The code in question is outdated.

OK'd by Wilfredo Sanchez wsanchez@MIT.EDU.

- The type of return value of setmode is a void * and getmode takes it,
rather than mode_t *.
- Free the storage allocated by setmode unless it is obvious that program
exits immediately.

Use strunvis instead of homebrewed "decode" routine

Encode file names using strvis(3) when creating a specification.
Escape white space characters and `#' characters. Do the inverse
when reading specs.

This is a temporary fix until we have strzouvis(3) (or some such)
which is an improved version of strvis (taking an additional list
of characters to encode).

Fixes PR 5006 and PR 7663

branches: 1.16.2; 1.16.6;
Appease gcc-2.8.1, make printf formats more portable.

Add an `md5' keyword to mtree, which takes as an argument the MD5 cryptographic
checksum of the file in question, optionally preceeded by `0x'.

This has been tested, and does work, AFAICT. There is currently no md5 equivalent
of the '-s' flag for use with checksum.

add support for 4.4BSD file flags. we take a `flags=xxx' parameter,
using the same style of naming as chflags, ls & xinstall use, plus an
additional `none' to turn off a default setting.

Use u_int32_t rather than Sys-V u_long.
Rename err() to mtree_err(), to avoid conflict with libc.

The return value of setmode(3) is a pointer to malloc()'ed area
and must be freed to avoid memory leaks if called repeatedly.

WARNSify, getopt returns -1 not EOF

merge lite-2

fix printf() formats, make explicit comparisons to NULL (gcc -Wall)

use appropriate strto*() functions, and cast results when appropriate;
fixes PR bin/1582.

Rename struct timespec fields to conform to POSIX.1b

add NetBSD id's

clean up import, fix a few bugs, etc.

off_t exp.

latest mtree from uunet. doesn't compile, but that'll be fixed soon.
(want to have 'pseudo-virgin' sources, for when fixed fnmatch stuff
is released...)

Add RCS identifiers.

branches: 1.1.1;
Initial revision

PR/57735: Jose Louis Duran: mtree -f file1 -f file2 does not consider type
changes.
(FreeBSD https://github.com/freebsd/freebsd-src/commit/\
4f4b548b1d21ae16955b09f9b717524167a7d97e)

use uintmax_t for nlink_t from FreeBSD

st_nlink on Linux is not int, so add an explicit cast.

branches: 1.2.2; 1.2.4;
NetBSD needs <time.h>

Implement FreeBSD's -f spec1 -f spec2 spec comparision function. This
is more useful than -C and diff because it highlights the keys that have
changed value so for example, you can see that the size of a file has
not changed but the contents have. (brooks)

PR/58885: Jose Luis Duran: Fix output formatting for extra entries for
FreeBSD-9 compatibility.

PR/58875: Jose Luis Duran: Produce consistent checksums in verification
by scanning directories in the same order as usual. While here, fix some
incorrect types.

branches: 1.48.2;
PR/57735: Jose Louis Duran: check if fts_read(3) had an error.
(FreeBSD commit https://github.com/freebsd/freebsd-src/commit/\
2dfa4b66b3d0caaaae6ce2df476b5615f8415a19)

mtree: use POSIX type uint32_t instead of u_int32_t

fix tools build.