Deprecate extsrc/, EXTSRCSRCDIR, MKEXTSRC

Remove support for building extsrc/:
1. Makefile: remove do-extsrc target.
2. build.sh: remove options -y and -Y extsrcdir.
3. distrib/sets: remove support for extsrc in various tools
including the options -L ext and -y, and the extsrc sets.
4. doc/BUILDING.mdoc: remove docs for extsrc/, EXTSRCSRCDIR, MKEXTSRC (etc)
5. bsd.own.mk and various Makefiles: remove support for extsrc/,
EXTSRCSRCDIR, MKEXTSRC.

As proposed on tech-kern and tech-userlevel on 2022-01-07
and followed up on 2022-08-21.

no more /usr/X11R6

also look in ./usr/X11R7/ for x.

make metalog files trully optional (don't print errors when they don't exist)

branches: 1.42.6; 1.42.12;
Put intermediate lists on the top of ${DESTDIR} (${DESTDIR}/SETS.*) and leave
them. Teach list generators to ignore them. Always generate metalog
missing/extra lists too, but don't check it (for now). I'll change "flist"
to be generated only when set lists have changed.

No functional changes.

Thinko; missing/extra are generated differently for metalog/destdir comparison
methods. Fix mistakenly merged generation functions.

Style. Better variable names.

Share more code. Call functions in one place.

Refactor; move generate_* function calls in single place.

Use more functions. But still no functional changes.

Use shell function because I like it. No functional changes.

Support "extsrc", externally added programs and libraries. Users can write
their own reach-overs, cross-build, install, and get set files just like base
and X11 / X.org. (These sets are not included as TNF releases.)

Simplify the way EXCLUDE_REGEXP is built.

branches: 1.33.18; 1.33.28;
fixes for building into case preserving, but case insensitive $DESTDIR
- have checkflist do a second possibly case insensitive check for
files which are missing from DESTDIR
- have postinstall require exact case matches for obsolete files

No space in "function()", "command;" or "$(command)".
Requested by christos.

Disable checking of metalog in checkflist. In future,
this checking should be reinstated, but with proper
handling of obsolete files.

Reviewed by christos

* Add a check for consistency between DESTDIR and METALOG, in addition
to the existing check for consistency between DESTDIR and the output
from makeflist.
* Use egrep instead of awk to ignore differences that are expected.
This should be easier to maintain, and should also make it easier
for users to add their own custom exceptions by editing the
regexp.

Reviewed by agc

Ensure that (almost) all shell variables in distrib/sets/* scripts are
used with curly braces and quotes, as in "${var}".

Also ensure that command substitution is quoted, as in "$(command)", and
convert `command` to $(command).

Reviewed by agc

Add shell variables defined in sets.subr for almost all commands used
by scripts in distrib/sets/*. This is intended to be useful when cross
building. Only trivial commands like echo, cat, and rm are excluded
from this treatment.

While I am here, make ${MTREE} and ${DB} follow the pattern used by most
other such variables.

Reviewed by agc.

Move default values for several shell variables into sets.subr.

Reviewed by agc

Set default vars so blank and unset both end up w. a reasonable default

show count of missing and extra files. idea from rtr@

Significant overhaul of how the sets are used:

* Add an optional third field to the sets file which is a list of
comma separated keywords that control if the line is printed.
Currently supported keywords
kerberos4 ${MKKERBEROS4} != no
kerberos ${MKKERBEROS} != no
lint ${MKLINT} != no
obsolete ${obsolete} != 0.
In this case, non obsolete files are not printed.
(This will allow future support for builds with variables such as
MKHESIOD and MKYP set to "no".)

* Use sh(1)'s getopts where appropriate, and otherwise cleanup the
various scripts.

* Move defaults for sets.subr from sets.defaults into sets.subr.
Move replicated code for determining stuff such as shlibs type
from various scripts into sets.subr.

* Merge the obsolete.*, krb.*, krb4.* and lint.* into the appropriate
main lists with the relevant third field keyword(s).

add a -print to the find statement to avoid problems on solaris where
each file/directory/link is printed twice. Fixes PR23270

* add -e; extra files are not fatal
* add -m; missing files are not fatal
* use ${MKTEMP} -d instead of home-grown tempdir creation code

make "extra files" a fatal error again, based on various feedback.

i'll consider adding an optional switch and associated build infrastructure
to enable the "non fatal extra files" functionality instead.

Rework to use comm(1) instead of diff(1),
and separate "missing files in DESTDIR" (which is fatal)
from "extra files in DESTDIR" (which is now non-fatal).

change test to skip metalog to be a case,
to easily allow other entries to skip to be added.

X11 directories and files in "/etc".

-path is an extension to find(1) that not all systems have, so don't
use it. Instead, filter out the metalog file using a separate loop
in the pipeline.

improve the description of the key to the output

checkflist:
- add "-M metalog". if metalog starts with "${DESTDIR}/", it
will be skipped in the flist check, to prevent unnecessary
warnings about extraneous ./METALOG
- if the diff returned a non-zero exit code, print a warning message,
and for the default diff output, print a blurb describing the
meaning of the output.

Makefile:
- call checkflist with "-M ${METALOG}" if UNPRIVED
- run "${MAKE} checkflist" if ${DESTDIR} != ""

NOTE: this change will prevent maketars from running if the
sets lists are out of date. This is intentional.
Better to fail than to build tar files with missing files.

branches: 1.14.2;
fix usage output

branches: 1.13.2;
Merge a bunch of things from crypto-us and crypto-intl into basesrc,
adding support for Heimdal/KTH Kerberos where easy to do so. Eliminate
bsd.crypto.mk.

There is still a bunch more work to do, but crypto is now more-or-less
fully merged into the base NetBSD distribution.

Allow -u and -c to set diff style (from Todd Whitesel <toddpw@best.com>)

add crypto-intl support in the "cryptint" set.

branches: 1.10.2; 1.10.6;
Add options for including the "domestic" ("secr") sets.

Make the -x11 case actually work. Make the tempfile symbolic.

Revert yesterday's change.

Compensate for the /usr/include/${MACHINE_ARCH} change.

* replace '-arch arch' with '-m machine' (as per make(1)'s $MACHINE). e.g, i386
* add '-a arch' - $MACHINE_ARCH. e.g, m68k, powerpc
* replace long -options with short -options (with backward compat)
* ``knf''ify scripts

add -both arg for when one puts xsrc and src builds in same DESTDIR

Add a new option '-x11' for the xsets.

branches: 1.3.2;
translate to sh, add RCSID and a comment

don't assume that . is in path. pr 964 from Greg Hudson <ghudson@mit.edu>

branches: 1.1.1;
Initial revision