More conservative umask

Set the file creation mask consistently because the files these scripts
create can become part of an iso image.

Deprecate extsrc/, EXTSRCSRCDIR, MKEXTSRC

Remove support for building extsrc/:
1. Makefile: remove do-extsrc target.
2. build.sh: remove options -y and -Y extsrcdir.
3. distrib/sets: remove support for extsrc in various tools
including the options -L ext and -y, and the extsrc sets.
4. doc/BUILDING.mdoc: remove docs for extsrc/, EXTSRCSRCDIR, MKEXTSRC (etc)
5. bsd.own.mk and various Makefiles: remove support for extsrc/,
EXTSRCSRCDIR, MKEXTSRC.

As proposed on tech-kern and tech-userlevel on 2022-01-07
and followed up on 2022-08-21.

there is no -X flag

branches: 1.76.6; 1.76.12;
Fix syntax error on OSX 10.5.
While here, simplify handling with OPTARG using IFS.

developped with and 'go for it' dsl@

Support "extsrc", externally added programs and libraries. Users can write
their own reach-overs, cross-build, install, and get set files just like base
and X11 / X.org. (These sets are not included as TNF releases.)

branches: 1.74.20;
Use "$*", not "$@", where we want a space-separated list of words.

Ensure that (almost) all shell variables in distrib/sets/* scripts are
used with curly braces and quotes, as in "${var}".

Also ensure that command substitution is quoted, as in "$(command)", and
convert `command` to $(command).

Reviewed by agc

Add shell variables defined in sets.subr for almost all commands used
by scripts in distrib/sets/*. This is intended to be useful when cross
building. Only trivial commands like echo, cat, and rm are excluded
from this treatment.

While I am here, make ${MTREE} and ${DB} follow the pattern used by most
other such variables.

Reviewed by agc.

Make all the sets scripts work even when not run from the directory that
they reside in.
Change any uses of dirname to use shell variable substitution instead.

implement -l, to list the selected set names rather than the contents

optimise invocation of list_set_files()

rename all the <bsd.own.mk> derived variables to uppercase, and export
them to the environment.

Significant overhaul of how the sets are used:

* Add an optional third field to the sets file which is a list of
comma separated keywords that control if the line is printed.
Currently supported keywords
kerberos4 ${MKKERBEROS4} != no
kerberos ${MKKERBEROS} != no
lint ${MKLINT} != no
obsolete ${obsolete} != 0.
In this case, non obsolete files are not printed.
(This will allow future support for builds with variables such as
MKHESIOD and MKYP set to "no".)

* Use sh(1)'s getopts where appropriate, and otherwise cleanup the
various scripts.

* Move defaults for sets.subr from sets.defaults into sets.subr.
Move replicated code for determining stuff such as shlibs type
from various scripts into sets.subr.

* Merge the obsolete.*, krb.*, krb4.* and lint.* into the appropriate
main lists with the relevant third field keyword(s).

Redo creation of sets/sums so this can be parallelized if using -j.
Suggested in PR#20115 and PR#20116

Factor duplicated code for setting variables such as machine_cpu
and machine_arch to their defaults into sets.defaults, and source
it at the top of each script.

Also, to be consistent with variable naming, s/arch/machine_arch/.

sets.defaults introduces two new variables, krb and krb4, which
will affect whether Kerberos- or Kerberos IV-only files are put
into the set lists.

Backout last change, it's not necessary.

Fix sets handling so that GCC specific lists are really used.

add initial support for gcc3

Don't duplicate code for listing sets used by makeplist, regpkg.
Instead, use the routine in sets.subr.

Check if md.${MACHINE}.${MACHINE_ARCH} exists and prefer it over
md.${MACHINE}.

Do not try to build LKMs on evbppc, because we cannot predetermine
the PPC_INTR_IMPL, which is used to include a file in <machine/intr.h>.
To allow evbppc to complete a ``make release'', we also need to
factor out the LKM files to a new set list file, and tweak makeflist
to not include this file for evbppc.

Discussed with Matt Thomas.

Enable MIPS now that crunchgen works again for this arch.

Remove conditional for x86_64 to not include lintlibs, they're ok now.

use ${make} instead of ${MAKE} in the shell scripts, to prevent weird
lossage with the modified ${MAKE} getting passed to child make's

explicitly add " -f `dirname $0`/Makefile " to ${MAKE}, so that
these scripts do not need to be run from this directory in order
to function correctly

Some systems/shells have difficulty if the closing ] of a test does not have
preceding whitespace. Ensure that there is whitespace.

* Clean up the handling of rescue.* files makeflist.
* Move all md.* and ad.* /rescue/... files into their
own list files.

* Move the rescue bits into their own file.
* In makeflist, skip rescue bits in arch's that don't yet support it
(MIPS, currently, due to an objcopy-crunch bug).

switch from checking USE_NEW_TOOLCHAIN == yes to TOOLCHAIN_MISSING != yes,
and don't bother with tc.old any more

If ad.${machine_arch} exists, perfer it over ad.${machine_cpu},
because it will have more specific information.

set MAKE="${MAKE:-make} -j 1" to prevent problems where the output
of make -j N>1 would result in junk in various shell variables

Added support for ad.${machine_cpu}.shl listfiles. These list
architecture-dependent files that are present when shared libraries are in use.

branches: 1.47.2;
cat the tc.shl setlist, instead of trying to run it.

Not all new-toolchain platforms support shared libraries (m68000 and
sh3, for example, due to defincencies in the current compiler version).

Deal with this by splitting toolchain shlibs into tc.shl, and teachking
makeflist about it.

Also, fix an old-toolchain remnant in the sh3 set lists.

According to thorpej, only x86_64 doesn't provide lint files now.
Remove corresponding conditionals for sparc64, m68000, and sh3, and
add one for x86_64.

Solve problem with with XFree86 3.3.6/4.x set "maketars" differently:
We don't want e.g. a seperate "xbase.tgz" and "xbase3.tgz" but one
combined archive.

Added support for stl.mi and a stl.${objfmt} files for each set,
which respectively contain MI and objfmt-specific things associated
with static libraries. Useful for systems that don't have shared
libraries. Currently only used by lists/comp/stl.elf.

" egrep ... | awk" can be replaced by a smarter bit of awk...

First step of supporting XFree86 4.x binary sets:
- Recognize version dependent X11 lists files.
- Move all files which only belong to XFree86 3.3.6 in lists for version 3.

Oversight in last.

Use the logic in bsd.own.mk to figure out the toolchain format,
and to convert MACHINE_ARCH to a CPU. Make this cross-build
friendly by specifying MACHINE, MACHINE_ARCH, and MAKE in the
makeflist/checkflist environment.

The test for USE_NEW_TOOLCHAIN is based on the variable being *set*, not
a comparison of its value. Instead, use ${USE_NEW_TOOLCHAIN:Dyes} and
change the if block back to [ ... = "yes" ].

Change to check for nowarn instead of yes as USE_NEW_TOOLCHAIN changed formats
in bsd.own.mk.
Also, change the make line to use

Change sets so the differing toolchain bits only (not the complete toolchain
at the moment) are pulled out into tc.mi and tc.old files.

tc.mi has any files which are in the new toolchain only and tc.old contains
the reverse (files which only existed in the old toolchain). The remainder
of the toolchain that is common between both old and new is still contained
in the various mi/md files.

Check for USE_NEW_TOOLCHAIN=yes in makeflist to determine which tc files to
pull in if they exist.

XXX - All the toolchain bits should eventually be pulled out into separate
lists as tc.mi and tc.<arch>. Leaving it for now until the new toolchain is
fully integrated on all ports.

Atari now has ELF shared libs.

switch cesfic to ELF.

add support for ports that do not (can not) build lint libraries.

next68k is now ELF.

Switch news68k to ELF:
- Set shlib=elf on news68k

sun3 now uses ELF.

Kill package field much more efficiently when creating flists from set lists.

sh3e[bl] -> sh3.
sh3 has no shlib yet.

Add hp300 to the increasingly ugly `Compute toolchain' test.

Set shlib=elf for mvme68k.
(The mvme68k set lists still have a few problems even with this
change, but until all m68k ports -> ELF, there's no Other Way).

arm26 is ELF, even if its shared libraries are currently broken.

Obliterate "man/md_share". The difference is negligible relative to the
size of the man set, and with all architectures' man pages in "man", we
can now share both "man" and "misc" on all platforms (CD-ROM space saver).

Add the xmisc set.

sparc64 is elf, too

branches: 1.21.2;
Merge a bunch of things from crypto-us and crypto-intl into basesrc,
adding support for Heimdal/KTH Kerberos where easy to do so. Eliminate
bsd.crypto.mk.

There is still a bunch more work to do, but crypto is now more-or-less
fully merged into the base NetBSD distribution.

Compatible upgrade to set list behavior -- any (whitespace separated) fields
after the first in each line of a set list are ignored. This introduces
no change in current set list behavior, but will need to be revisited if
we wish to distribute files with spaces in their names -- in this case we can
move to tab separated fields or some other behavior.

Current set lists work exactly as before, but in the future set lists may
have additional fields for use in a system packaging script.

I will post a detailed work plan for the system package system which uses
this to tech-pkg and tech-install in the next day or so.

add crypto-intl support in the "cryptint" set.

branches: 1.18.4;
Bring setlists up to date for i386 switch to ELF and ipv6.

Make the sparc use the elf sets.

Ignore lines starting with a ``#'' in the set list files.

branches: 1.15.2;
Convert mipse[lb] to mips _after_ processing command line arguments.

Use "$MAKE" instead of "make" for cross environment.

Remap mipse[bl] to mips for set lists.

Remove the machine check for missing shared libs support on the vax.

powerpc ports use ELF shlib.

Add options for including the "domestic" ("secr") sets.

Do datbase normalization on sets/lists/*/* containing shared-library
related files. Partition into three categories:
* shl.mi -- MI sharedlib files present on all ports with shared libs
* shl.aout -- MI sharedlib files present on all ELF ports
* shl.elf -- MI sharedlib files present on all a.out ports

Add rudimentary shl* support to makeflist.
Remove files left empty after shlib normalization.
Add ad.mips where appropriate.
Make libc, libedit numbers consistent (libc.so.12.29, libedit.2.1).

* replace '-arch arch' with '-m machine' (as per make(1)'s $MACHINE). e.g, i386
* add '-a arch' - $MACHINE_ARCH. e.g, m68k, powerpc
* replace long -options with short -options (with backward compat)
* ``knf''ify scripts

Fix syntax error with multiple "set" args

add -both arg for when one puts xsrc and src builds in same DESTDIR

When deciding which md set lists to use, prefer $MACHINE over
uname -m

Add xcontrib to xlists.

Add a new option '-x11' for the xsets.

branches: 1.2.4;
More portable versions of makeflist and maketars that can be used to
cross-make a distribution. Also avoid duplicating makeflist logic.

branches: 1.1.1;
Initial revision