rbtree: Rename rb.c to rbtree.c to match it's header rbtree.h

While here rename RBTEST to RBLOCAL to give a better meaning as we
want to include the local rbtest.h rather than the system one.

Also allow to compile outside of NetBSD source tree by guarding some
things we would otherwise expect to be defined.

Compensate for src/common/lib/libc/atomic/atomic_init_cas.c losing
the extern declaration of __libc_atomic_init() and instead using a
new one added to src/lib/libc/include/extern.h

That file is outside src/common and src/sys so is unavailable to
kernel builds, so instead make a new include file in src/common
which the kernel can read which contains the needed extern decl
for __libc_atomic_init()

This seems to fix the evbarm builds (maybe others) - but it is
all MUCH TOO UGLY to keep. Someone please make a better fix,
even if that just means reverting rev 1.5 of
src/common/lib/libc/atomic/atomic_init_cas.c
and all of the changes here (the addition to libc/include/extern.h
should be harmless to keep).

avoid dependency on NETBSDSRCDIR being set.

(similar to an earlier commit)
due to various things, non-build.sh builds don't have it set already
and it ends up defaulting to /usr/src, so if that doesn't exist or
is the wrong version, building libkern fails.

revert previous: http://mail-index.netbsd.org/source-changes/2021/10/25/msg133295.html

going to add __always_inline to the functions called from _mcount()
discussed on http://mail-index.netbsd.org/source-changes-d/2021/10/25/msg013480.html

In some arch, _mcount() would be called recursively when built with COPTS=-O0.

Normally, functions called from mcount.c are expected to be expanded inline,
so _mcount() will never be called recursively. But when build with COPTS=-O0,
`static inline' functions aren't inlined, and _mcount() will be called
recursively.

Even if _mcount() has `__attribute__((__no_ instrument_function__))',
it has no effect on the calling external (no-inlined) function.

To avoid this, PROF.<fn> is added can be set the profiling flag of any file.
"PROF.mcount.c" is set to blank by default, mcount.c itself is compiled
without -pg.

move bi-endian disklabel support from the kernel and libsa into libkern.

- dkcksum() and dkcksum_sized() move from subr_disk.c and from
libsa into libkern/dkcksum.c (which is missing _sized() version),
using the version from usr.sbin/disklabel.

- swap_disklabel() moves from subr_disk_mbr.c into libkern, now called
disklabel_swap(). (the sh3 version should be updated to use this.)

- DISKLABEL_EI becomes a first-class option with opt_disklabel.h.

- add libkern.h to libsa/disklabel.c.

this enables future work for bi-endian libsa/ufs.c (relevant for ffsv1,
ffsv2, lfsv1, and lfsv2), as well as making it possible for ports not
using subr_disk_mbr.c to include bi-endian disklabel support (which,
afaict, includes any disk on mbr-supporting platforms that do not have
an mbr as well as disklabel.)

builds successsfully on: alpha, i386, amd64, sun2, sun3, evbarm64,
evbarm64-eb, sparc, and sparc64. tested in anita on i386 and sparc,
testing in hardware on evbarm64*.

branches: 1.50.4; 1.50.6;
Build strlist.c.

(Sigh, missed in original commit.)

branches: 1.49.2;
Make copystr() a MI C function, part of libkern and shared on all
architectures.

Notes:

- On alpha and ia64 the function is kept but gets renamed locally to avoid
symbol collision. This is because on these two arches, I am not sure
whether the ASM callers do not rely on fixed registers, so I prefer to
keep the ASM body for now.
- On Vax, only the symbol is removed, because the body is used from other
functions.
- On RISC-V, this change fixes a bug: copystr() was just a wrapper around
strlcpy(), but strlcpy() makes the operation less safe (strlen on the
source beyond its size).
- The kASan, kCSan and kMSan wrappers are removed, because now that
copystr() is in C, the compiler transformations are applied to it,
without the need for manual wrappers.

Could test on amd64 only, but should be fine.

Rewrite entropy subsystem.

Primary goals:

1. Use cryptography primitives designed and vetted by cryptographers.
2. Be honest about entropy estimation.
3. Propagate full entropy as soon as possible.
4. Simplify the APIs.
5. Reduce overhead of rnd_add_data and cprng_strong.
6. Reduce side channels of HWRNG data and human input sources.
7. Improve visibility of operation with sysctl and event counters.

Caveat: rngtest is no longer used generically for RND_TYPE_RNG
rndsources. Hardware RNG devices should have hardware-specific
health tests. For example, checking for two repeated 256-bit outputs
works to detect AMD's 2019 RDRAND bug. Not all hardware RNGs are
necessarily designed to produce exactly uniform output.

ENTROPY POOL

- A Keccak sponge, with test vectors, replaces the old LFSR/SHA-1
kludge as the cryptographic primitive.

- `Entropy depletion' is available for testing purposes with a sysctl
knob kern.entropy.depletion; otherwise it is disabled, and once the
system reaches full entropy it is assumed to stay there as far as
modern cryptography is concerned.

- No `entropy estimation' based on sample values. Such `entropy
estimation' is a contradiction in terms, dishonest to users, and a
potential source of side channels. It is the responsibility of the
driver author to study the entropy of the process that generates
the samples.

- Per-CPU gathering pools avoid contention on a global queue.

- Entropy is occasionally consolidated into global pool -- as soon as
it's ready, if we've never reached full entropy, and with a rate
limit afterward. Operators can force consolidation now by running
sysctl -w kern.entropy.consolidate=1.

- rndsink(9) API has been replaced by an epoch counter which changes
whenever entropy is consolidated into the global pool.
. Usage: Cache entropy_epoch() when you seed. If entropy_epoch()
has changed when you're about to use whatever you seeded, reseed.
. Epoch is never zero, so initialize cache to 0 if you want to reseed
on first use.
. Epoch is -1 iff we have never reached full entropy -- in other
words, the old rnd_initial_entropy is (entropy_epoch() != -1) --
but it is better if you check for changes rather than for -1, so
that if the system estimated its own entropy incorrectly, entropy
consolidation has the opportunity to prevent future compromise.

- Sysctls and event counters provide operator visibility into what's
happening:
. kern.entropy.needed - bits of entropy short of full entropy
. kern.entropy.pending - bits known to be pending in per-CPU pools,
can be consolidated with sysctl -w kern.entropy.consolidate=1
. kern.entropy.epoch - number of times consolidation has happened,
never 0, and -1 iff we have never reached full entropy

CPRNG_STRONG

- A cprng_strong instance is now a collection of per-CPU NIST
Hash_DRBGs. There are only two in the system: user_cprng for
/dev/urandom and sysctl kern.?random, and kern_cprng for kernel
users which may need to operate in interrupt context up to IPL_VM.

(Calling cprng_strong in interrupt context does not strike me as a
particularly good idea, so I added an event counter to see whether
anything actually does.)

- Event counters provide operator visibility into when reseeding
happens.

INTEL RDRAND/RDSEED, VIA C3 RNG (CPU_RNG)

- Unwired for now; will be rewired in a subsequent commit.

Nix trailing whitespace.

Remove never-used Mersenne twister from libkern.

Omit vestigial unused commented-out experiment.

Include radixtree in the kernel.

Rename min.c -> uimin.c, max.c -> uimax.c in libkern.

provide memmem

branches: 1.41.2; 1.41.4;
coalesce the two copies of hexdump into libkern

Import SHA-3 code into libc and libkern.

No new public symbols in libc, but publishing the symbols is a simple
matter if/when we decide to do so.

Proposed on tech-kern and tech-userlevel with no objections:

https://mail-index.NetBSD.org/tech-kern/2017/11/11/msg022581.html
https://mail-index.NetBSD.org/tech-userlevel/2017/11/11/msg010968.html

move scsipi_strvis -> libkern:strnvisx()
change the prototype to match userland
fix sizes of strings passed to it

remove various HAVE_GCC=45 fragments.

strtoi and strtou for the kernel

branches: 1.36.2;
bcdtobin and bintobcd are now inlines in <dev/clock_subr.h>

branches: 1.35.2;
Merge tls-earlyentropy branch into HEAD.

lua: updated from 5.1 to 5.3 work3

* lua(1):
- changed lua_Integer to intmax_t
- updated distrib/sets/lists and etc/mtree
- updated bsd.lua.mk
- fixed bozohttpd (lua-bozo.c)
- compatibilized bindings: gpio, sqlite
* lua(4):
- removed floating-point and libc dependencies using '#ifndef _KERNEL'
- fixed division by zero and exponentiation
- libkern: added isalnum(), iscntrl(), isgraph(), isprint() and ispunct()
- acpica: removed isprint() from acnetbsd.h
- libc: moved strcspn.c, strpbrk.c and strspn.c to common
- removed stub headers
- compatibilized bindings: luapmf, luasystm
* reorganized luaconf.h
* updated doc/CHANGES and doc/RESPONSIBLE

${.PARSEDIR} not .${PARSEDIR} for Makefile.compiler-rt include

branches: 1.32.2;
Do not include compiler-rt in librump. librump is not a toolchain support
library, and toolchain support belongs in the host libc or libgcc or
equivalent entity that is actually supposed to complement the toolchain.

Fixes librump build on (a) Linux ARM (collisions with libgcc)

Include compiler-rt in libc, libm and libkern.

For HAVE_LIBGCC=no, use compiler-rt for quad support.

add strncat (for acpi)

Allow kernel code to access constant databases by moving cdbr(3) and the
required mi_vector_hash(3) into src/common.

changed lua_Number to int64_t

Another codegen bug that fixes (in this case lots of) atf tests.
rump_server doesn't die so much now.

branches: 1.25.2;
libkern: add murmurhash module.

branches: 1.24.2;
Replace consttime_bcmp/explicit_bzero by consttime_memequal/explicit_memset.

consttime_memequal is the same as the old consttime_bcmp.
explicit_memset is to memset as explicit_bzero was to bcmp.

Passes amd64 release and i386/ALL, but I'm sure I missed some spots,
so please let me know.

undo sparc64 kludge

Make sparc64 32-bit kernel buildable.

amend previous sparc64 fix: rump does not know about memcpyset.s

On sparc64 memcpy and memset are provided by memcpyset.s (should have been
memcpyset.S, but...). Don't include them in the build because it breaks the
modular build where the kernel library is built as an object.

Add strnlen.c to SRCS (which will automatically use the .S version if it
exists).

branches: 1.18.2;
Add "consttime_bcmp" and "explicit_bzero" functions for both kernel
abd userland, as proposed on tech-security, with explicit_bzero using
a volatile function pointer as suggested by Alan Barrett.
Both do what the name says. For userland, both are prefixed by "__"
to keep them out of the user namespace.
Change some memset/memcmp uses to the new functions where it makes
sense -- these are just some examples, more to come.

branches: 1.17.2; 1.17.4; 1.17.6; 1.17.8;
Migrate one last leftover bit (used only by the kernel now) to
sys/ufs/ufs and remove the old quota headers and no-longer-used shared
code. Ok by releng.

Don't use __cmsg_alignbytes in the kernel. Mark it as constant function
for userland as its value never changes. This allows the compiler to
optimise most invocations away.

branches: 1.15.2;
First step of random number subsystem rework described in
<20111022023242.BA26F14A158@mail.netbsd.org>. This change includes
the following:

An initial cleanup and minor reorganization of the entropy pool
code in sys/dev/rnd.c and sys/dev/rndpool.c. Several bugs are
fixed. Some effort is made to accumulate entropy more quickly at
boot time.

A generic interface, "rndsink", is added, for stream generators to
request that they be re-keyed with good quality entropy from the pool
as soon as it is available.

The arc4random()/arc4randbytes() implementation in libkern is
adjusted to use the rndsink interface for rekeying, which helps
address the problem of low-quality keys at boot time.

An implementation of the FIPS 140-2 statistical tests for random
number generator quality is provided (libkern/rngtest.c). This
is based on Greg Rose's implementation from Qualcomm.

A new random stream generator, nist_ctr_drbg, is provided. It is
based on an implementation of the NIST SP800-90 CTR_DRBG by
Henric Jungheim. This generator users AES in a modified counter
mode to generate a backtracking-resistant random stream.

An abstraction layer, "cprng", is provided for in-kernel consumers
of randomness. The arc4random/arc4randbytes API is deprecated for
in-kernel use. It is replaced by "cprng_strong". The current
cprng_fast implementation wraps the existing arc4random
implementation. The current cprng_strong implementation wraps the
new CTR_DRBG implementation. Both interfaces are rekeyed from
the entropy pool automatically at intervals justifiable from best
current cryptographic practice.

In some quick tests, cprng_fast() is about the same speed as
the old arc4randbytes(), and cprng_strong() is about 20% faster
than rnd_extract_data(). Performance is expected to improve.

The AES code in src/crypto/rijndael is no longer an optional
kernel component, as it is required by cprng_strong, which is
not an optional kernel component.

The entropy pool output is subjected to the rngtest tests at
startup time; if it fails, the system will reboot. There is
approximately a 3/10000 chance of a false positive from these
tests. Entropy pool _input_ from hardware random numbers is
subjected to the rngtest tests at attach time, as well as the
FIPS continuous-output test, to detect bad or stuck hardware
RNGs; if any are detected, they are detached, but the system
continues to run.

A problem with rndctl(8) is fixed -- datastructures with
pointers in arrays are no longer passed to userspace (this
was not a security problem, but rather a major issue for
compat32). A new kernel will require a new rndctl.

The sysctl kern.arandom() and kern.urandom() nodes are hooked
up to the new generators, but the /dev/*random pseudodevices
are not, yet.

Manual pages for the new kernel interfaces are forthcoming.

branches: 1.14.2;
Build and install ppath(3), property-list paths library.

Add a new libquota library, which contains some blocks to build and/or
parse quota plists; as well as a getfsquota() function to retrieve quotas
for a single id from a single filesystem (whatever filesystem this is:
a local quota-enabled fs or NFS). This is build on functions getufsquota()
(for local filesystems with UFS-like quotas) and getnfsquota();
which are also available to userland programs.
move functions from quota2_subr.c to libquota or libprop as appropriate,
and ajust in-tree quota tools.
move some declarations from kernel headers to either sys/quota.h or
quota/quota.h as appropriate. ufs/ufs/quota.h still installed because
it's needed by other installed ufs headers.
ufs/ufs/quota1.h still installed as a quick&dirty way to get a code
using the old quotactl() to compile (just include ufs/ufs/quota1.h instead of
ufs/ufs/quota.h - old code won't compile without this change and this is
on purpose).
Discussed on tech-kern@ and tech-net@ (long thread, but not much about
libquota itself ...)

Add quad srcs if building x86_64 32-bit libkern.

Add quad srcs if using MIPS O32 ABI

branches: 1.10.2; 1.10.4;
fix typo

branches: 1.9.2; 1.9.4;
Get rid of last "easy" kernel symbols starting with __:
__assert -> kern_assert
__sigtimedwait1 -> sigtimedwait1
__wdstart -> wdstart1

The rest are MD and/or shared with userspace, so they will require
a little more involvement than what is available for this quick
"ride the 5.99.24 bump" action.

Build fix for the mips64 merge:
- If memset2.c is in ${SRCS}, exclude conflicting memset.c.
- If MD byte_swap_8.* is in ${SRCS}, exclude conflicting bswap64.c.

Redo the previous inverted logic. Sort alphabetically.

Don't build quad support code on 64-bit platforms.

Move that majority of the 'SRCS+= foo.c' into the main Makefile.libkern.
Any .S files added by the arch/*/Makefile.inc cause the .c file to
be excluded.
Specific exclusions added using NO_SRCS to match previous files.
At least sparc, sparc64, i386, amd64 and vax GENERIC still build.
(There is a fubar with the naming of the byte-swap files ...)

Use stuff from libc/Makefile to auto-remove .c files if a .S has been added.
Use it to dispose of tne .c files that were already only conditionally
added.

Move popcount et al to src/common and add popcount32/popcount64.
Requested by rmind@. MD should now override popcount32/popcount64 and
provide the aliases as fitting.

branches: 1.2.2;
Fixes PR kern/41069 and PR kern/41070.

Extends the Opencrypto API to allow the destination buffer size to be
specified when its not the same size as the input buffer (i.e. for
operations like compress and decompress).
The crypto_op and crypt_n_op structures gain a u_int dst_len field.
The session_op structure gains a comp_alg field to specify a compression
algorithm.
Moved four ioctls to new ids; CIOCGSESSION, CIOCNGSESSION, CIOCCRYPT,
and CIOCNCRYPTM.
Added four backward compatible ioctls; OCIOCGSESSION, OCIOCNGSESSION,
OCIOCCRYPT, and OCIOCNCRYPTM.

Backward compatibility is maintained in ocryptodev.h and ocryptodev.c which
implement the original ioctls and set dst_len and comp_alg to 0.

Adds user-space access to compression features.

Adds software gzip support (CRYPTO_GZIP_COMP).

Adds the fast version of crc32 from zlib to libkern. This should be generally
useful and provide a place to start normalizing the various crc32 routines
in the kernel. The crc32 routine is used in this patch to support GZIP.

With input and support from tls@NetBSD.org.

branches: 1.1.2; 1.1.4; 1.1.6;
Split variables out of Makefile into Makefile.libkern so that we
can easily just .include it for the proper SRCS etc. definitions
in case we're interested in the files but not in building an actual
libkern.{a,o}. for librump