Use the standard build rules.

branches: 1.100.16;
remove a bunch of "@echo done" from the tail of rules. these messages
were vaguely useful back when we didn't run make -j, but now you end
up with a single line "done" every so often, with no idea what it is
for. very few other targets claim they're done so just remove these.

Order library object build.

Sprinkle more done messages.

branches: 1.97.6;
Revert previous revision. It bkeaks profiling kernel builds.
LIBISPRIVATE unconditionnally set MKPROFILE to no, preventing
profiling kernel library.

Simplify by using LIBISPRIVATE?= yes

branches: 1.95.24; 1.95.28;
Change the use of formally undocumented features, which have now been
made to fail. Specifically, change
.ifdef(SYMBOL) -> .ifdef SYMBOL or .if defined(SYMBOL),
and corresponding for .ifndef.

Also correct one error in lib/libm/Makefile (.ifdef (${MKCOMPLEX} != "no")?!?).

Split variables out of Makefile into Makefile.libkern so that we
can easily just .include it for the proper SRCS etc. definitions
in case we're interested in the files but not in building an actual
libkern.{a,o}. for librump

replace bitmask_snprintf(9) with snprintb(3)

Add new ptree (Patricia / RADIX tree) implementation to NetBSD.

Our qsort() is inappropriate for kernel use because it makes recursive
calls. Replace it with a kheapsort() function in kernel. Pointed out
by tron@.

Make qsort() available in libkern.

branches: 1.89.2; 1.89.4; 1.89.6; 1.89.10;
Move rb.c from sys/lib/libkern to common so it can be in both libc and libkern.

Include cpuset.

branches: 1.87.6; 1.87.8; 1.87.10; 1.87.12; 1.87.14;
Add Mersenne Twister prototypes and state struct to libkern.h
Cleanup a comment. s/RLEN/MTPRNG_RLEN/g s/POS1/MTPRNG_POS1/g
Remove unneeded test code.

remove insque/remque from libkern, they are no longer used

build rb.c.

branches: 1.84.22; 1.84.24; 1.84.28; 1.84.30;
Add strsep(3) to libkern.

To avoid code duplication, move strsep.c to the kernel/userland
common files.

Soon I will commit source-address selection (options IPSELSRC).
It will use strsep(3).

Merge kernel and userland rmd160 and sha2 implementation.
XXX: We still install rmd160.h and sha2.h in /usr/include/crypto, unlike
the other hash functions which get installed in /usr/include for compatibility.

- Move strtoll.c and strtoull.c from libc/stdlib to common/libc/stdlib.
- Add strtoll.c and strtoull.c to libkern.

branches: 1.81.2; 1.81.4;
Add src/common/include to include header search path so that
standalone programs can be built without installed ${DESTDIR} tree.
Okayed by christos, and should close PR 33431.

Support for building proplib in the kernel.

Move strtoumax.c from libc/stdlib to common/libc/stdlib and include it
in libkern. Required for new code coming soon.

branches: 1.78.2;
Provide BCD<->binary conversion in libkern and turn <dev/clock_subr.h>'s
FROMBCD()/TOBCD() macros into wrappers around it, resulting in both
smaller code footprint and elimination of possible issues due to
multiple evaluation of macro arguments.

Suggested by Simon Burge and Anders Gavare on tech-kern.

branches: 1.77.4; 1.77.6; 1.77.8; 1.77.10;
Use common sources with userland.

merge ktrace-lwp.

put mcount.c in the normal object list, so that "make depend" works.
use a gcc attribute to prevent it from being instrumented.

Redo previous; don't add mcount.po to POBJS when MKPROFILE=no.

Create mcount.po only when MKPROFILE != no.

branches: 1.72.4;
nuke trailing whitespace

branches: 1.71.8; 1.71.10;
Add xlat_mbr_fstype - converts mbr type to disklabel type

branches: 1.70.2;
add strl{cpy,cat} to libkern. code from lib/libc/string (originally from openbsd).

If LIBKERN_ARCH is defined, use that as the architecture-specific
subdirectory. Needed for the x86_64 32bit compile case.

set LLIBS= so we don't try to check the lint lib against llib-lc.ln

Unconditionally add strstr.c. Bill Studenmund.

Add strstr() to libkern. For now, it's only used in i386 (for processor
identification).

Updated version of cscope/mkid support. Check libkern and compat lib
for source files. Also include header files in mkid run.

have arc4random(9).

branches: 1.63.8; 1.63.10;
MKfoo=no -> NOfoo

Undefine DESTDIR; we don't want the special DESTDIR <bsd.lib.mk>
handling for kernel libraries.

XXX this build method should be replaced by something more sane

Move _insque()/_remque() to libkern. Once remaining uses would
be converted to <sys/queue.h> macros, _insque()/_remque() would be eliminated
altogether.

branches: 1.60.2; 1.60.4;
Don't set ARCHSUBDIR to empty string.

Use MACHINE_CPU to find the architecture subdir, or MACHINE_ARCH if that
exists. Same algorithm as libc uses.

Use arm arch directory for arm32 as well as arm26.

prepare for sh3e[bl] split

Create a shared set of ARM sources for libkern and move arm26 over to using
them.

These are identical to the current arm32 sources with the following exceptions:
- References to C labels are wrapped in _C_LABEL().
- Function returns have an alternate version inside #ifdef __APCS_26__.

add md4c.c

make CMSG_ALIGN always synchronize with kernel's idea of ALIGNBYTES.
ancillary data alignment will be ALIGNBYTES, not sizeof(long) - 1, from now.

CMSG_xx will NOT resolve into constant. if you use CMSG_xx to allocate
arrays, you'll lose.

bump shlib minor for libc.

NOTE: if you are on top of arch with ALIGNBYTES != sizeof(long) - 1,
you need to recompile IPv6-related binaries. there is no way to guarantee
backward compat in this aspect. sorry for this. this should be the last
backward compat breakage for IPv6-related ancillary data manipulation.
(we still have PR 9516 for unix-domain sockets...)

branches: 1.53.2;
move intoa() from libsa:net.c to libkern, turn inet_ntoa() into a macro,
nuke ip_convertaddr()

Build libraries for kernel/standalone code from within their compilation
directories and use .PATH to lookup the source files.
(Formerly, the libs were built from the source dirs, with MAKEOBJDIR
set to the compilation directory.)
This solves 2 problems:
-"mkdep" and "make" are now consistent about the file lookup, this fixes
bad interactions with amd reported in PR bin/7374 (Arne Juul) and
lossage reported by Andrew Gillham ("obj" dirs and relative paths still
don't work well together)
-kernel compile trees can be moved around without forcing a new
"make depend" - fixing PR kern/4021 by Martin Husemann

Add inet_addr.c (for nfs_bootparam.c)

branches: 1.50.4;
make the mipse[lb]->mips translation really work
(the S/// doesn't work as I'm now telling the 3rd time),
put the translation onto a separate line to make it more understandable

Makefile ${MACHINE_ARCH} changes for mips{eb,el}, as suggested by
Izumi Tsutsui <tsutsui@ceres.dti.ne.jp>

no need to add ${OBJS} to POBJS here, bsd.lib.mk does it automatically.

convert from NOxxx= to MKxxx=no.
include <bsd.own.mk> if testing a MKxxx variable.

branches: 1.46.2;
replace several special targets with stub .c files, as in libc/string

fix silly comment typo

make libkern build memmove() properly

branches: 1.43.2;
Make the combination "profiled kernel" and "${KERN_AS} == "library""
work too. (mcount.po was forgotten)

Add a small string pattern matching function to facilitate pcmcia cis string
matches.

add sha1.c, and build it.

Build memcpy() out of bcopy() by default.

update to match (now-fixed) libc rules for strchr and strrchr.
Brokenness pointed out by Matthias Drochner.

add rules like those in libc to build strchr() and strrchr() (the latter
is being added to libkern by this commit). remove strchr.c from
archs' Makefile.inc files to go along with this. (If an arch has
a .S version, it should be copied down and added to the apropriate
Makefile.inc.)

Use LORDER, TSORT

branches: 1.36.8;
Back out BUILDDIR and NOINSTALL changes.

Use new NOINSTALL option for libraries that are used only during the build
of a program, and not installed into the final system.

Use libinstall::, not install:, to prevent instaling LIB.

Build md5c.c

allow includers to specify library-specific CPP flags via ${LIB}MISCCPPFLAGS

branches: 1.31.6;
Add anchors so that only exactly `-I. is modified.

Convert the -I. to -I${.OBJDIR}, so that the `machine symlink is available.

We dont need to pass down COMPATDIR and KERNDIR.

Run the child make in the source directory, so that nothing here needs to be
changed when the compile directory is outside the source tree.

Make sure we clean up libkern.o and libkern.po, too. Closes PRs 1951
and 2387.

don't get rid of the -D_KERNEL in KERNCPPFLAGS

Simplify, and add support for passing CPPFLAGS.

set, don't add to, CFLAGS from KERNCFLAGS. Do assignment of CC and
CFLAGS earlier, so that the arch-specific Makefile can add to them.

Build libkern as one .o file, so that all symbols defined therein are
always available to loadable modules.

Remove old .PATH directives that referenced libc source.
Libkern has its own copies of all the requisite source files.

Add the ability to override KERNREL and KERNDST.

Build libkern in the kernel compilation directory, like libcompat.

slight cleanup/reorg. don't use quad code on the alpha.

Change a dependency to work when not using `all' target.

silence link creation

new RCS ID format.

compile -O3

clean obj better

to find the DEFS.h files here, not in libc..

mv ${arch}/ to arch/${arch}, so that libkern builds without obj/

be careful about mcount.po

need a backslash!

attempts to reduce our reliance on /usr/include

add libkern mcount

kernel/disklib.c goes away, not very portable

Make FFS optional

quad support

branches: 1.4.2;
change to use MACHINE_ARCH and update from libc

use obj.${MACHINE} dirs to compile into

slight cleanup, and ... (gasp) add comments

move 'boot' and 'libkern' into 'lib'. generally clean up 'kernel library'
building and dependencies.

New port: mac68ksf - Macintosh 68k softfloat.

This commit is all the Makefile glue for the new port which is the mac68k
port with MKSOFTFLOAT set to "yes".

As posted here:
https://mail-index.netbsd.org/port-mac68k/2025/10/14/msg001037.html

avoid dependency on NETBSDSRCDIR being set.

due to various things, non-build.sh builds don't have it set already
and it ends up defaulting to /usr/src, so if that doesn't exist or
is the wrong version, building libkern fails.

XXX: pullup-10, ..?

PR port-arm/55897

Fix ABI mismatch for armhf runtime routines for floating-point arithmetics;
For hard-float arm variants, provide

(1) generic runtime routines with correct calling convention, and
(2) EABI runtime routines at the same time.

I've confirmed that no binary changes for kernels.

LGTM by skrll

branches: 1.12.6;
For m68k, do not compile in {,u}divmodsi4 also, that are not used by
codes generated by gcc (confirmed not only for kernel, but also all
executables and libraries for userland).

Fix build failure for MODULAR kernel, noticed by mlelstv.

For m68k, do not compile in {,u}divsi3 and {,u}modsi3 since codes
generated by gcc do not depend on them.

We should not have unused functions in libkern for MODULAR.

branches: 1.10.6;
convert HAVE_GCC == 7 to HAVE_GCC >= 7.

branches: 1.9.4;
clzdi2 and ctzdi2 are needed on alpha. (counting instructions are in the
optional CIX extension, and thus not used by the default compiler configuation).

don't use -fnon-call-exceptions on arm. not needed and triggers eh_frame generation.

apply the previous to all GCC-7 platforms. it affects vax as well.

for sh3 and GCC 7 pass -fnon-call-exceptions for udivmoddi4.c.
avoids it emitting a call to abort() for an explicit divide by zero.

branches: 1.5.4; 1.5.30;
Changes to existing files to enable building AARCH64 userland.
evbarm64-el
This is clang only. While gcc4.8 supports aarch64, no netbsd support has
been written for aarch64 with gcc4.8.

branches: 1.4.4; 1.4.6;
Update compiler-rt glue for the new source layout.

For ARM EABI, includ divide-by-0 trap handlers.

This is not libc, so don't check the libc arch macro.

libkern build glue for compiler-rt.

Revert previous. I guess I'll have to solve the m68k build issue
a different way.

Make libkern depend on assym.h.

mark cleankernlib and cleandirkernlib as .MAKE.

fixes a couple of:
nbmake[1]: warning: Invalid internal option "-J" in <..>
problems that appear with "build.sh -U .. kernel=" but not with -u.

solution from rillig@ in private email.

explicitly pass COPTS

libkern: fix wrong escaping in makefile

In the unlikely event that KERNDIR contains a '$' character, that '$'
should not be doubled to '$$'. It only needs the usual shell quoting.

Introduce LIBKERN_MD_FLAGS to pass some MD flags for libkern.

This is necessary since libkern is built by coprocess, and
threfore subsets of make(1) variables are passed.

- Determine KERN_AS automatically depending on whether OPT_MODULAR is
set or not, in the same way as libcompat.

- Specify OPT_MODULAR in the port Makefile instead of KERN_AS.

Now, KERN_AS=library is used for kernels without module(7) for all ports.

OK christos

branches: 1.45.2;
- Introduce :q modifier for make variables and make it double escape $'s so
that passing variables to recursive makes with :q works as expected.
- Revert :Q to work as before.
- Adjust makefiles that use recursive make to use :q

Discussed on tech-toolchain@
XXX: pullup 8

branches: 1.44.10; 1.44.16;
remove "@echo done" from 4 rules that don't need it and the random "done"
in my build logs don't help understanding.

In kernel lib build, print message when things not only start bug also end.

branches: 1.42.14;
Don't pass down CPUFLAGS since its value is already in CFLAGS.

branches: 1.41.2;
Deal with optional HAVE_GCC.

branches: 1.40.94; 1.40.110; 1.40.118;
Use common sources with userland.

merge ktrace-lwp.

branches: 1.38.4;
nuke trailing whitespace

branches: 1.37.8; 1.37.10;
Make 'dependall' generate dependencies in these libraries.

branches: 1.36.6;
Add lint infrastructure

Don't make clean and cleandir depend on the lib subdir. Just check for it's
existance before running the submake. This makes it possible to run a
make build (which runs cleandir before make obj) in a r/o source tree

branches: 1.34.2; 1.34.4;
Have recursive make invocations depend on the .MAKE pseudo-target so
make knows to handle them specially.

Revert STRIPPROG -> STRIP

STRIP -> STRIPPROG

branches: 1.31.2;
Build libraries for kernel/standalone code from within their compilation
directories and use .PATH to lookup the source files.
(Formerly, the libs were built from the source dirs, with MAKEOBJDIR
set to the compilation directory.)
This solves 2 problems:
-"mkdep" and "make" are now consistent about the file lookup, this fixes
bad interactions with amd reported in PR bin/7374 (Arne Juul) and
lossage reported by Andrew Gillham ("obj" dirs and relative paths still
don't work well together)
-kernel compile trees can be moved around without forcing a new
"make depend" - fixing PR kern/4021 by Martin Husemann

branches: 1.30.8;
change handling of libkern:
-sys/lib/libkern builds as library per default (as it was documented all
the time)
-ports able to LKM set "KERN_AS=obj" explicitely in their Makefiles

distclean is a synonym for cleandir

make sure that LORDER, TSORT, AR, NM, RANLIB, and SIZE are correctly
and consistently set when doing kernel library builds.

Pass down the values of: AR NM RANLIB
(Fixes PR#4426 from SAITOH Masanobu)
Also pass down LORDER TSORT.

branches: 1.26.8;
Allow depend target to be parallized, from Andrew Wheadon
<andrew@wipux2.wifo.uni-mannheim.de>, PR #3257.

allow includers to specify library-specific CPP flags via ${LIB}MISCCPPFLAGS

branches: 1.24.4;
clean up substantially. Transform clean, depend, cleandir targets from ::
to :, so they can be used e.g. with <bsd.prog.mk>. Standardize variable
names used to configure kernel libraries.

Variables used by these Makefile.inc's are:

S must be set to the top of the 'sys' tree.
${LIB}DST may be set to the location of the directory where library
objects are to be built. Defaults to ${.OBJDIR}/lib/${lib}.
${LIB}_AS may be set to 'obj' to build a object from the library's
object files. (Otherwise, a library will be built.)
Defaults to 'library'.
${LIB}MISCMAKEFLAGS
Miscellaneous flags to be passed to the library's Makefile when
building. See library's Makefile for more details about
supported flags and their default values.

(where LIB is KERN, SA, or Z, and lib is kern, sa, or z, depending on which
library's Makefile.inc is being discussed.)

PR/2981: use ${VAR:Q} instead of '${VAR}' to avoid quoting problems

Allow compilation of libkern as library for standalone programs

Handle relative paths to the source directory if they begin with a '.'.

Pass $STRIP to the sub-make.

Add anchors so that only exactly `-I. is modified.

Convert the -I. to -I${.OBJDIR}, so that the `machine symlink is available.

We dont need to pass down COMPATDIR and KERNDIR.

Run the child make in the source directory, so that nothing here needs to be
changed when the compile directory is outside the source tree.

use ' not " when passing variables to a sub-make. pr#1199/pr#1296

Make `make depend' work for libraries.

Simplify, and add support for passing CPPFLAGS.

Build libkern as one .o file, so that all symbols defined therein are
always available to loadable modules.

Add the ability to override KERNREL and KERNDST.

Build libkern in the kernel compilation directory, like libcompat.

Only build the version of the library that's needed.

new RCS ID format.

Use $(MAKE) instead of make

Changed to use make's new .OBJDIR

use ${MAKE} rather than "make", as suggested by Roland McGrath

branches: 1.4.2;
use obj.${MACHINE} dirs to compile into

punt on doing minimal work in favor of less output in 'happy' case.

add a bunch of .NOTMAINs

slight cleanup, and ... (gasp) add comments

rbtree: Rename rb.c to rbtree.c to match it's header rbtree.h

While here rename RBTEST to RBLOCAL to give a better meaning as we
want to include the local rbtest.h rather than the system one.

Also allow to compile outside of NetBSD source tree by guarding some
things we would otherwise expect to be defined.

Compensate for src/common/lib/libc/atomic/atomic_init_cas.c losing
the extern declaration of __libc_atomic_init() and instead using a
new one added to src/lib/libc/include/extern.h

That file is outside src/common and src/sys so is unavailable to
kernel builds, so instead make a new include file in src/common
which the kernel can read which contains the needed extern decl
for __libc_atomic_init()

This seems to fix the evbarm builds (maybe others) - but it is
all MUCH TOO UGLY to keep. Someone please make a better fix,
even if that just means reverting rev 1.5 of
src/common/lib/libc/atomic/atomic_init_cas.c
and all of the changes here (the addition to libc/include/extern.h
should be harmless to keep).

avoid dependency on NETBSDSRCDIR being set.

(similar to an earlier commit)
due to various things, non-build.sh builds don't have it set already
and it ends up defaulting to /usr/src, so if that doesn't exist or
is the wrong version, building libkern fails.

revert previous: http://mail-index.netbsd.org/source-changes/2021/10/25/msg133295.html

going to add __always_inline to the functions called from _mcount()
discussed on http://mail-index.netbsd.org/source-changes-d/2021/10/25/msg013480.html

In some arch, _mcount() would be called recursively when built with COPTS=-O0.

Normally, functions called from mcount.c are expected to be expanded inline,
so _mcount() will never be called recursively. But when build with COPTS=-O0,
`static inline' functions aren't inlined, and _mcount() will be called
recursively.

Even if _mcount() has `__attribute__((__no_ instrument_function__))',
it has no effect on the calling external (no-inlined) function.

To avoid this, PROF.<fn> is added can be set the profiling flag of any file.
"PROF.mcount.c" is set to blank by default, mcount.c itself is compiled
without -pg.

move bi-endian disklabel support from the kernel and libsa into libkern.

- dkcksum() and dkcksum_sized() move from subr_disk.c and from
libsa into libkern/dkcksum.c (which is missing _sized() version),
using the version from usr.sbin/disklabel.

- swap_disklabel() moves from subr_disk_mbr.c into libkern, now called
disklabel_swap(). (the sh3 version should be updated to use this.)

- DISKLABEL_EI becomes a first-class option with opt_disklabel.h.

- add libkern.h to libsa/disklabel.c.

this enables future work for bi-endian libsa/ufs.c (relevant for ffsv1,
ffsv2, lfsv1, and lfsv2), as well as making it possible for ports not
using subr_disk_mbr.c to include bi-endian disklabel support (which,
afaict, includes any disk on mbr-supporting platforms that do not have
an mbr as well as disklabel.)

builds successsfully on: alpha, i386, amd64, sun2, sun3, evbarm64,
evbarm64-eb, sparc, and sparc64. tested in anita on i386 and sparc,
testing in hardware on evbarm64*.

branches: 1.50.4; 1.50.6;
Build strlist.c.

(Sigh, missed in original commit.)

branches: 1.49.2;
Make copystr() a MI C function, part of libkern and shared on all
architectures.

Notes:

- On alpha and ia64 the function is kept but gets renamed locally to avoid
symbol collision. This is because on these two arches, I am not sure
whether the ASM callers do not rely on fixed registers, so I prefer to
keep the ASM body for now.
- On Vax, only the symbol is removed, because the body is used from other
functions.
- On RISC-V, this change fixes a bug: copystr() was just a wrapper around
strlcpy(), but strlcpy() makes the operation less safe (strlen on the
source beyond its size).
- The kASan, kCSan and kMSan wrappers are removed, because now that
copystr() is in C, the compiler transformations are applied to it,
without the need for manual wrappers.

Could test on amd64 only, but should be fine.

Rewrite entropy subsystem.

Primary goals:

1. Use cryptography primitives designed and vetted by cryptographers.
2. Be honest about entropy estimation.
3. Propagate full entropy as soon as possible.
4. Simplify the APIs.
5. Reduce overhead of rnd_add_data and cprng_strong.
6. Reduce side channels of HWRNG data and human input sources.
7. Improve visibility of operation with sysctl and event counters.

Caveat: rngtest is no longer used generically for RND_TYPE_RNG
rndsources. Hardware RNG devices should have hardware-specific
health tests. For example, checking for two repeated 256-bit outputs
works to detect AMD's 2019 RDRAND bug. Not all hardware RNGs are
necessarily designed to produce exactly uniform output.

ENTROPY POOL

- A Keccak sponge, with test vectors, replaces the old LFSR/SHA-1
kludge as the cryptographic primitive.

- `Entropy depletion' is available for testing purposes with a sysctl
knob kern.entropy.depletion; otherwise it is disabled, and once the
system reaches full entropy it is assumed to stay there as far as
modern cryptography is concerned.

- No `entropy estimation' based on sample values. Such `entropy
estimation' is a contradiction in terms, dishonest to users, and a
potential source of side channels. It is the responsibility of the
driver author to study the entropy of the process that generates
the samples.

- Per-CPU gathering pools avoid contention on a global queue.

- Entropy is occasionally consolidated into global pool -- as soon as
it's ready, if we've never reached full entropy, and with a rate
limit afterward. Operators can force consolidation now by running
sysctl -w kern.entropy.consolidate=1.

- rndsink(9) API has been replaced by an epoch counter which changes
whenever entropy is consolidated into the global pool.
. Usage: Cache entropy_epoch() when you seed. If entropy_epoch()
has changed when you're about to use whatever you seeded, reseed.
. Epoch is never zero, so initialize cache to 0 if you want to reseed
on first use.
. Epoch is -1 iff we have never reached full entropy -- in other
words, the old rnd_initial_entropy is (entropy_epoch() != -1) --
but it is better if you check for changes rather than for -1, so
that if the system estimated its own entropy incorrectly, entropy
consolidation has the opportunity to prevent future compromise.

- Sysctls and event counters provide operator visibility into what's
happening:
. kern.entropy.needed - bits of entropy short of full entropy
. kern.entropy.pending - bits known to be pending in per-CPU pools,
can be consolidated with sysctl -w kern.entropy.consolidate=1
. kern.entropy.epoch - number of times consolidation has happened,
never 0, and -1 iff we have never reached full entropy

CPRNG_STRONG

- A cprng_strong instance is now a collection of per-CPU NIST
Hash_DRBGs. There are only two in the system: user_cprng for
/dev/urandom and sysctl kern.?random, and kern_cprng for kernel
users which may need to operate in interrupt context up to IPL_VM.

(Calling cprng_strong in interrupt context does not strike me as a
particularly good idea, so I added an event counter to see whether
anything actually does.)

- Event counters provide operator visibility into when reseeding
happens.

INTEL RDRAND/RDSEED, VIA C3 RNG (CPU_RNG)

- Unwired for now; will be rewired in a subsequent commit.

Nix trailing whitespace.

Remove never-used Mersenne twister from libkern.

Omit vestigial unused commented-out experiment.

Include radixtree in the kernel.

Rename min.c -> uimin.c, max.c -> uimax.c in libkern.

provide memmem

branches: 1.41.2; 1.41.4;
coalesce the two copies of hexdump into libkern

Import SHA-3 code into libc and libkern.

No new public symbols in libc, but publishing the symbols is a simple
matter if/when we decide to do so.

Proposed on tech-kern and tech-userlevel with no objections:

https://mail-index.NetBSD.org/tech-kern/2017/11/11/msg022581.html
https://mail-index.NetBSD.org/tech-userlevel/2017/11/11/msg010968.html

move scsipi_strvis -> libkern:strnvisx()
change the prototype to match userland
fix sizes of strings passed to it

remove various HAVE_GCC=45 fragments.

strtoi and strtou for the kernel

branches: 1.36.2;
bcdtobin and bintobcd are now inlines in <dev/clock_subr.h>

branches: 1.35.2;
Merge tls-earlyentropy branch into HEAD.

lua: updated from 5.1 to 5.3 work3

* lua(1):
- changed lua_Integer to intmax_t
- updated distrib/sets/lists and etc/mtree
- updated bsd.lua.mk
- fixed bozohttpd (lua-bozo.c)
- compatibilized bindings: gpio, sqlite
* lua(4):
- removed floating-point and libc dependencies using '#ifndef _KERNEL'
- fixed division by zero and exponentiation
- libkern: added isalnum(), iscntrl(), isgraph(), isprint() and ispunct()
- acpica: removed isprint() from acnetbsd.h
- libc: moved strcspn.c, strpbrk.c and strspn.c to common
- removed stub headers
- compatibilized bindings: luapmf, luasystm
* reorganized luaconf.h
* updated doc/CHANGES and doc/RESPONSIBLE

${.PARSEDIR} not .${PARSEDIR} for Makefile.compiler-rt include

branches: 1.32.2;
Do not include compiler-rt in librump. librump is not a toolchain support
library, and toolchain support belongs in the host libc or libgcc or
equivalent entity that is actually supposed to complement the toolchain.

Fixes librump build on (a) Linux ARM (collisions with libgcc)

Include compiler-rt in libc, libm and libkern.

For HAVE_LIBGCC=no, use compiler-rt for quad support.

add strncat (for acpi)

Allow kernel code to access constant databases by moving cdbr(3) and the
required mi_vector_hash(3) into src/common.

changed lua_Number to int64_t

Another codegen bug that fixes (in this case lots of) atf tests.
rump_server doesn't die so much now.

branches: 1.25.2;
libkern: add murmurhash module.

branches: 1.24.2;
Replace consttime_bcmp/explicit_bzero by consttime_memequal/explicit_memset.

consttime_memequal is the same as the old consttime_bcmp.
explicit_memset is to memset as explicit_bzero was to bcmp.

Passes amd64 release and i386/ALL, but I'm sure I missed some spots,
so please let me know.

undo sparc64 kludge

Make sparc64 32-bit kernel buildable.

amend previous sparc64 fix: rump does not know about memcpyset.s

On sparc64 memcpy and memset are provided by memcpyset.s (should have been
memcpyset.S, but...). Don't include them in the build because it breaks the
modular build where the kernel library is built as an object.

Add strnlen.c to SRCS (which will automatically use the .S version if it
exists).

branches: 1.18.2;
Add "consttime_bcmp" and "explicit_bzero" functions for both kernel
abd userland, as proposed on tech-security, with explicit_bzero using
a volatile function pointer as suggested by Alan Barrett.
Both do what the name says. For userland, both are prefixed by "__"
to keep them out of the user namespace.
Change some memset/memcmp uses to the new functions where it makes
sense -- these are just some examples, more to come.

branches: 1.17.2; 1.17.4; 1.17.6; 1.17.8;
Migrate one last leftover bit (used only by the kernel now) to
sys/ufs/ufs and remove the old quota headers and no-longer-used shared
code. Ok by releng.

Don't use __cmsg_alignbytes in the kernel. Mark it as constant function
for userland as its value never changes. This allows the compiler to
optimise most invocations away.

branches: 1.15.2;
First step of random number subsystem rework described in
<20111022023242.BA26F14A158@mail.netbsd.org>. This change includes
the following:

An initial cleanup and minor reorganization of the entropy pool
code in sys/dev/rnd.c and sys/dev/rndpool.c. Several bugs are
fixed. Some effort is made to accumulate entropy more quickly at
boot time.

A generic interface, "rndsink", is added, for stream generators to
request that they be re-keyed with good quality entropy from the pool
as soon as it is available.

The arc4random()/arc4randbytes() implementation in libkern is
adjusted to use the rndsink interface for rekeying, which helps
address the problem of low-quality keys at boot time.

An implementation of the FIPS 140-2 statistical tests for random
number generator quality is provided (libkern/rngtest.c). This
is based on Greg Rose's implementation from Qualcomm.

A new random stream generator, nist_ctr_drbg, is provided. It is
based on an implementation of the NIST SP800-90 CTR_DRBG by
Henric Jungheim. This generator users AES in a modified counter
mode to generate a backtracking-resistant random stream.

An abstraction layer, "cprng", is provided for in-kernel consumers
of randomness. The arc4random/arc4randbytes API is deprecated for
in-kernel use. It is replaced by "cprng_strong". The current
cprng_fast implementation wraps the existing arc4random
implementation. The current cprng_strong implementation wraps the
new CTR_DRBG implementation. Both interfaces are rekeyed from
the entropy pool automatically at intervals justifiable from best
current cryptographic practice.

In some quick tests, cprng_fast() is about the same speed as
the old arc4randbytes(), and cprng_strong() is about 20% faster
than rnd_extract_data(). Performance is expected to improve.

The AES code in src/crypto/rijndael is no longer an optional
kernel component, as it is required by cprng_strong, which is
not an optional kernel component.

The entropy pool output is subjected to the rngtest tests at
startup time; if it fails, the system will reboot. There is
approximately a 3/10000 chance of a false positive from these
tests. Entropy pool _input_ from hardware random numbers is
subjected to the rngtest tests at attach time, as well as the
FIPS continuous-output test, to detect bad or stuck hardware
RNGs; if any are detected, they are detached, but the system
continues to run.

A problem with rndctl(8) is fixed -- datastructures with
pointers in arrays are no longer passed to userspace (this
was not a security problem, but rather a major issue for
compat32). A new kernel will require a new rndctl.

The sysctl kern.arandom() and kern.urandom() nodes are hooked
up to the new generators, but the /dev/*random pseudodevices
are not, yet.

Manual pages for the new kernel interfaces are forthcoming.

branches: 1.14.2;
Build and install ppath(3), property-list paths library.

Add a new libquota library, which contains some blocks to build and/or
parse quota plists; as well as a getfsquota() function to retrieve quotas
for a single id from a single filesystem (whatever filesystem this is:
a local quota-enabled fs or NFS). This is build on functions getufsquota()
(for local filesystems with UFS-like quotas) and getnfsquota();
which are also available to userland programs.
move functions from quota2_subr.c to libquota or libprop as appropriate,
and ajust in-tree quota tools.
move some declarations from kernel headers to either sys/quota.h or
quota/quota.h as appropriate. ufs/ufs/quota.h still installed because
it's needed by other installed ufs headers.
ufs/ufs/quota1.h still installed as a quick&dirty way to get a code
using the old quotactl() to compile (just include ufs/ufs/quota1.h instead of
ufs/ufs/quota.h - old code won't compile without this change and this is
on purpose).
Discussed on tech-kern@ and tech-net@ (long thread, but not much about
libquota itself ...)

Add quad srcs if building x86_64 32-bit libkern.

Add quad srcs if using MIPS O32 ABI

branches: 1.10.2; 1.10.4;
fix typo

branches: 1.9.2; 1.9.4;
Get rid of last "easy" kernel symbols starting with __:
__assert -> kern_assert
__sigtimedwait1 -> sigtimedwait1
__wdstart -> wdstart1

The rest are MD and/or shared with userspace, so they will require
a little more involvement than what is available for this quick
"ride the 5.99.24 bump" action.

Build fix for the mips64 merge:
- If memset2.c is in ${SRCS}, exclude conflicting memset.c.
- If MD byte_swap_8.* is in ${SRCS}, exclude conflicting bswap64.c.

Redo the previous inverted logic. Sort alphabetically.

Don't build quad support code on 64-bit platforms.

Move that majority of the 'SRCS+= foo.c' into the main Makefile.libkern.
Any .S files added by the arch/*/Makefile.inc cause the .c file to
be excluded.
Specific exclusions added using NO_SRCS to match previous files.
At least sparc, sparc64, i386, amd64 and vax GENERIC still build.
(There is a fubar with the naming of the byte-swap files ...)

Use stuff from libc/Makefile to auto-remove .c files if a .S has been added.
Use it to dispose of tne .c files that were already only conditionally
added.

Move popcount et al to src/common and add popcount32/popcount64.
Requested by rmind@. MD should now override popcount32/popcount64 and
provide the aliases as fitting.

branches: 1.2.2;
Fixes PR kern/41069 and PR kern/41070.

Extends the Opencrypto API to allow the destination buffer size to be
specified when its not the same size as the input buffer (i.e. for
operations like compress and decompress).
The crypto_op and crypt_n_op structures gain a u_int dst_len field.
The session_op structure gains a comp_alg field to specify a compression
algorithm.
Moved four ioctls to new ids; CIOCGSESSION, CIOCNGSESSION, CIOCCRYPT,
and CIOCNCRYPTM.
Added four backward compatible ioctls; OCIOCGSESSION, OCIOCNGSESSION,
OCIOCCRYPT, and OCIOCNCRYPTM.

Backward compatibility is maintained in ocryptodev.h and ocryptodev.c which
implement the original ioctls and set dst_len and comp_alg to 0.

Adds user-space access to compression features.

Adds software gzip support (CRYPTO_GZIP_COMP).

Adds the fast version of crc32 from zlib to libkern. This should be generally
useful and provide a place to start normalizing the various crc32 routines
in the kernel. The crc32 routine is used in this patch to support GZIP.

With input and support from tls@NetBSD.org.

branches: 1.1.2; 1.1.4; 1.1.6;
Split variables out of Makefile into Makefile.libkern so that we
can easily just .include it for the proper SRCS etc. definitions
in case we're interested in the files but not in building an actual
libkern.{a,o}. for librump

Get rid of last "easy" kernel symbols starting with __:
__assert -> kern_assert
__sigtimedwait1 -> sigtimedwait1
__wdstart -> wdstart1

The rest are MD and/or shared with userspace, so they will require
a little more involvement than what is available for this quick
"ride the 5.99.24 bump" action.

branches: 1.11.22; 1.11.40;
Rename __assert() to __kernassert() so that it doesn't collide
with the libc version. They take different arguments.

branches: 1.10.2; 1.10.4; 1.10.6; 1.10.8;
Do the panicstr check only if _KERNEL.

Disable kernel assertions if panicstr != NULL.

branches: 1.8.30; 1.8.40;
merge ktrace-lwp.

branches: 1.7.4;
KNF

branches: 1.6.6; 1.6.14; 1.6.16;
remove trailing \n in panic(). approved perry.

branches: 1.5.6; 1.5.8;
Don't prototype __assert() if not STANDALONE - <sys/systm.h> has already
included <lib/libkern/libkern.h> to get the prototype by this stage.

XXX: should we just change <sys/systm.h> from
#ifdef _KERNEL
#include <sys/systm.h>
#endif
to
#if defined(_KERNEL) || defined(_STANDALONE)
#include <sys/systm.h>
#endif
anyways?

branches: 1.4.2;
make libkern compile in userland - for testing purposes

branches: 1.3.4;
Can't include libkern.h twice which is what happens if _KERNEL is on.
For now, protect here with _STANDALONE.
XXX - must probably fix sys/systm.h

Pull libkern.h in.

Add machine-independent assertion-checking support. macros are:

assert() which always does assertion checking
unless "NDEBUG" is defined.

KASSERT() which does assertion checking if DIAGNOSTIC
is defined.

KDASSERT() which does assertion checking if DEBUG is
define. This macro exists for compatibility
with existing ports' assertion checking macros.
(Assertion checking is not typically an
"expensive" operation, and DIAGNOSTIC should be
used for inexpensive consistency checks.)

Use common sources with userland.

branches: 1.3.32;
don't need <sys/types.h> when including <sys/param.h>

branches: 1.2.6; 1.2.8; 1.2.10;
change copyright notice from 3-clause TNF -> 4-clause TNF.
not sure if which we prefer, but the point is to use exactly the same
copyright. From: matthew green <mrg@eterna.com.au>

make CMSG_ALIGN always synchronize with kernel's idea of ALIGNBYTES.
ancillary data alignment will be ALIGNBYTES, not sizeof(long) - 1, from now.

CMSG_xx will NOT resolve into constant. if you use CMSG_xx to allocate
arrays, you'll lose.

bump shlib minor for libc.

NOTE: if you are on top of arch with ALIGNBYTES != sizeof(long) - 1,
you need to recompile IPv6-related binaries. there is no way to guarantee
backward compat in this aspect. sorry for this. this should be the last
backward compat breakage for IPv6-related ancillary data manipulation.
(we still have PR 9516 for unix-domain sockets...)

ansify function definitions

Remove all the __P() from sys (excluding sys/dist)
Diff checked with grep and MK1 eyeball.
i386 and amd64 GENERIC and sys still build.

branches: 1.4.166; 1.4.176; 1.4.182;
- Add missing prototypes
- Bring prototypes into scope
- Correct prototype for skpc

new RCS ID format.

Fix spelling error in Copyright notice

branches: 1.1.2;
add __main()

remove insque/remque from libkern, they are no longer used

branches: 1.3.44; 1.3.46; 1.3.52;
merge ktrace-lwp.

branches: 1.2.4;
nuke trailing whitespace

branches: 1.1.2; 1.1.4; 1.1.24; 1.1.32; 1.1.34;
Move _insque()/_remque() to libkern. Once remaining uses would
be converted to <sys/queue.h> macros, _insque()/_remque() would be eliminated
altogether.

Use common sources with userland.

merge ktrace-lwp.

branches: 1.8.16;
Move UCB-licensed code from 4-clause to 3-clause licence.

Patches provided by Joel Baker in PR 22364, verified by myself.

branches: 1.7.6;
Sync with libc/quad:

As discussed (briefly) on tech-userlevel, fix our quad support to work
correctly on LP64 platforms. This is mostly just s/long/int/ in the
appropriate places.

branches: 1.6.26;
sync with libc

Update from libc.

copy back from libc

from libc

new RCS ID format.

branches: 1.1.2;
quad support copied in from libc.

Use common sources with userland.

merge ktrace-lwp.

branches: 1.7.16;
Move UCB-licensed code from 4-clause to 3-clause licence.

Patches provided by Joel Baker in PR 22364, verified by myself.

branches: 1.6.48;
sync with libc

Update from libc.

copy back from libc

from libc

new RCS ID format.

branches: 1.1.2;
quad support copied in from libc.

Merge tls-earlyentropy branch into HEAD.

branches: 1.35.6;
Replace consttime_bcmp/explicit_bzero by consttime_memequal/explicit_memset.

consttime_memequal is the same as the old consttime_bcmp.
explicit_memset is to memset as explicit_bzero was to bcmp.

Passes amd64 release and i386/ALL, but I'm sure I missed some spots,
so please let me know.

Account bytes drawn from initial call to libkern arc4random.

XXX This unlocked initialization looks rather bogus...

Rework rndsink(9) abstraction and adapt arc4random(9) and cprng(9).

rndsink(9):
- Simplify API.
- Simplify locking scheme.
- Add a man page.
- Avoid races in destruction.
- Avoid races in requesting entropy now and scheduling entropy later.

Periodic distribution of entropy to sinks reduces the need for the
last one, but this way we don't need to rely on periodic distribution
(e.g., in a future tickless NetBSD).

rndsinks_lock should probably eventually merge with the rndpool lock,
but we'll put that off for now.

cprng(9):
- Make struct cprng_strong opaque.
- Move rndpseudo.c parts that futz with cprng guts to subr_cprng.c.
- Fix kevent locking. (Is kevent locking documented anywhere?)
- Stub out rump cprng further until we can rumpify rndsink instead.
- Strip code to grovel through struct cprng_strong in fstat.

branches: 1.32.2;

Add a spin mutex to the rndsink structure; it is used to avoid lock
ordering and sleep-holding-locks problems when rekeying, and thus
to avoid a nasty race between cprng destruction and reseeding.

branches: 1.31.2;
void function _arc4randbytes cannot return a value.

Entropy-pool implementation move and cleanup.

1) Move core entropy-pool code and source/sink/sample management code
to sys/kern from sys/dev.

2) Remove use of NRND as test for presence of entropy-pool code throughout
source tree.

3) Remove use of RND_ENABLED in device drivers as microoptimization to
avoid expensive operations on disabled entropy sources; make the
rnd_add calls do this directly so all callers benefit.

4) Fix bug in recent rnd_add_data()/rnd_add_uint32() changes that might
have lead to slight entropy overestimation for some sources.

5) Add new source types for environmental sensors, power sensors, VM
system events, and skew between clocks, with a sample implementation
for each.

ok releng to go in before the branch due to the difficulty of later
pullup (widespread #ifdef removal and moved files). Tested with release
builds on amd64 and evbarm and live testing on amd64.

branches: 1.29.2;
make "rs" static -- this name is too unspecific for the global namespace

Remove rnd_extract_data from the public kernel API (it is for use by the
stream generators only). Clean up some related minor issues.

Remove arc4random() and arc4randbytes() from the kernel API. Replace
arc4random() hacks in rump with stubs that call the host arc4random() to
get numbers that are hopefully actually random (arc4random() keyed with
stack junk is not). This should fix some of the currently failing anita
tests -- we should no longer generate duplicate "random" MAC addresses in
the test environment.

Disable a DIAGNOSTIC test which causes kernel stack overflow on all arm ports.

Boot failure is reported from several arm users (nonaka@, ryo@, and others),
and too large auto variable in arc4_randrekey() is pointed out by enami@.

Use exact 32bit integral type to declare variable to store
32bit random value. This is necessary since it is overwritten
by four bytes.

Fix white space usage.

make it compile without DIAGNOSTIC defined

First step of random number subsystem rework described in
<20111022023242.BA26F14A158@mail.netbsd.org>. This change includes
the following:

An initial cleanup and minor reorganization of the entropy pool
code in sys/dev/rnd.c and sys/dev/rndpool.c. Several bugs are
fixed. Some effort is made to accumulate entropy more quickly at
boot time.

A generic interface, "rndsink", is added, for stream generators to
request that they be re-keyed with good quality entropy from the pool
as soon as it is available.

The arc4random()/arc4randbytes() implementation in libkern is
adjusted to use the rndsink interface for rekeying, which helps
address the problem of low-quality keys at boot time.

An implementation of the FIPS 140-2 statistical tests for random
number generator quality is provided (libkern/rngtest.c). This
is based on Greg Rose's implementation from Qualcomm.

A new random stream generator, nist_ctr_drbg, is provided. It is
based on an implementation of the NIST SP800-90 CTR_DRBG by
Henric Jungheim. This generator users AES in a modified counter
mode to generate a backtracking-resistant random stream.

An abstraction layer, "cprng", is provided for in-kernel consumers
of randomness. The arc4random/arc4randbytes API is deprecated for
in-kernel use. It is replaced by "cprng_strong". The current
cprng_fast implementation wraps the existing arc4random
implementation. The current cprng_strong implementation wraps the
new CTR_DRBG implementation. Both interfaces are rekeyed from
the entropy pool automatically at intervals justifiable from best
current cryptographic practice.

In some quick tests, cprng_fast() is about the same speed as
the old arc4randbytes(), and cprng_strong() is about 20% faster
than rnd_extract_data(). Performance is expected to improve.

The AES code in src/crypto/rijndael is no longer an optional
kernel component, as it is required by cprng_strong, which is
not an optional kernel component.

The entropy pool output is subjected to the rngtest tests at
startup time; if it fails, the system will reboot. There is
approximately a 3/10000 chance of a false positive from these
tests. Entropy pool _input_ from hardware random numbers is
subjected to the rngtest tests at attach time, as well as the
FIPS continuous-output test, to detect bad or stuck hardware
RNGs; if any are detected, they are detached, but the system
continues to run.

A problem with rndctl(8) is fixed -- datastructures with
pointers in arrays are no longer passed to userspace (this
was not a security problem, but rather a major issue for
compat32). A new kernel will require a new rndctl.

The sysctl kern.arandom() and kern.urandom() nodes are hooked
up to the new generators, but the /dev/*random pseudodevices
are not, yet.

Manual pages for the new kernel interfaces are forthcoming.

branches: 1.21.12;
Keep arc4_i and arc4_j synchronised after a rekeying. This prevents
accidentally ending up in a short ARC4 cycle.

branches: 1.20.10; 1.20.16; 1.20.18;
Remove clause 3 and 4 from TNF licenses

branches: 1.19.26; 1.19.28; 1.19.30; 1.19.32;
include sys/cdefs.h

branches: 1.18.2; 1.18.8; 1.18.18; 1.18.22;
- call the initialization routine for arc4randbytes.
- knf (add continues)

branches: 1.17.6; 1.17.8;
merge FreeBSD timecounters from branch simonb-timecounters
- struct timeval time is gone
time.tv_sec -> time_second
- struct timeval mono_time is gone
mono_time.tv_sec -> time_uptime
- access to time via
{get,}{micro,nano,bin}time()
get* versions are fast but less precise
- support NTP nanokernel implementation (NTP API 4)
- further reading:
Timecounter Paper: http://phk.freebsd.dk/pubs/timecounter.pdf
NTP Nanokernel: http://www.eecis.udel.edu/~mills/ntp/html/kern.html

branches: 1.16.4; 1.16.6; 1.16.8; 1.16.14;
Remove leading __ from __(const|inline|signed|volatile) -- it is obsolete.

merge ktrace-lwp.

branches: 1.14.4;
nuke trailing whitespace

branches: 1.13.4; 1.13.6;
Redo part of rev. 1.10.

arc4random: fix a bug introduced by rev.1.7.
actually generate four bytes random value, rather than
leaving a byte always zero.

branches: 1.11.2;
fix build for bootloaders (no /usr/include/sys/kernel.h any more).

KNF

branches: 1.9.6;
let this compile in the non KERNEL case without NRND.

ESP output was drawing down the entropy pool at a ferocious rate, a
particular problem on hosts with only wireless interfaces that are
definitely not safe to use as entropy sources.

Add arc4randbytes() which hands out bytes from the same source used
by arc4random(). This is intended to be a _temporary_ interface
until we can design and implement a better general PRNG interface
that is decoupled from the entropy-pool implementation.

Modify key_randomfill() (used only for initialization vectors on
SA creation and via key_sa_stir_iv(), which does not "stir",
despite its name) to use arc4randbytes() instead of pulling bits
directly from the entropy pool. It is my hope that this change
will pose minimal integration problems for the KAME folks as the
random-pool interface is *already* different between each BSD
variant; this just simplifies the NetBSD case and solves a
fairly serious problem.

Note that it is generally considered acceptable cryptographic
practice to use a fast stream cipher to generate IVs for encryption
with stronger block ciphers. For example, the use of "non-Approved"
PRNGs to generate IVs for "Approved" block ciphers is explicitly
sanctioned by FIPS 140-2.

This commit includes two major changes:

1) Speed up arc4random(). We make arc4randbyte() inline, which makes this
not much slower than, say, the other arc4 implementation in our kernel.

We also replace four calls to arc4randbyte() with a loop, saving about
20% on some processors where the "unrolled" arc4randbyte() calls would
needlessly stomp the cache.

2) Address various problems with the initialization/"stirring" code,
primarily in the area of handling of the source data from the kernel
entropy pool. We used to:

a) Ask the entropy pool for 32 bytes

b) If we got zero bytes, key with junk from the stack (ouch!)
which has some nasty implications, to say the least. For
example, we're most likely to get zero bytes at boot time,
when the stack contents are even more predictable than usual.

c) If we got less than 32 bytes but more than zero bytes, use
however many bytes we got as the arc4 key, copying it
repeatedly as per usual arc4 key setup.

Because of the way NetBSD's entropy pool works, this was
mostly harmless, because if you ask for RND_EXTRACT_ANY,
you always get as many bytes as you ask for. However,
this is probably a security hole in the original FreeBSD
code, where AFAICT you might end up using an 8-bit arc4
key -- not good, much worse than using the output of the
entropy pool hash function even when it thinks it only
has 8 bits of entropy to give you.

One thing this code could do on NetBSD that was not so
good was to replace a key with a lot of entropy with
one with less entropy. That's clearly counterproductive.

The new code, instead:

a) Asks for 32 good bytes. If it gets them, use them as the
arc4 key in the usual way.

b) Tracks how many entropy bytes the key it's replacing had.
If the new entropy request got less bytes, leave the old
key in place. Note that the first time through, the "old
key" had zero bytes, so we'll always replace it.

c) If we get less then 32 bytes but more than we had, request
EXTRACT_ANY bytes from the entropy pool, padding the key
out to 32 bytes which we then use as the arc4 key in the
usual way.

This is still really all rather backwards. Instead of this generator
deciding to rekey itself using a basically arbitrary metric, it should
register a callback so that the entropy pool code could rekey it when
a lot of bits were available. Details at 11.

Finally, rename the "stir" function (which did not stir) to "rekey",
which is what it actually does.

include rnd.h only under kernel build.
caveat: arc4random() will not get stirred in bootstrap code.

add missing "rnd.h" include - noted by simonb

branches: 1.4.2; 1.4.4;
discard 256 bytes of output every time we stir (not just when initializing)

branches: 1.3.2;
need libkern.h for bootloaders

no need for libkern.h

have arc4random(9).

Use common sources with userland.

merge ktrace-lwp.

branches: 1.8.16;
Move UCB-licensed code from 4-clause to 3-clause licence.

Patches provided by Joel Baker in PR 22364, verified by myself.

branches: 1.7.6;
Sync with libc/quad:

As discussed (briefly) on tech-userlevel, fix our quad support to work
correctly on LP64 platforms. This is mostly just s/long/int/ in the
appropriate places.

branches: 1.6.26;
sync with libc

Update from libc.

copy back from libc

from libc

new RCS ID format.

branches: 1.1.2;
quad support copied in from libc.

Use common sources with userland.

merge ktrace-lwp.

branches: 1.9.16;
Move UCB-licensed code from 4-clause to 3-clause licence.

Patches provided by Joel Baker in PR 22364, verified by myself.

branches: 1.8.6;
Sync with libc/quad:

As discussed (briefly) on tech-userlevel, fix our quad support to work
correctly on LP64 platforms. This is mostly just s/long/int/ in the
appropriate places.

branches: 1.7.14;
sync with libc

sync with libc

Update from libc.

copy back from libc

from libc

new RCS ID format.

branches: 1.1.2;
quad support copied in from libc.

bcdtobin and bintobcd are now inlines in <dev/clock_subr.h>

branches: 1.1.2; 1.1.6; 1.1.8; 1.1.14; 1.1.20; 1.1.120;
Provide BCD<->binary conversion in libkern and turn <dev/clock_subr.h>'s
FROMBCD()/TOBCD() macros into wrappers around it, resulting in both
smaller code footprint and elimination of possible issues due to
multiple evaluation of macro arguments.

Suggested by Simon Burge and Anders Gavare on tech-kern.

Use common sources with userland.

merge ktrace-lwp.

branches: 1.12.16;
Move UCB-licensed code from 4-clause to 3-clause licence.

Patches provided by Joel Baker in PR 22364, verified by myself.

branches: 1.11.42;
Sync with src/lib/libc/string/bcmp.c

sync with libc

rcsid fascism

branches: 1.8.10;
when casting const pointers, preserve const qualifiers. (-Wcast-qual)

don't include user-land headers in libkern files when compiling for
the kernel.

- Add missing prototypes
- Bring prototypes into scope
- Correct prototype for skpc

Update from libc.

copy back from libc

from libc

new RCS ID format.

branches: 1.1.2;
Did a ``make copy-to-libkern'' from libc to copy libc files to libkern.
Checking them in to make people happy.

Use common sources with userland.

merge ktrace-lwp.

branches: 1.6.2;
add missing const

Move UCB-licensed code from 4-clause to 3-clause licence.

Patches provided by Joel Baker in PR 22364, verified by myself.

branches: 1.4.2;
When MEMCOPY is defined, don't bother checking if a backwards-copy is
required. That's what memmove() is for.

This should fix port-powerpc/16889. The backwards copyin can confuse
uiomove/genfs_getpages, resulting in corruption of files written over NFS.

sync with libc

Sync with libc.

Need bcopy.c to generate memcpy().

Use common sources with userland.

merge ktrace-lwp.

branches: 1.3.16;
netbsd.org -> NetBSD.org

This was the last commit of this kind to src/sys, which is now totally
"NetBSD.org clean". Thanks for the patiance, and sorry for all the commits.

branches: 1.2.42;
Move the bswap functions from libutil to libc (this bups the
minor of libc and the major of libutil). For little-endian architectures
merge the bnswap() assembly versions with nto* and hton* using symbols
aliasing. Use symbol renaming for the bswap function in this case to avoid
namespace pollution.
Declare bswap* in machine/bswap.h, not machine/endian.h. For little-endian
machines, common code for inline macros go in machine/byte_swap.h
Sync libkern with libc.
Adjust #include in kernel sources for machine/bswap.h.

branches: 1.1.2;
Add byte-swapping functions (bswap16, bswap32, bswap64) to libkern.
Only assembly version for i386 bswap16 and bswap32 for now (bswap64 uses
bswap32). Contribution of assembly versions of these are welcome.
Add byte-swapping of ext2fs metadata for big-endian systems.
Tested on i386 and sparc.

Use common sources with userland.

merge ktrace-lwp.

branches: 1.3.16;
netbsd.org -> NetBSD.org

This was the last commit of this kind to src/sys, which is now totally
"NetBSD.org clean". Thanks for the patiance, and sorry for all the commits.

branches: 1.2.42;
Move the bswap functions from libutil to libc (this bups the
minor of libc and the major of libutil). For little-endian architectures
merge the bnswap() assembly versions with nto* and hton* using symbols
aliasing. Use symbol renaming for the bswap function in this case to avoid
namespace pollution.
Declare bswap* in machine/bswap.h, not machine/endian.h. For little-endian
machines, common code for inline macros go in machine/byte_swap.h
Sync libkern with libc.
Adjust #include in kernel sources for machine/bswap.h.

branches: 1.1.2;
Add byte-swapping functions (bswap16, bswap32, bswap64) to libkern.
Only assembly version for i386 bswap16 and bswap32 for now (bswap64 uses
bswap32). Contribution of assembly versions of these are welcome.
Add byte-swapping of ext2fs metadata for big-endian systems.
Tested on i386 and sparc.

Use common sources with userland.

merge ktrace-lwp.

branches: 1.5.16;
netbsd.org -> NetBSD.org

This was the last commit of this kind to src/sys, which is now totally
"NetBSD.org clean". Thanks for the patiance, and sorry for all the commits.

Sync with userland version.

branches: 1.3.42;
Sync with libc version.

Move the bswap functions from libutil to libc (this bups the
minor of libc and the major of libutil). For little-endian architectures
merge the bnswap() assembly versions with nto* and hton* using symbols
aliasing. Use symbol renaming for the bswap function in this case to avoid
namespace pollution.
Declare bswap* in machine/bswap.h, not machine/endian.h. For little-endian
machines, common code for inline macros go in machine/byte_swap.h
Sync libkern with libc.
Adjust #include in kernel sources for machine/bswap.h.

branches: 1.1.2;
Add byte-swapping functions (bswap16, bswap32, bswap64) to libkern.
Only assembly version for i386 bswap16 and bswap32 for now (bswap64 uses
bswap32). Contribution of assembly versions of these are welcome.
Add byte-swapping of ext2fs metadata for big-endian systems.
Tested on i386 and sparc.

Use common sources with userland.

branches: 1.9.62;
update bzero.c to stub from libc/string

sync with libc

rcsid fascism

branches: 1.6.12;
don't include user-land headers in libkern files when compiling for
the kernel.

Update from libc.

copy back from libc

from libc

new RCS ID format.

branches: 1.1.2;
Did a ``make copy-to-libkern'' from libc to copy libc files to libkern.
Checking them in to make people happy.

Use common sources with userland.

merge ktrace-lwp.

branches: 1.7.16;
Move UCB-licensed code from 4-clause to 3-clause licence.

Patches provided by Joel Baker in PR 22364, verified by myself.

branches: 1.6.48;
sync with libc

Update from libc.

copy back from libc

from libc

new RCS ID format.

branches: 1.1.2;
quad support copied in from libc.

Make copystr() a MI C function, part of libkern and shared on all
architectures.

Notes:

- On alpha and ia64 the function is kept but gets renamed locally to avoid
symbol collision. This is because on these two arches, I am not sure
whether the ASM callers do not rely on fixed registers, so I prefer to
keep the ASM body for now.
- On Vax, only the symbol is removed, because the body is used from other
functions.
- On RISC-V, this change fixes a bug: copystr() was just a wrapper around
strlcpy(), but strlcpy() makes the operation less safe (strlen on the
source beyond its size).
- The kASan, kCSan and kMSan wrappers are removed, because now that
copystr() is in C, the compiler transformations are applied to it,
without the need for manual wrappers.

Could test on amd64 only, but should be fine.

Revert previous for now:
http://mail-index.netbsd.org/source-changes/2020/04/16/msg116278.html

The reasoning turned out to be wrong; __KERNEL_RCSID() in header files
does *not* overwrite RCSID in main source files. The real problem is that
it inserts its RCSID into *every* object files. However, it can be still
useful even if heavily duplicated.

Stop using __KERNEL_RCSID() in header files; it confuses ident(1) by
overwriting RCSID in main source files.

XXX
The first argument of __KERNEL_RCSID() is neglected for ELF. If we wish
to have RCSID of header files in kernel binary, we need something like
__FBSDID() macro in FreeBSD.

branches: 1.1.2; 1.1.4; 1.1.18; 1.1.32;
extract crc16 code from utoppy driver to a separate file in libkern, for use
by ext2fs; every usage gets their own copy of the table for now, as it's
fairly small, and too rare to be used to be useful pulling into every kernel

branches: 1.4.2; 1.4.4; 1.4.6; 1.4.8; 1.4.14;
Make the function declaration the same for the big-endian case as
for the little-endian case, and consistent with the libkern.h
declaration. Fixes build problem for at least hp700 and evbbarm-eb.

Fix compilation error on 64-bit platforms.

Fix build problems caused by crc32 addition to libkern. Also, this makes
the i386 bootblocks about 2K smaller than they were before we monkeyed
with crc32 at all.

Fixes PR kern/41069 and PR kern/41070.

Extends the Opencrypto API to allow the destination buffer size to be
specified when its not the same size as the input buffer (i.e. for
operations like compress and decompress).
The crypto_op and crypt_n_op structures gain a u_int dst_len field.
The session_op structure gains a comp_alg field to specify a compression
algorithm.
Moved four ioctls to new ids; CIOCGSESSION, CIOCNGSESSION, CIOCCRYPT,
and CIOCNCRYPTM.
Added four backward compatible ioctls; OCIOCGSESSION, OCIOCNGSESSION,
OCIOCCRYPT, and OCIOCNCRYPTM.

Backward compatibility is maintained in ocryptodev.h and ocryptodev.c which
implement the original ioctls and set dst_len and comp_alg to 0.

Adds user-space access to compression features.

Adds software gzip support (CRYPTO_GZIP_COMP).

Adds the fast version of crc32 from zlib to libkern. This should be generally
useful and provide a place to start normalizing the various crc32 routines
in the kernel. The crc32 routine is used in this patch to support GZIP.

With input and support from tls@NetBSD.org.

branches: 1.1.2; 1.1.4; 1.1.6; 1.1.8; 1.1.14;
Fixes PR kern/41069 and PR kern/41070.

Extends the Opencrypto API to allow the destination buffer size to be
specified when its not the same size as the input buffer (i.e. for
operations like compress and decompress).
The crypto_op and crypt_n_op structures gain a u_int dst_len field.
The session_op structure gains a comp_alg field to specify a compression
algorithm.
Moved four ioctls to new ids; CIOCGSESSION, CIOCNGSESSION, CIOCCRYPT,
and CIOCNCRYPTM.
Added four backward compatible ioctls; OCIOCGSESSION, OCIOCNGSESSION,
OCIOCCRYPT, and OCIOCNCRYPTM.

Backward compatibility is maintained in ocryptodev.h and ocryptodev.c which
implement the original ioctls and set dst_len and comp_alg to 0.

Adds user-space access to compression features.

Adds software gzip support (CRYPTO_GZIP_COMP).

Adds the fast version of crc32 from zlib to libkern. This should be generally
useful and provide a place to start normalizing the various crc32 routines
in the kernel. The crc32 routine is used in this patch to support GZIP.

With input and support from tls@NetBSD.org.

branches: 1.1.2; 1.1.6;
move bi-endian disklabel support from the kernel and libsa into libkern.

- dkcksum() and dkcksum_sized() move from subr_disk.c and from
libsa into libkern/dkcksum.c (which is missing _sized() version),
using the version from usr.sbin/disklabel.

- swap_disklabel() moves from subr_disk_mbr.c into libkern, now called
disklabel_swap(). (the sh3 version should be updated to use this.)

- DISKLABEL_EI becomes a first-class option with opt_disklabel.h.

- add libkern.h to libsa/disklabel.c.

this enables future work for bi-endian libsa/ufs.c (relevant for ffsv1,
ffsv2, lfsv1, and lfsv2), as well as making it possible for ports not
using subr_disk_mbr.c to include bi-endian disklabel support (which,
afaict, includes any disk on mbr-supporting platforms that do not have
an mbr as well as disklabel.)

builds successsfully on: alpha, i386, amd64, sun2, sun3, evbarm64,
evbarm64-eb, sparc, and sparc64. tested in anita on i386 and sparc,
testing in hardware on evbarm64*.

Use common sources with userland.

merge ktrace-lwp.

branches: 1.7.16;
Move UCB-licensed code from 4-clause to 3-clause licence.

Patches provided by Joel Baker in PR 22364, verified by myself.

branches: 1.6.48;
sync with libc

Update from libc.

copy back from libc

from libc

new RCS ID format.

branches: 1.1.2;
quad support copied in from libc.

branches: 1.1.2; 1.1.6;
move bi-endian disklabel support from the kernel and libsa into libkern.

- dkcksum() and dkcksum_sized() move from subr_disk.c and from
libsa into libkern/dkcksum.c (which is missing _sized() version),
using the version from usr.sbin/disklabel.

- swap_disklabel() moves from subr_disk_mbr.c into libkern, now called
disklabel_swap(). (the sh3 version should be updated to use this.)

- DISKLABEL_EI becomes a first-class option with opt_disklabel.h.

- add libkern.h to libsa/disklabel.c.

this enables future work for bi-endian libsa/ufs.c (relevant for ffsv1,
ffsv2, lfsv1, and lfsv2), as well as making it possible for ports not
using subr_disk_mbr.c to include bi-endian disklabel support (which,
afaict, includes any disk on mbr-supporting platforms that do not have
an mbr as well as disklabel.)

builds successsfully on: alpha, i386, amd64, sun2, sun3, evbarm64,
evbarm64-eb, sparc, and sparc64. tested in anita on i386 and sparc,
testing in hardware on evbarm64*.

Rewrite entropy subsystem.

Primary goals:

1. Use cryptography primitives designed and vetted by cryptographers.
2. Be honest about entropy estimation.
3. Propagate full entropy as soon as possible.
4. Simplify the APIs.
5. Reduce overhead of rnd_add_data and cprng_strong.
6. Reduce side channels of HWRNG data and human input sources.
7. Improve visibility of operation with sysctl and event counters.

Caveat: rngtest is no longer used generically for RND_TYPE_RNG
rndsources. Hardware RNG devices should have hardware-specific
health tests. For example, checking for two repeated 256-bit outputs
works to detect AMD's 2019 RDRAND bug. Not all hardware RNGs are
necessarily designed to produce exactly uniform output.

ENTROPY POOL

- A Keccak sponge, with test vectors, replaces the old LFSR/SHA-1
kludge as the cryptographic primitive.

- `Entropy depletion' is available for testing purposes with a sysctl
knob kern.entropy.depletion; otherwise it is disabled, and once the
system reaches full entropy it is assumed to stay there as far as
modern cryptography is concerned.

- No `entropy estimation' based on sample values. Such `entropy
estimation' is a contradiction in terms, dishonest to users, and a
potential source of side channels. It is the responsibility of the
driver author to study the entropy of the process that generates
the samples.

- Per-CPU gathering pools avoid contention on a global queue.

- Entropy is occasionally consolidated into global pool -- as soon as
it's ready, if we've never reached full entropy, and with a rate
limit afterward. Operators can force consolidation now by running
sysctl -w kern.entropy.consolidate=1.

- rndsink(9) API has been replaced by an epoch counter which changes
whenever entropy is consolidated into the global pool.
. Usage: Cache entropy_epoch() when you seed. If entropy_epoch()
has changed when you're about to use whatever you seeded, reseed.
. Epoch is never zero, so initialize cache to 0 if you want to reseed
on first use.
. Epoch is -1 iff we have never reached full entropy -- in other
words, the old rnd_initial_entropy is (entropy_epoch() != -1) --
but it is better if you check for changes rather than for -1, so
that if the system estimated its own entropy incorrectly, entropy
consolidation has the opportunity to prevent future compromise.

- Sysctls and event counters provide operator visibility into what's
happening:
. kern.entropy.needed - bits of entropy short of full entropy
. kern.entropy.pending - bits known to be pending in per-CPU pools,
can be consolidated with sysctl -w kern.entropy.consolidate=1
. kern.entropy.epoch - number of times consolidation has happened,
never 0, and -1 iff we have never reached full entropy

CPRNG_STRONG

- A cprng_strong instance is now a collection of per-CPU NIST
Hash_DRBGs. There are only two in the system: user_cprng for
/dev/urandom and sysctl kern.?random, and kern_cprng for kernel
users which may need to operate in interrupt context up to IPL_VM.

(Calling cprng_strong in interrupt context does not strike me as a
particularly good idea, so I added an event counter to see whether
anything actually does.)

- Event counters provide operator visibility into when reseeding
happens.

INTEL RDRAND/RDSEED, VIA C3 RNG (CPU_RNG)

- Unwired for now; will be rewired in a subsequent commit.

Rewrite entropy subsystem.

Primary goals:

1. Use cryptography primitives designed and vetted by cryptographers.
2. Be honest about entropy estimation.
3. Propagate full entropy as soon as possible.
4. Simplify the APIs.
5. Reduce overhead of rnd_add_data and cprng_strong.
6. Reduce side channels of HWRNG data and human input sources.
7. Improve visibility of operation with sysctl and event counters.

Caveat: rngtest is no longer used generically for RND_TYPE_RNG
rndsources. Hardware RNG devices should have hardware-specific
health tests. For example, checking for two repeated 256-bit outputs
works to detect AMD's 2019 RDRAND bug. Not all hardware RNGs are
necessarily designed to produce exactly uniform output.

ENTROPY POOL

- A Keccak sponge, with test vectors, replaces the old LFSR/SHA-1
kludge as the cryptographic primitive.

- `Entropy depletion' is available for testing purposes with a sysctl
knob kern.entropy.depletion; otherwise it is disabled, and once the
system reaches full entropy it is assumed to stay there as far as
modern cryptography is concerned.

- No `entropy estimation' based on sample values. Such `entropy
estimation' is a contradiction in terms, dishonest to users, and a
potential source of side channels. It is the responsibility of the
driver author to study the entropy of the process that generates
the samples.

- Per-CPU gathering pools avoid contention on a global queue.

- Entropy is occasionally consolidated into global pool -- as soon as
it's ready, if we've never reached full entropy, and with a rate
limit afterward. Operators can force consolidation now by running
sysctl -w kern.entropy.consolidate=1.

- rndsink(9) API has been replaced by an epoch counter which changes
whenever entropy is consolidated into the global pool.
. Usage: Cache entropy_epoch() when you seed. If entropy_epoch()
has changed when you're about to use whatever you seeded, reseed.
. Epoch is never zero, so initialize cache to 0 if you want to reseed
on first use.
. Epoch is -1 iff we have never reached full entropy -- in other
words, the old rnd_initial_entropy is (entropy_epoch() != -1) --
but it is better if you check for changes rather than for -1, so
that if the system estimated its own entropy incorrectly, entropy
consolidation has the opportunity to prevent future compromise.

- Sysctls and event counters provide operator visibility into what's
happening:
. kern.entropy.needed - bits of entropy short of full entropy
. kern.entropy.pending - bits known to be pending in per-CPU pools,
can be consolidated with sysctl -w kern.entropy.consolidate=1
. kern.entropy.epoch - number of times consolidation has happened,
never 0, and -1 iff we have never reached full entropy

CPRNG_STRONG

- A cprng_strong instance is now a collection of per-CPU NIST
Hash_DRBGs. There are only two in the system: user_cprng for
/dev/urandom and sysctl kern.?random, and kern_cprng for kernel
users which may need to operate in interrupt context up to IPL_VM.

(Calling cprng_strong in interrupt context does not strike me as a
particularly good idea, so I added an event counter to see whether
anything actually does.)

- Event counters provide operator visibility into when reseeding
happens.

INTEL RDRAND/RDSEED, VIA C3 RNG (CPU_RNG)

- Unwired for now; will be rewired in a subsequent commit.

Use common sources with userland.

merge ktrace-lwp.

branches: 1.10.16;
Move UCB-licensed code from 4-clause to 3-clause licence.

Patches provided by Joel Baker in PR 22364, verified by myself.

branches: 1.9.6;
#undef ffs before the ffs() function definition, so that the name
isn't rewrote to __builtin_ffs() on gcc 2.95+; this fixes compilation
on sun2
also change the comment for function

branches: 1.8.26; 1.8.28; 1.8.42;
sync with libc

rcsid fascism

branches: 1.6.12;
don't include user-land headers in libkern files when compiling for
the kernel.

Update from libc.

copy back from libc

from libc

new RCS ID format.

branches: 1.1.2;
Did a ``make copy-to-libkern'' from libc to copy libc files to libkern.
Checking them in to make people happy.

Even smaller and takes print function.

don't ifdef _KERNEL

use a different, more readable approach.

coalesce the two copies of hexdump into libkern

Use common sources with userland.

merge ktrace-lwp.

branches: 1.12.4;
nuke trailing whitespace

branches: 1.11.8; 1.11.10;
netbsd.org -> NetBSD.org

This was the last commit of this kind to src/sys, which is now totally
"NetBSD.org clean". Thanks for the patiance, and sorry for all the commits.

branches: 1.10.20;
sync argument/return type of [hn]to[nh][ls] to XNET 5.2 (uint{16,32}_t).
as discussed on tech-net.

branches: 1.9.26; 1.9.28;
sync with libc

pull down from libc

Copy from libkern:
Add some casts so that we're not bitten by type promotions on 64-bit
systems. Not a great concern since the only 64-bit system currently
supported by NetBSD (Alpha) uses assembly versions of these routines.

branches: 1.6.6;
Update from libc.

copy back from libc

from libc

new RCS ID format.

Fix spelling error in Copyright notice

branches: 1.1.2;
Did a ``make copy-to-libkern'' from libc to copy libc files to libkern.
Checking them in to make people happy.

Use common sources with userland.

merge ktrace-lwp.

branches: 1.11.16;
netbsd.org -> NetBSD.org

This was the last commit of this kind to src/sys, which is now totally
"NetBSD.org clean". Thanks for the patiance, and sorry for all the commits.

branches: 1.10.20;
sync argument/return type of [hn]to[nh][ls] to XNET 5.2 (uint{16,32}_t).
as discussed on tech-net.

branches: 1.9.26; 1.9.28;
sync with libc

pull down from libc

Copy from libkern:
Add some casts so that we're not bitten by type promotions on 64-bit
systems. Not a great concern since the only 64-bit system currently
supported by NetBSD (Alpha) uses assembly versions of these routines.

branches: 1.6.6;
Update from libc.

copy back from libc

from libc

new RCS ID format.

Fix spelling error in Copyright notice

branches: 1.1.2;
Did a ``make copy-to-libkern'' from libc to copy libc files to libkern.
Checking them in to make people happy.

ANSIfy another 1261 function definitions.
The only ones left in sys are beyond by sed script!
(or in sys/dist or sys/external)
Mostly they have function pointer parameters.

branches: 1.5.74; 1.5.84; 1.5.90;
merge ktrace-lwp.

Move UCB-licensed code from 4-clause to 3-clause licence.

Patches provided by Joel Baker in PR 22364, verified by myself.

branches: 1.3.64;
- Add missing prototypes
- Bring prototypes into scope
- Correct prototype for skpc

new RCS ID format.

branches: 1.1.2;
move 'boot' and 'libkern' into 'lib'. generally clean up 'kernel library'
building and dependencies.

ANSIfy another 1261 function definitions.
The only ones left in sys are beyond by sed script!
(or in sys/dist or sys/external)
Mostly they have function pointer parameters.

branches: 1.5.74; 1.5.84; 1.5.90;
merge ktrace-lwp.

Move UCB-licensed code from 4-clause to 3-clause licence.

Patches provided by Joel Baker in PR 22364, verified by myself.

branches: 1.3.64;
- Add missing prototypes
- Bring prototypes into scope
- Correct prototype for skpc

new RCS ID format.

branches: 1.1.2;
move 'boot' and 'libkern' into 'lib'. generally clean up 'kernel library'
building and dependencies.

g/c index.c and rindex.c (not used anywhere)

Move UCB-licensed code from 4-clause to 3-clause licence.

Patches provided by Joel Baker in PR 22364, verified by myself.

branches: 1.3.48;
sync with libc

Don't even think about cdefs.h here...

add rules like those in libc to build strchr() and strrchr() (the latter
is being added to libkern by this commit). remove strchr.c from
archs' Makefile.inc files to go along with this. (If an arch has
a .S version, it should be copied down and added to the apropriate
Makefile.inc.)

Use common sources with userland.

branches: 1.6.38;
ctype-like functions are now in libkern.

branches: 1.5.6;
Kill some more register declarations.

branches: 1.4.2;
make libkern compile in userland - for testing purposes

This was an n_long, not u_long, so u_int32_t is correct.

libkern just got an inet_addr(), but it won't compile, no prototype. Cleanup...
* Add prototype to libkern.h.
* Remove the almost-identical-copy from libsa/net.[ch].
* Change its type back to the (wrong, but harmless) historical one. (u_long)
* Kill the XXX local prototype in nfs_bootparam.c

branches: 1.1.10;
Copied from src/lib/libc/net/inet_pton.c
and adapted for kernel and libsa use.

Change about 4500 of the K&R function definitions to ANSI ones.
There are still about 1600 left, but they have ',' or /* ... */
in the actual variable definitions - which my awk script doesn't handle.
There are also many that need () -> (void).
(The script does handle misordered arguments.)

Remove all the __P() from sys (excluding sys/dist)
Diff checked with grep and MK1 eyeball.
i386 and amd64 GENERIC and sys still build.

branches: 1.1.2; 1.1.140; 1.1.150; 1.1.156;
move intoa() from libsa:net.c to libkern, turn inet_ntoa() into a macro,
nuke ip_convertaddr()

Use common sources with userland.

merge ktrace-lwp.

branches: 1.7.16;
Move UCB-licensed code from 4-clause to 3-clause licence.

Patches provided by Joel Baker in PR 22364, verified by myself.

branches: 1.6.48;
sync with libc

Update from libc.

copy back from libc

from libc

new RCS ID format.

branches: 1.1.2;
quad support copied in from libc.

revert rev 1.4 ("Add garbage instructions at end of kern_assert after vpanic.")

that change had no effect because vpanic() is marked __dead / noreturn
and thus the compiler would optimize away everything after a call to vpanic().
the original problem has now been fixed differently (but only for x86 so far).

Add garbage instructions at end of kern_assert after vpanic.

This avoids having the return address passed to vpanic confusingly
appear to be some other symbol.

branches: 1.3.6; 1.3.10; 1.3.14;
For the purposes of static analysis, this never returns.

branches: 1.2.2; 1.2.12; 1.2.16;
Don't include <sys/systm.h> because it brings in too much stuff that
conflicts with standalone code. Instead modify kern_assert() to be like
panic() and call that.

branches: 1.1.4;
Get rid of last "easy" kernel symbols starting with __:
__assert -> kern_assert
__sigtimedwait1 -> sigtimedwait1
__wdstart -> wdstart1

The rest are MD and/or shared with userspace, so they will require
a little more involvement than what is available for this quick
"ride the 5.99.24 bump" action.

libc: New _r variants of heapsort, mergesort, qsort.

Also kheapsort_r for kernel/standalone use.

These variants allow the caller to pass a cookie through to the
comparison function, e.g. if you want to sort an array of indices
into a buffer.

qsort_r is new in POSIX.1-2024; the others are obvious analogues of
our nonstandard extensions for heapsort and mergesort.

PR lib/58931: qsort_r() missing

string.h: Fix various symbol visibility issues.

1. Order declarations according to POSIX 2024 to make this easier to
review side-by-side with the spec.
2. Fix visibility of memccpy: XSI-only, not POSIX in general; require
_XOPEN_SOURCE, not just _POSIX_C_SOURCE.
3. Omit redundant _XOPEN_SOURCE test around stpcpy/stpncpy.
4. Hide strdup in POSIX 2001. Not POSIX (without XSI) until 2008.
5. Hide strerror_r until POSIX 2001. Can't find evidence of it in
any earlier POSIX or X/Open. (Not 100% sure on this one, maybe
someone can double-check my research.)
6. Add restrict to strlcat/strlcpy.
7. Omit redundant _XOPEN_SOURCE test around strndup and strnlen.
8. Hide strtok_r until POSIX 2001. Can't find evidence of it in
any earlier POSIX or X/Open. (Not 100% sure on this one, maybe
someone can double-check my research.)

Carry the restrict qualifiers on strlcat/strlcpy to libkern too.

Main reference:
https://pubs.opengroup.org/onlinepubs/9799919799/basedefs/string.h.html

PR standards/58804: string.h: wrong visibility for memccpy

deduplicate offsetof (stddef.h) and container_of (container_of.h)

branches: 1.145.6;
fix the example for container_of().

needs to be a pointer into the containing structure, not the
value of a pointer inside the structure.

libkern: Make KASSERT verify expression is valid if !DIAGNOSTIC.

This way it is no longer necessary to mark variables __diagused if
they are used in KASSERT conditions.

Fix fallout from this by removing now-unnecessary and `#ifdef
DIAGNOSTIC'.

Don't do the same for KDASSERT if !DEBUG -- unlike KASSERT and
DIAGNOSTIC, variables needed by KDASSERT and DEBUG are likely to be
expensive to compute (and potentially difficult for a compiler to
prove flushable), so we don't want to require them under !DEBUG.

move bi-endian disklabel support from the kernel and libsa into libkern.

- dkcksum() and dkcksum_sized() move from subr_disk.c and from
libsa into libkern/dkcksum.c (which is missing _sized() version),
using the version from usr.sbin/disklabel.

- swap_disklabel() moves from subr_disk_mbr.c into libkern, now called
disklabel_swap(). (the sh3 version should be updated to use this.)

- DISKLABEL_EI becomes a first-class option with opt_disklabel.h.

- add libkern.h to libsa/disklabel.c.

this enables future work for bi-endian libsa/ufs.c (relevant for ffsv1,
ffsv2, lfsv1, and lfsv2), as well as making it possible for ports not
using subr_disk_mbr.c to include bi-endian disklabel support (which,
afaict, includes any disk on mbr-supporting platforms that do not have
an mbr as well as disklabel.)

builds successsfully on: alpha, i386, amd64, sun2, sun3, evbarm64,
evbarm64-eb, sparc, and sparc64. tested in anita on i386 and sparc,
testing in hardware on evbarm64*.

branches: 1.142.4; 1.142.6;
Add a generic set of routines for interacting with OpenFirmware-style
string lists.

remove unused "_DIAGNOSTIC" option and opt_diagnostic.h.
note that this is unrelated to the widely used "DIAGNOSTIC" option.

branches: 1.140.2;
Slightly reorder for clarity, and add header.

branches: 1.139.2;
Fix KASAN build on aarch64

Add KASAN instrumentation on strcat/strchr/strrchr.

Remove never-used Mersenne twister from libkern.

#ifdef notyet ---> never

Ah, strcat/strchr/strrchr are ASM functions, so instrument them.

Add support for Kernel Memory Sanitizer (kMSan). It detects uninitialized
memory used by the kernel at run time, and just like kASan and kCSan, it
is an excellent feature. It has already detected 38 uninitialized variables
in the kernel during my testing, which I have since discreetly fixed.

We use two shadows:
- "shad", to track uninitialized memory with a bit granularity (1:1).
Each bit set to 1 in the shad corresponds to one uninitialized bit of
real kernel memory.
- "orig", to track the origin of the memory with a 4-byte granularity
(1:1). Each uint32_t cell in the orig indicates the origin of the
associated uint32_t of real kernel memory.

The memory consumption of these shadows is consequent, so at least 4GB of
RAM is recommended to run kMSan.

The compiler inserts calls to specific __msan_* functions on each memory
access, to manage both the shad and the orig and detect uninitialized
memory accesses that change the execution flow (like an "if" on an
uninitialized variable).

We mark as uninit several types of memory buffers (stack, pools, kmem,
malloc, uvm_km), and check each buffer passed to copyout, copyoutstr,
bwrite, if_transmit_lock and DMA operations, to detect uninitialized memory
that leaves the system. This allows us to detect kernel info leaks in a way
that is more efficient and also more user-friendly than KLEAK.

Contrary to kASan, kMSan requires comprehensive coverage, ie we cannot
tolerate having one non-instrumented function, because this could cause
false positives. kMSan cannot instrument ASM functions, so I converted
most of them to __asm__ inlines, which kMSan is able to instrument. Those
that remain receive special treatment.

Contrary to kASan again, kMSan uses a TLS, so we must context-switch this
TLS during interrupts. We use different contexts depending on the interrupt
level.

The orig tracks precisely the origin of a buffer. We use a special encoding
for the orig values, and pack together in each uint32_t cell of the orig:
- a code designating the type of memory (Stack, Pool, etc), and
- a compressed pointer, which points either (1) to a string containing
the name of the variable associated with the cell, or (2) to an area
in the kernel .text section which we resolve to a symbol name + offset.

This encoding allows us not to consume extra memory for associating
information with each cell, and produces a precise output, that can tell
for example the name of an uninitialized variable on the stack, the
function in which it was pushed on the stack, and the function where we
accessed this uninitialized variable.

kMSan is available with LLVM, but not with GCC.

The code is organized in a way that is similar to kASan and kCSan, so it
means that other architectures than amd64 can be supported.

Add Kernel Concurrency Sanitizer (kCSan) support. This sanitizer allows us
to detect race conditions at runtime. It is a variation of TSan that is
easy to implement and more suited to kernel internals, albeit theoretically
less precise than TSan's happens-before.

We do basically two things:

- On every KCSAN_NACCESSES (=2000) memory accesses, we create a cell
describing the access, and delay the calling CPU (10ms).

- On all memory accesses, we verify if the memory we're reading/writing
is referenced in a cell already.

The combination of the two means that, if for example cpu0 does a read that
is selected and cpu1 does a write at the same address, kCSan will fire,
because cpu1's write collides with cpu0's read cell.

The coverage of the instrumentation is the same as that of kASan. Also, the
code is organized in a way similar to kASan, so it is easy to add support
for more architectures than amd64. kCSan is compatible with KCOV.

Reviewed by Kamil.

Add ifdefs to eliminate false positives on lgtm, same as coverity.

Add KASAN instrumentation for memmove.

Rename min/max -> uimin/uimax for better honesty.

These functions are defined on unsigned int. The generic name
min/max should not silently truncate to 32 bits on 64-bit systems.
This is purely a name change -- no functional change intended.

HOWEVER! Some subsystems have

#define min(a, b) ((a) < (b) ? (a) : (b))
#define max(a, b) ((a) > (b) ? (a) : (b))

even though our standard name for that is MIN/MAX. Although these
may invite multiple evaluation bugs, these do _not_ cause integer
truncation.

To avoid `fixing' these cases, I first changed the name in libkern,
and then compile-tested every file where min/max occurred in order to
confirm that it failed -- and thus confirm that nothing shadowed
min/max -- before changing it.

I have left a handful of bootloaders that are too annoying to
compile-test, and some dead code:

cobalt ews4800mips hp300 hppa ia64 luna68k vax
acorn32/if_ie.c (not included in any kernels)
macppc/if_gm.c (superseded by gem(4))

It should be easy to fix the fallout once identified -- this way of
doing things fails safe, and the goal here, after all, is to _avoid_
silent integer truncations, not introduce them.

Maybe one day we can reintroduce min/max as type-generic things that
never silently truncate. But we should avoid doing that for a while,
so that existing code has a chance to be detected by the compiler for
conversion to uimin/uimax without changing the semantics until we can
properly audit it all. (Who knows, maybe in some cases integer
truncation is actually intended!)

Add kasan interceptors for strcpy/strcmp/strlen.

Add support for kASan on amd64. Written by me, with some parts inspired
from Siddharth Muralee's initial work. This feature can detect several
kinds of memory bugs, and it's an excellent feature.

It can be enabled by uncommenting these three lines in GENERIC:

#makeoptions KASAN=1 # Kernel Address Sanitizer
#options KASAN
#no options SVS

The kernel is compiled without SVS, without DMAP and without PCPU area.
A shadow area is created at boot time, and it can cover the upper 128TB
of the address space. This area is populated gradually as we allocate
memory. With this design the memory consumption is kept at its lowest
level.

The compiler calls the __asan_* functions each time a memory access is
done. We verify whether this access is legal by looking at the shadow
area.

We declare our own special memcpy/memset/etc functions, because the
compiler's builtins don't add the __asan_* instrumentation.

Initially all the mappings are marked as valid. During dynamic
allocations, we add a redzone, which we mark as invalid. Any access on
it will trigger a kASan error message. Additionally, the compiler adds
a redzone on global variables, and we mark these redzones as invalid too.
The illegal-access detection works with a 1-byte granularity.

For now, we cover three areas:

- global variables
- kmem_alloc-ated areas
- malloc-ated areas

More will come, but that's a good start.

provide memmem

branches: 1.126.2; 1.126.4;
Even smaller and takes print function.

coalesce the two copies of hexdump into libkern

KNF. Remove extra spaces. No functional change.