Home | History | Annotate | Download | only in gen
History log of /src/lib/libc/gen/arc4random.c
RevisionDateAuthorComments
 1.50  11-Mar-2025  riastradh Assign static MIB numbers for kern.entropy.epoch.

This sidesteps any need for dynamically sized memory in userland to
resolve sysctl names to read it out, or for a new syscall interface
to sysctl resolution by name.

It would really be better to expose this through a page shared with
userland, so querying it doesn't cost a syscall, but this will serve
for now.

PR lib/59148: arc4random calls malloc so it can't be used in an ELF
constructor
 1.49  11-Mar-2025  riastradh arc4random(3): Isolate endian-dependence of crypto_onetimestream.

Make it all happen in crypto_le32enc/dec (which in this file is
actually just memcpy). This way you can swap out crypto_le32enc/dec
to test either byte order on any machine, e.g. for generating the
test vectors. And if we ever decide that it's not worthwhile to take
advantage of the purely nondeterministic nature of the API to make
this have potentially less performance penalty on big-endian systems
(which I'm not sure has ever been measured anyway), it'll all
continue to work under

#define crypto_le32enc le32enc
#define crypto_le32dec le32dec

with all the _BYTE_ORDER == _BIG_ENDIAN conditionals removed from the
self-tests.

No change to the code of crypto_onetimstream on hppa under tools gcc.
 1.48  11-Mar-2025  riastradh arc4random(4): Fix crypto_onetimestream_selftest on big-endian.

Had computed the nonce increment the wrong way around when simulating
a big-endian machine to generate this.
 1.47  10-Mar-2025  riastradh arc4random(3): Add self-tests for PRNG algorithm.

Independently generated by running:

head -c 64 </dev/zero \
| openssl enc -chacha -20 \
-K 000102030405060708090a0b0c0d0e0f101112131415161718191a1b1c1d1e1f \
-iv -iv 00000000000000000000000000000000 \
| hexdump -C

and then repeating with the first 32 bytes of output as the updated
key for -K, extracting bytes [32, 32 + n) for the n-byte output in
each call.
 1.46  09-Mar-2025  riastradh arc4random(3): Provide a fallback in case pthread_atfork fails.

This is considerably more work and burden on testing than simply
statically preallocating a bit of storage for pthread_atfork to
eliminate the failure mode altogether, but it is less work than
arguing further over the atfork interface:
https://mail-index.NetBSD.org/source-changes-d/2025/03/02/msg014387.html

PR lib/59117: arc4random has some failure modes it shouldn't
 1.45  06-Mar-2025  riastradh t_arc4random: Test arc4random_global.per_thread, not .initialized.

If arc4random_initialize has been called, and thr_keycreate failed,
then .initialized will be true but .per_thread will be false -- and
.thread_key will be garbage (some other thread key for another
purpose, most likely). This path was enabled by allowing
thr_keycreate to fail instead of aborting the process.

This hasn't caused trouble yet, mainly because we don't do anything
to inject faults into thr_keycreate in these tests. Tweak the
global_threadkeylimit test while here to provoke a crash with the
wrong conditional.

Fix a similar edge case in the little test program embedded in
arc4random.c (which should maybe just go away now that we have atf
tests).

PR lib/59117: arc4random has some failure modes it shouldn't
 1.44  06-Mar-2025  riastradh arc4random(3): Switch to use thr_once (libc pthread_once symbol).

This way, we reduce the problem of arc4random initialization
fork-safety to the problem of pthread_once fork-safety -- and
anything else to do with one-time lazy initialization.

PR lib/59124: arc4random(3): first call in process races with
concurrent fork
 1.43  04-Mar-2025  riastradh Revert __libc_atfork addition.

This reverts the following revisions:

lib/libc/gen/arc4random.c 1.40-41 (but not 1.42 which is independent)
lib/libc/gen/pthread_atfork.c 1.24
lib/libc/include/atfork.h 1.1

This additionally updates the comments in arc4random.c to reflect the
current state.

Requested by kre:
https://mail-index.netbsd.org/source-changes-d/2025/03/03/msg014388.html

Since the new symbol __libc_atfork has not been used outside libc,
this poses no ABI compatibility issues.
 1.42  02-Mar-2025  riastradh arc4random(3): Don't ifdef out the self-test.

The runtime cost of this test is negligible (64 bytes of data and a
handful of instructions for the subroutine) and the security value of
verifying a known input/output for the deterministic cryptographic
function on whose security the whole API relies is high.
 1.41  02-Mar-2025  riastradh arc4random(3): Update comments to reflect removal of failure modes.

PR lib/59117: arc4random has some failure modes it shouldn't
 1.40  02-Mar-2025  riastradh libc: New __libc_atfork.

This uses caller-provided storage for the callback queues.

Use it in arc4random(3) in order to avoid possible failure modes.

This is a private symbol, not designed for use outside NetBSD, and
the API is not intended to be stable (yet) -- I just took the
existing purely internal structure (struct atfork_callback) and
reused it for this API without changing any of the calling-side
logic. We could change it, e.g. to use a single structure per call,
to make the API a little less unwieldy, at the cost of
microscopically more storage and runtime for the users that don't use
all three callbacks; to be considered in a future change.

We might reasonably use __libc_atfork in libpthread for use in the
pthread_tsd_init constructor, in order to be confident it never
attempts malloc(3), but let's do that in a separate commit just in
case anything goes awry with that plan.

PR lib/59112: libpthread constructors use malloc
PR lib/59117: arc4random has some failure modes it shouldn't
 1.39  02-Mar-2025  riastradh arc4random(3): Avoid failure due to thread key limits.

If thr_keycreate (a.k.a. pthread_key_create) fails, fall back to
using globally serialized state instead of per-thread state. This is
unlikely to happen but arc4random(3) should work even if it does.
New test case forces exercising this path (at least, simulating the
effect of key creation failure).

PR lib/59117: arc4random has some failure modes it shouldn't
 1.38  29-Aug-2024  riastradh arc4random(3): Pacify some of lint's complaints.

PR kern/58632: getentropy(2) and arc4random(3) do not reseed on VM
fork
 1.37  27-Aug-2024  riastradh arc4random(3): Add automatic tests.

This verifies that:
- arc4random zeroes its state and reseeds itself on fork
- arc4random reseeds itself on entropy consolidation (e.g., VM clone)
- arc4random falls back to global state if it can't allocate local
state because address space limits cause mmap to fail

NOTE: This adds a new libc symbol __arc4random_global, but it's in
the reserved namespace and only used by t_arc4random, so no libc
minor bump.

PR kern/58632: getentropy(2) and arc4random(3) do not reseed on VM
fork
 1.36  26-Aug-2024  riastradh arc4random.c: Fix test program.

This isn't wired up anywhere, but let's reduce the bitrot. It was
helpful in reminding me that kern.entropy.epoch was, for reasons I
can't remember, restricted to privileged access.

PR kern/58632: getentropy(2) and arc4random(3) do not reseed on VM
fork
 1.35  26-Aug-2024  riastradh arc4random(3): Reseed if system entropy epoch changes.

This can happen, for example, if the system is a VM instance, and the
VM is cloned.

This incurs the cost of a system call on every arc4random call, which
is unfortunate, but

1. we don't currently have a (machine-independent) mechanism for
exposing a read-only page to userland shared by the kernel to
enable a cheaper access path to the entropy epoch; and

2. the algorithm here -- a simple application of ChaCha -- is likely
also a bottleneck and could be much cheaper by

(a) using sys/crypto/chacha for machine-dependent vectorized
ChaCha code, and

(b) filling a buffer (somewhere between a cipher block and a page)
in a batch at a time, instead of running ChaCha to generate
only 32 bytes at a time.

So although this might be a performance hit, the security benefit is
worthwhile and we have a clear path to do better than reversing the
performance hit later.

PR kern/58632: getentropy(2) and arc4random(3) do not reseed on VM
fork
 1.34  20-Jan-2024  christos branches: 1.34.2;
Catch up with all the lint warnings since exit on warning was disabled.
Disable 'missing header declaration' and 'nested extern' warnings for now.
 1.33  19-Apr-2022  rillig branches: 1.33.2;
lib: remove CONSTCOND comment

Since 2021-01-31, lint doesn't need it anymore for the common pattern of
'do ... while (0)'.
 1.32  23-Sep-2019  christos lint is not smart enough to figure out that ilog2() is constant.
 1.31  25-Mar-2016  riastradh branches: 1.31.16;
KNF
 1.30  13-May-2015  justin Missing MAP_PRIVATE on mmap
 1.29  19-Mar-2015  riastradh Remove #ifdef MAP_INHERIT_ZERO.

This is essential for fork-safety, so don't merely #warn about it.
Attaining fork-safety without it requires restructuring things -- in
particular, there's no clear way to make it per-thread and fork-safe
without some global list of states to zero on fork.
 1.28  21-Jan-2015  riastradh Use ChaCha20 here as advertised, not ChaCha8.

Oops.

Fortunately, there is no public cryptanalysis even of ChaCha8: the
best published attack is on ChaCha7 with time complexity 2^248.
 1.27  20-Jan-2015  christos Fix non _REENTRANT build.
 1.26  16-Nov-2014  riastradh Rewrite arc4random(3) with ChaCha20-based PRNG and per-thread state.

Explain the security model in the man page.

No more RC4!

XXX pullup to netbsd-6, netbsd-5
 1.25  19-Jul-2014  roy branches: 1.25.2;
Document that our use of pthread_mutex_unlock(3) is async-signal-safe
and as such can be used in a phtread_atfork(3) child handler.
 1.24  12-Jun-2014  apb fix missing backslash in previous
 1.23  12-Jun-2014  apb Wrap complex macros in do { ... } while (0). Also replace the magic
number 1600000 with a macro.
 1.22  07-Jun-2014  roy Re-stir after forking, fixes PR lib/25367.
Re-stir after consuming 1600000 bytes, fixes PR lib/45952.
 1.21  17-Oct-2013  christos branches: 1.21.2;
remove always inline because new gcc bitches.
 1.20  20-Aug-2012  dsl branches: 1.20.2;
I'm fairly sure the libc rules require that arc4random_addrandom,
arc4random_buf, arc4random_stir and arc4random_uniform be weak.
 1.19  20-Aug-2012  dsl arc4 is a random number sequence, there is no point using its own
output values to determine a number of output values to skip.
Skipping values on any possibly random event might be worth while, as
might using the keying algorithm to stir in a possibly random value.
 1.18  20-Aug-2012  dsl Since 'rs' is statically initialised (not bss) its s[] might as well be
compile-time initialised as well.
arc4_init) is the same as arc4_stir().
Initialise rs.i to 0xff (not 0) so that the first key byte is processed
with rs.i == 0 without the splurios rs.i-- in arc4_addrandom().
Remove the assignment rs.j = rs.i at the end of arc4_addrandom(), it isn't
necessary and I can see no justificationm for it.
Replace RSIZE with __arraycount(as->s), however it is manifestly 256 (more
correctly the number of values in rs.i and rs.j).
 1.17  18-Aug-2012  dsl cache rs.i and rs.j in arc4random_buf() since the compiler can't be
told that buf[] dosn't alias them.
Reduces the number of instructions inside the loop.
All the other functions are ok.
 1.16  18-Aug-2012  dsl Change the logic used for thread locking to make the code more readable.
The effect on the object code is to replace a function call with a
branch that will predict correctly in the non-threaded case.
Minor optimise of some other paths.
 1.15  18-Aug-2012  dsl Use an inline function to check for initialisation, and an non-inlined
one to do the actual initialise.
Fixes lib/46751 by removing all the replicated tests in each function.
 1.14  29-Jul-2012  dsl Make this compile with the compiler I'm using.
Move variable defs to top of function.
Don't use const static mib[] - run time initialisation won't matter,
and not using static data may actually help in a .so.
 1.13  05-Mar-2012  christos misc cleanups:
- const for mibs
- #define for magic constants
- casts
 1.12  04-Mar-2012  tls Fix bug in previous: don't reinitialize on every call! Fix arc4random_buf so it actually ever initializes -- a security problem with revisions 1.9 and 1.10.
 1.11  27-Feb-2012  tls Make arc4random far less greedy for entropy. Make arc4random actually
implement arc4 when used by threaded programs.
 1.10  04-Feb-2011  christos branches: 1.10.4; 1.10.6;
add arc4random_{buf,uniform}, from OpenBSD.
 1.9  24-Dec-2005  perry branches: 1.9.40;
Remove leading __ from __(const|inline|signed|volatile) -- it is obsolete.
 1.8  12-Jun-2005  lukem Add missing __RCSID()
 1.7  09-Feb-2005  kleink A little libc namespace housekeeping exercise:
* Make vfprintf_unlocked() an internal function, c.f. __svfscanf_unlocked().
* Add internal names for arc4random(), endnetpath(), fhstatvfs(),
fstatvfs(), mkstemp(), shquote(), statvfs(), taddr2uaddr(), uaddr2taddr(),
uuid_create_nil(), uuid_is_nil(), and wcwidth().
* Include namespace.h where supposed to.
 1.6  09-Feb-2005  kleink Declare rs_initialized static.
 1.5  11-Nov-2002  thorpej branches: 1.5.2;
Fix signed/unsigned comparison warnings.
 1.4  02-Jul-2002  itojun use sysctl(kern.urandom) if /dev/urandom is not present (like chroot jail)
 1.3  14-Jun-2002  itojun branches: 1.3.2;
discard first 256 words when we stir.
 1.2  24-May-2002  itojun /dev/arandom does not exist in netbsd. use /dev/urandom.
 1.1  24-May-2002  itojun add arc4random(3). from openbsd
 1.3.2.4  11-Nov-2002  nathanw Catch up to -current
 1.3.2.3  01-Aug-2002  nathanw Catch up to -current.
 1.3.2.2  21-Jun-2002  nathanw Catch up to -current.
 1.3.2.1  14-Jun-2002  nathanw file arc4random.c was added on branch nathanw_sa on 2002-06-21 18:18:07 +0000
 1.5.2.2  26-Mar-2004  jmc Pullup rev 1.1-1.5 (requested by provos in ticket #1386)

Introduce bcrypt password scheme. Adds the arc4random API for creating
cryptographically strong random numbers.
 1.5.2.1  11-Nov-2002  jmc file arc4random.c was added on branch netbsd-1-6 on 2004-03-26 22:52:50 +0000
 1.9.40.1  08-Feb-2011  bouyer Sync with HEAD
 1.10.6.1  08-Mar-2012  riz Pull up following revision(s) (requested by tls in ticket #92):
lib/libc/gen/arc4random.c: revision 1.11
lib/libc/gen/arc4random.c: revision 1.12
Make arc4random far less greedy for entropy. Make arc4random actually
implement arc4 when used by threaded programs.
Fix bug in previous: don't reinitialize on every call! Fix arc4random_buf so it actually ever initializes -- a security problem with revisions 1.9 and 1.10.
 1.10.4.3  22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.10.4.2  30-Oct-2012  yamt sync with head
 1.10.4.1  17-Apr-2012  yamt sync with head
 1.20.2.1  20-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.21.2.1  10-Aug-2014  tls Rebase.
 1.25.2.1  18-Mar-2015  snj Pull up following revision(s) (requested by riastradh in ticket #597):
lib/libc/gen/arc4random.c: revisions 1.26-1.28
lib/libc/gen/arc4random.3:: revisions 1.10-1.19
Rewrite arc4random(3) with ChaCha20-based PRNG and per-thread state.
Explain the security model in the man page.
No more RC4!
--
Grammar.
--
Note relation of arc4random(3) to rand(3)/random(3).
--
Ruminate on security model choices and API design in arc4random(3).
--
Amplify comment about how quickly RC4 was known to be bad.
--
Markup for BUGS note about arc4random_uniform.
--
Cross-reference rnd(4).
--
Remove unnecessary Ns before punctuation. Fix a line.
--
Fix Google Groups link.
--
Tweak wording, define `output', remove misplaced scaremongering.
--
Fix non _REENTRANT build.
--
Use ChaCha20 here as advertised, not ChaCha8.
Oops.
Fortunately, there is no public cryptanalysis even of ChaCha8: the
best published attack is on ChaCha7 with time complexity 2^248.
 1.31.16.1  13-Apr-2020  martin Mostly merge changes from HEAD upto 20200411
 1.33.2.1  09-Oct-2024  martin Pull up following revision(s) (requested by riastradh in ticket #939):

distrib/sets/lists/debug/module.md.amd64: revision 1.18
sys/modules/Makefile: revision 1.292
lib/libc/gen/arc4random.c: revision 1.34
lib/libc/gen/arc4random.c: revision 1.35
lib/libc/gen/arc4random.c: revision 1.36
lib/libc/gen/arc4random.c: revision 1.37
sys/kern/kern_entropy.c: revision 1.70
lib/libc/gen/arc4random.c: revision 1.38
sys/kern/kern_entropy.c: revision 1.71
lib/libc/gen/getentropy.3: revision 1.8
distrib/sets/lists/modules/md.amd64: revision 1.103
share/man/man4/rnd.4: revision 1.42
share/man/man4/rnd.4: revision 1.44
lib/libc/include/arc4random.h: revision 1.1
distrib/sets/lists/man/mi: revision 1.1786
sys/arch/i386/conf/GENERIC: revision 1.1258
sys/modules/acpivmgenid/acpivmgenid.ioconf: revision 1.1
sys/arch/amd64/conf/ALL: revision 1.190
distrib/sets/lists/debug/mi: revision 1.446
sys/arch/i386/conf/ALL: revision 1.521
lib/libc/gen/Makefile.inc: revision 1.219
distrib/sets/lists/debug/module.md.i386: revision 1.12
sys/dev/acpi/acpi_vmgenid.c: revision 1.1
sys/dev/acpi/acpi_vmgenid.c: revision 1.2
lib/libc/include/reentrant.h: revision 1.22
sys/arch/evbarm/conf/GENERIC64: revision 1.219
share/man/man4/Makefile: revision 1.735
distrib/sets/lists/modules/md.i386: revision 1.100
distrib/sets/lists/tests/mi: revision 1.1334
lib/libc/gen/arc4random.3: revision 1.22
sys/dev/acpi/files.acpi: revision 1.133
lib/libc/gen/arc4random.3: revision 1.23
tests/lib/libc/gen/t_arc4random.c: revision 1.1
sys/sys/entropy.h: revision 1.6
sys/arch/amd64/conf/GENERIC: revision 1.614
sys/modules/acpivmgenid/Makefile: revision 1.1
share/man/man4/acpivmgenid.4: revision 1.1
lib/libc/gen/Makefile.inc: revision 1.220
tests/lib/libc/gen/Makefile: revision 1.56
share/man/man4/acpivmgenid.4: revision 1.2
share/man/man4/acpivmgenid.4: revision 1.3

(all via patch)

Catch up with all the lint warnings since exit on warning was disabled.

Disable 'missing header declaration' and 'nested extern' warnings for now.
acpivmgenid(4): New driver for virtual machine generation ID.

Added to amd64/ALL and i386/ALL kernel configurations, and made
available as a loadable module acpivmgenid.kmod on x86, for now.
TBD: Add to all ACPI-supporting GENERIC kernels.
PR kern/58632: getentropy(2) and arc4random(3) do not reseed on VM
fork

entropy(9): Factor out subroutines to reset and gather entropy.
`Reset' means we keep the data in the pool, but assume it had zero
entropy. `Gather' means we request samples from all on-demand
sources and wait for the synchronous ones to complete.

No functional change intended, other than to expose new symbols --
just preparation to expose these to acpivmgenid(4), so it can use
these when the VM host notifies us that we, the guest, have been
cloned.
PR kern/58632: getentropy(2) and arc4random(3) do not reseed on VM
fork

acpivmgenid(4): Reset and gather entropy on VM clone notification.
PR kern/58632: getentropy(2) and arc4random(3) do not reseed on VM
fork

arc4random(3): Reseed if system entropy epoch changes.
This can happen, for example, if the system is a VM instance, and the
VM is cloned.

This incurs the cost of a system call on every arc4random call, which
is unfortunate, but
1. we don't currently have a (machine-independent) mechanism for
exposing a read-only page to userland shared by the kernel to
enable a cheaper access path to the entropy epoch; and
2. the algorithm here -- a simple application of ChaCha -- is likely
also a bottleneck and could be much cheaper by
(a) using sys/crypto/chacha for machine-dependent vectorized
ChaCha code, and
(b) filling a buffer (somewhere between a cipher block and a page)
in a batch at a time, instead of running ChaCha to generate
only 32 bytes at a time.
So although this might be a performance hit, the security benefit is
worthwhile and we have a clear path to do better than reversing the
performance hit later.
PR kern/58632: getentropy(2) and arc4random(3) do not reseed on VM
fork

acpivmgenid(4): Nix BUGS that have been squashed.
Reference kern.entropy.epoch for the remaining bug (which is a
performance issue, not a security issue).
PR kern/58632: getentropy(2) and arc4random(3) do not reseed on VM
fork

entropy(9): Allow unprivileged reads of sysctl kern.entropy.epoch.

Applications need this in order to know when to reseed. (We should
also expose it through a page shared read-only with userland for
cheaper access, but until we do, let's let applications get at it
through sysctl.)
PR kern/58632: getentropy(2) and arc4random(3) do not reseed on VM
fork

arc4random.c: Fix test program.

This isn't wired up anywhere, but let's reduce the bitrot. It was
helpful in reminding me that kern.entropy.epoch was, for reasons I
can't remember, restricted to privileged access.
PR kern/58632: getentropy(2) and arc4random(3) do not reseed on VM
fork

amd64, evbarm, i386: Add acpivmgenid(4) to GENERIC.
PR kern/58632: getentropy(2) and arc4random(3) do not reseed on VM
fork

rnd(4): Document kern.entropy.epoch is unprivileged and elaborate.
Cross-reference acpivmgenid(4).
PR kern/58632: getentropy(2) and arc4random(3) do not reseed on VM
fork

arc4random(3): Note that arc4random respects kern.entropy.epoch.
PR kern/58632: getentropy(2) and arc4random(3) do not reseed on VM
fork
Add debug info for new acpivmgenid module

arc4random(3): Add automatic tests.

This verifies that:
- arc4random zeroes its state and reseeds itself on fork
- arc4random reseeds itself on entropy consolidation (e.g., VM clone)
- arc4random falls back to global state if it can't allocate local
state because address space limits cause mmap to fail

NOTE: This adds a new libc symbol __arc4random_global, but it's in
the reserved namespace and only used by t_arc4random, so no libc
minor bump.
PR kern/58632: getentropy(2) and arc4random(3) do not reseed on VM
fork

getentropy(3): Note intent to reseed on VM clone, and caveats.

Tidy markup and pacify some mandoc -Tlint complaints while here.
PR kern/58632: getentropy(2) and arc4random(3) do not reseed on VM
fork

Bump dates on man pages recently updated to mention VM clones.
PR kern/58632: getentropy(2) and arc4random(3) do not reseed on VM
fork

arc4random(3): Pacify some of lint's complaints.
PR kern/58632: getentropy(2) and arc4random(3) do not reseed on VM
fork

arc4random: suppress another lint warning
 1.34.2.1  02-Aug-2025  perseant Sync with HEAD

RSS XML Feed