Home | History | Annotate | only in /src/sys/opencrypto
History log of /src/sys/opencrypto
RevisionDateAuthorComments
 1.3 11-Dec-2005  christos merge ktrace-lwp.
 1.2 06-Sep-2003  cjep branches: 1.2.4;
comment typo
 1.1 25-Jul-2003  jonathan Install cryptodev.h into /usr/include/crypto/ for userspace applications.
 1.2.4.4 21-Sep-2004  skrll Fix the sync with head I botched.
 1.2.4.3 18-Sep-2004  skrll Sync with HEAD.
 1.2.4.2 03-Aug-2004  skrll Sync with HEAD
 1.2.4.1 06-Sep-2003  skrll file Makefile was added on branch ktrace-lwp on 2004-08-03 10:56:25 +0000
 1.3 29-Jun-2020  riastradh opencrypto: Switch from legacy rijndael API to new aes API.

While here, apply various rijndael->aes renames, reduce the size
of aesxcbc_ctx by 480 bytes, and convert some malloc->kmem.

Leave in the symbol enc_xform_rijndael128 for now, though, so this
doesn't break any kernel ABI.
 1.2 26-Sep-2016  christos From Alexander Nasonov:
- Make constants static: Shrinks code and data size.
- Avoid overflow in limit calculation.
- Use uint8_t instead of u_char to match types
While here:
- Remove unnecessary casts
- s/u_int8_t/uint8_t/g
 1.1 24-May-2011  drochner branches: 1.1.2; 1.1.6; 1.1.18; 1.1.36; 1.1.40;
copy AES-XCBC-MAC support from KAME IPSEC to FAST_IPSEC
For this to fit, an API change in cryptosoft was adopted from OpenBSD
(addition of a "Setkey" method to hashes) which was done for GCM/GMAC
support there, so it might be useful in the future anyway.
tested against KAME IPSEC
AFAICT, FAST_IPSEC now supports as much as KAME.
 1.1.40.1 04-Nov-2016  pgoyette Sync with HEAD
 1.1.36.1 05-Oct-2016  skrll Sync with HEAD
 1.1.18.1 03-Dec-2017  jdolecek update from HEAD
 1.1.6.2 06-Jun-2011  jruoho Sync with HEAD.
 1.1.6.1 24-May-2011  jruoho file aesxcbcmac.c was added on branch jruoho-x86intr on 2011-06-06 09:10:03 +0000
 1.1.2.2 31-May-2011  rmind sync with head
 1.1.2.1 24-May-2011  rmind file aesxcbcmac.c was added on branch rmind-uvmplock on 2011-05-31 03:05:10 +0000
 1.2 29-Jun-2020  riastradh opencrypto: Switch from legacy rijndael API to new aes API.

While here, apply various rijndael->aes renames, reduce the size
of aesxcbc_ctx by 480 bytes, and convert some malloc->kmem.

Leave in the symbol enc_xform_rijndael128 for now, though, so this
doesn't break any kernel ABI.
 1.1 24-May-2011  drochner branches: 1.1.2; 1.1.6;
copy AES-XCBC-MAC support from KAME IPSEC to FAST_IPSEC
For this to fit, an API change in cryptosoft was adopted from OpenBSD
(addition of a "Setkey" method to hashes) which was done for GCM/GMAC
support there, so it might be useful in the future anyway.
tested against KAME IPSEC
AFAICT, FAST_IPSEC now supports as much as KAME.
 1.1.6.2 06-Jun-2011  jruoho Sync with HEAD.
 1.1.6.1 24-May-2011  jruoho file aesxcbcmac.h was added on branch jruoho-x86intr on 2011-06-06 09:10:03 +0000
 1.1.2.2 31-May-2011  rmind sync with head
 1.1.2.1 24-May-2011  rmind file aesxcbcmac.h was added on branch rmind-uvmplock on 2011-05-31 03:05:10 +0000
 1.3 27-Aug-2003  thorpej Use the Blowfish in crypto/blowfish, which has hooks for using tuned
assembly for the transform.
 1.2 26-Aug-2003  thorpej In Blowfish_initstate(), make the initstate static, otherwise the
compiler will emit code to first copy it onto the stack before
copying it into the destination context structure. With this change,
it will only be copied once.
 1.1 25-Jul-2003  jonathan Commit initial NetBSD port of the OpenCrypto Framework (OCF). This
code is derived from Sam Leffler's FreeBSD port of OCF, which is in
turn a port of Angelos Keromytis's OpenBSD work.
Credit to Sam and Angelos, any blame for the NetBSD port to me.
 1.2 27-Aug-2003  thorpej Use the Blowfish in crypto/blowfish, which has hooks for using tuned
assembly for the transform.
 1.1 25-Jul-2003  jonathan Commit initial NetBSD port of the OpenCrypto Framework (OCF). This
code is derived from Sam Leffler's FreeBSD port of OCF, which is in
turn a port of Angelos Keromytis's OpenBSD work.
Credit to Sam and Angelos, any blame for the NetBSD port to me.
 1.2 26-Aug-2003  thorpej Move the opencrypto CAST-128 implementation to crypto/cast128, removing
the old one. Rename the functions/structures from cast_* to cast128_*.
Adapt the KAME IPsec to use the new CAST-128 code, which has a simpler
API and smaller footprint.
 1.1 25-Jul-2003  jonathan Commit initial NetBSD port of the OpenCrypto Framework (OCF). This
code is derived from Sam Leffler's FreeBSD port of OCF, which is in
turn a port of Angelos Keromytis's OpenBSD work.
Credit to Sam and Angelos, any blame for the NetBSD port to me.
 1.2 26-Aug-2003  thorpej Move the opencrypto CAST-128 implementation to crypto/cast128, removing
the old one. Rename the functions/structures from cast_* to cast128_*.
Adapt the KAME IPsec to use the new CAST-128 code, which has a simpler
API and smaller footprint.
 1.1 25-Jul-2003  jonathan Commit initial NetBSD port of the OpenCrypto Framework (OCF). This
code is derived from Sam Leffler's FreeBSD port of OCF, which is in
turn a port of Angelos Keromytis's OpenBSD work.
Credit to Sam and Angelos, any blame for the NetBSD port to me.
 1.2 26-Aug-2003  thorpej Move the opencrypto CAST-128 implementation to crypto/cast128, removing
the old one. Rename the functions/structures from cast_* to cast128_*.
Adapt the KAME IPsec to use the new CAST-128 code, which has a simpler
API and smaller footprint.
 1.1 25-Jul-2003  jonathan Commit initial NetBSD port of the OpenCrypto Framework (OCF). This
code is derived from Sam Leffler's FreeBSD port of OCF, which is in
turn a port of Angelos Keromytis's OpenBSD work.
Credit to Sam and Angelos, any blame for the NetBSD port to me.
 1.2 27-Jan-2020  pgoyette branches: 1.2.2; 1.2.6;
Remove left-over #includes
 1.1 27-Jan-2020  pgoyette Split the module glue out from the rest of opencrypto/ocryptodev to
make rump happy.

Rump doesn't have compat modules (the compat code is included in the
relevant librump*.so), so there's no module compat_50 listed in
link_set_modules, and thus ocryptodev's MODULE(...) can't "require"
it.

This fixes the problem of "built-in module compat_50 not found" when
starting up rump_allserver (or rump_server with -l rumpdev_opencrypto).

XXX This does not resolve the long-standing "crypto: unable to
XXX register devsw, error 17" message noted at line 78 of
XXX sys/rump/dev/lib/libopencrypto/opencrypto_component.c
 1.2.6.2 08-Apr-2020  martin Merge changes from current as of 20200406
 1.2.6.1 27-Jan-2020  martin file compat_crypto_50.c was added on branch phil-wifi on 2020-04-08 14:08:59 +0000
 1.2.2.2 29-Feb-2020  ad Sync with head.
 1.2.2.1 27-Jan-2020  ad file compat_crypto_50.c was added on branch ad-namecache on 2020-02-29 20:21:08 +0000
 1.9 03-Sep-2018  riastradh Rename min/max -> uimin/uimax for better honesty.

These functions are defined on unsigned int. The generic name
min/max should not silently truncate to 32 bits on 64-bit systems.
This is purely a name change -- no functional change intended.

HOWEVER! Some subsystems have

#define min(a, b) ((a) < (b) ? (a) : (b))
#define max(a, b) ((a) > (b) ? (a) : (b))

even though our standard name for that is MIN/MAX. Although these
may invite multiple evaluation bugs, these do _not_ cause integer
truncation.

To avoid `fixing' these cases, I first changed the name in libkern,
and then compile-tested every file where min/max occurred in order to
confirm that it failed -- and thus confirm that nothing shadowed
min/max -- before changing it.

I have left a handful of bootloaders that are too annoying to
compile-test, and some dead code:

cobalt ews4800mips hp300 hppa ia64 luna68k vax
acorn32/if_ie.c (not included in any kernels)
macppc/if_gm.c (superseded by gem(4))

It should be easy to fix the fallout once identified -- this way of
doing things fails safe, and the goal here, after all, is to _avoid_
silent integer truncations, not introduce them.

Maybe one day we can reintroduce min/max as type-generic things that
never silently truncate. But we should avoid doing that for a while,
so that existing code has a chance to be detected by the compiler for
conversion to uimin/uimax without changing the semantics until we can
properly audit it all. (Who knows, maybe in some cases integer
truncation is actually intended!)
 1.8 24-Feb-2011  drochner branches: 1.8.54; 1.8.56;
Don't panic, just truncate, if the iov is too short in a COPYBACK.
This case can be triggered from userland cryptodev if the buffer
for decompressed data is too small.
(It would look cleaner if the lengths would be passed explicitely
everywhere, but that would thwart the abstraction done by COPYDATA/COPYBACK
which allows to treat mbufs and iovs the same way.)
 1.7 14-Mar-2009  dsl branches: 1.7.4; 1.7.6; 1.7.8;
ANSIfy another 1261 function definitions.
The only ones left in sys are beyond by sed script!
(or in sys/dist or sys/external)
Mostly they have function pointer parameters.
 1.6 01-Feb-2008  tls branches: 1.6.10; 1.6.18; 1.6.24;
This code never worked on a released version of FreeBSD in the form it's
been in in our tree, and certainly does not work on any version of FreeBSD
now. Run through unifdef -D__NetBSD__ -U__FreeBSD__ yielding a small
reduction of size and a dramatic improvement in readability.

No, this does not yield any meaningful decrease in patchability (unlike
mechanical changes that touch live source lines) -- try it and see.
 1.5 04-Mar-2007  christos branches: 1.5.16; 1.5.22;
Kill caddr_t; there will be some MI fallout, but it will be fixed shortly.
 1.4 11-Dec-2005  christos branches: 1.4.26;
merge ktrace-lwp.
 1.3 26-Feb-2005  perry branches: 1.3.4;
nuke trailing whitespace
 1.2 30-Jul-2003  lha branches: 1.2.4; 1.2.10; 1.2.12;
make the panic messages match reality
 1.1 25-Jul-2003  jonathan Commit initial NetBSD port of the OpenCrypto Framework (OCF). This
code is derived from Sam Leffler's FreeBSD port of OCF, which is in
turn a port of Angelos Keromytis's OpenBSD work.
Credit to Sam and Angelos, any blame for the NetBSD port to me.
 1.2.12.1 19-Mar-2005  yamt sync with head. xen and whitespace. xen part is not finished.
 1.2.10.1 29-Apr-2005  kent sync with -current
 1.2.4.5 04-Mar-2005  skrll Sync with HEAD.

Hi Perry!
 1.2.4.4 21-Sep-2004  skrll Fix the sync with head I botched.
 1.2.4.3 18-Sep-2004  skrll Sync with HEAD.
 1.2.4.2 03-Aug-2004  skrll Sync with HEAD
 1.2.4.1 30-Jul-2003  skrll file criov.c was added on branch ktrace-lwp on 2004-08-03 10:56:25 +0000
 1.3.4.2 04-Feb-2008  yamt sync with head.
 1.3.4.1 03-Sep-2007  yamt sync with head.
 1.4.26.1 12-Mar-2007  rmind Sync with HEAD.
 1.5.22.1 18-Feb-2008  mjf Sync with HEAD.
 1.5.16.1 23-Mar-2008  matt sync with HEAD
 1.6.24.1 13-May-2009  jym Sync with HEAD.

Commit is split, to avoid a "too many arguments" protocol error.
 1.6.18.1 28-Apr-2009  skrll Sync with HEAD.
 1.6.10.1 04-May-2009  yamt sync with head.
 1.7.8.1 05-Mar-2011  bouyer Sync with HEAD
 1.7.6.1 06-Jun-2011  jruoho Sync with HEAD.
 1.7.4.1 05-Mar-2011  rmind sync with head
 1.8.56.1 10-Jun-2019  christos Sync with HEAD
 1.8.54.1 06-Sep-2018  pgoyette Sync with HEAD

Resolve a couple of conflicts (result of the uimin/uimax changes)
 1.131 26-Jun-2022  riastradh opencrypto(9): Fix missing initialization in error branch.

Reported-by: syzbot+8c519140cac567be1ee1@syzkaller.appspotmail.com
 1.130 22-May-2022  riastradh opencrypto: Assert session id is valid in crypto_freesession.

This gives us the opportunity to detect usage mistakes like
use-after-free.

Exception: Continue to silently ignore sid=0.
 1.129 22-May-2022  riastradh opencrypto: crypto_dispatch never fails now. Make it return void.

Same with crypto_kdispatch.
 1.128 22-May-2022  riastradh opencrypto: Assert driver process routine returns 0 or ERESTART.

No other errors are allowed -- other errors must be transmitted by
crypto_done. All drivers in tree (sun8i_crypto, glxsb, via_padlock,
mvcesa, mvxpsec, hifn, qat, ubsec, cryptosoft) have been audited for
this.
 1.127 22-May-2022  riastradh opencrypto: Rip out EAGAIN logic when unregistering crypto drivers.

I'm pretty sure this never worked reliably based on code inspection,
and it's unlikely to have ever been tested because it only applies
when unregistering a driver -- but we have no crypto drivers for
removable devices, so it would only apply if we went out of our way
to trigger detach with drvctl.

Instead, just make the operation fail with ENODEV, and remove all the
callback logic to resubmit the request on EAGAIN. (Maybe this should
be ENXIO, but crypto_kdispatch already does ENODEV.)
 1.126 22-May-2022  riastradh opencrypto: Assert nonnull callback up front in crypto_dispatch.

Same with crypto_kdispatch.

Convert some dead branches downstream to assertions too.
 1.125 22-May-2022  riastradh opencrypto: Make crypto_freesession return void.

No callers use the return value. It is not sensible to allow this to
fail.
 1.124 22-May-2022  riastradh opencrypto: Make freesession callback return void.

No functional change intended: all drivers already return zero
unconditionally.
 1.123 22-May-2022  riastradh opencrypto: Assert crp_desc and crp_buf are nonnull.

- crypto_getreq ensures crp_desc is nonnull.
- Caller is responsible for setting crp_buf.
 1.122 22-May-2022  riastradh opencrypto: Assert num>0 in crypto_getreq, num=1 in crypto_kgetreq.

- For crypto_getreq this makes downstream reasoning easier: on
success, crp_desc is guaranteed to be nonnull.

- For crypto_kgetreq, this was already assumed, just silently
ignored and not checked by anything.
 1.121 22-May-2022  riastradh opencrypto: Nix CRYPTO_F_DONE.

Nothing uses it any more.
 1.120 22-May-2022  riastradh opencrypto: Make sid=0 always invalid, but OK to free.

Previously, crypto_newsession could sometimes return 0 as the
driver-specific part of the session id, and 0 as the hid, for sid=0.
But netipsec assumes that it is always safe to free sid=0 from
zero-initialized memory even if crypto_newsession has never
succeeded. So it was up to every driver in tree to gracefully handle
sid=0, if it happened to get assigned hid=0. And, as long as the
freesession callback was expected to just return an error code when
given a bogus session id, that worked out fine...because nothing ever
used the error code.

That was a terrible fragile system that should never have been
invented. Instead, let's just ensure that valid session ids are
nonzero, and make crypto_freesession with sid=0 be a no-op.
 1.119 19-May-2022  riastradh opencrypto: Assert !cpu_intr_p() on dispatch and invoke.

These should only ever have been potentially called from hard
interrupt context by CRYPTO_F_CBIMM callbacks (CBIMM = call back
immediately). CRYPTO_F_CBIMM is no more, so there is no more need to
allow this case of call from hard interrupt context.
 1.118 19-May-2022  riastradh opencrypto: Nix CRYPTO_F_USER, CRYPTO_F_CBIMM, CRYPTO_F_CBIFSYNC.

CRYPTO_F_USER is no longer needed. It was introduced in 2008 by
darran@ in crypto.c 1.30, cryptodev.c 1.45 in an attempt to avoid
double-free between the issuing thread and asynchronous callback.
But the `fix' didn't work. In 2017, knakahara@ fixed it properly in
cryptodev.c 1.87 by distinguishing `the crypto operation has
completed' (CRYPTO_F_DONE) from `the callback is done touching the
crp object' (CRYPTO_F_DQRETQ, now renamed to CRYPTODEV_F_RET).

CRYPTO_F_CBIMM formerly served to invoke the callback synchronously
from the driver's interrupt completion routine, to reduce contention
on what was once a single cryptoret thread. Now, there is a per-CPU
queue and softint for much cheaper processing, so there is less
motivation for this in the first place. So let's remove the
complicated logic. This means the callbacks never run in hard
interrupt context, which means we don't need to worry about recursion
into crypto_dispatch in hard interrupt context.
 1.117 17-May-2022  riastradh opencrypto: Factor setting CRYPTO_F_DONE out of branches.

This had been done in 1.30 when the locking was different. No need
any more. No functional change intended.
 1.116 14-Aug-2021  andvar fix typo in CRK_ALGORITHM_MIN definition to match CRK_ALGORITHM_MAX one.
while here fix few typos in comments.
 1.115 09-Aug-2021  andvar fix typos in asymmetry, asymmetric(al), symmetrical.
 1.114 08-Apr-2020  pgoyette Revert previous change to use SYSCTL_SETUP since it breaks on macppc.

For some reason, the crypto module fails to link, and this results in
opencrypto sysctl failures.

Should resolve PR kern/55154
 1.113 16-Mar-2020  pgoyette Use the module subsystem's ability to process SYSCTL_SETUP() entries to
automate installation of sysctl nodes.

Note that there are still a number of device and pseudo-device modules
that create entries tied to individual device units, rather than to the
module itself. These are not changed.
 1.112 01-Feb-2020  riastradh softint_disestablish does xc_barrier(0) for us already.
 1.111 01-Feb-2020  riastradh Switch opencrypto to percpu_create.

Can't sleep for allocation in percpu_foreach.
 1.110 06-Oct-2019  uwe branches: 1.110.2;
xc_barrier - convenience function to xc_broadcast() a nop.

Make the intent more clear and also avoid a bunch of (xcfunc_t)nullop
casts that gcc 8 -Wcast-function-type is not happy about.
 1.109 01-Oct-2019  chs in many device attach paths, allocate memory with KM_SLEEP instead of KM_NOSLEEP
and remove code to handle failures that can no longer happen.
 1.108 11-Jul-2019  christos relinguish our lock while we are autoloading.
 1.107 13-Jun-2019  christos Try to load swcrypto if we we did not find any software drivers.
 1.106 06-Jun-2018  maya branches: 1.106.2;
Remove duplicate ;
 1.105 08-Jan-2018  knakahara branches: 1.105.2;
Fix PR kern/52910. Reported and implemented a patch by Sevan Janiyan, thanks.
 1.104 08-Jan-2018  knakahara Committed debugging logs by mistake, sorry. Revert cryoto.c:r.1.103 and ip6_flow.c:r.1.37.
 1.103 08-Jan-2018  knakahara Fix PR kern/52910. Reported and implemented a patch by Sevan Janiyan, thanks.
 1.102 09-Nov-2017  christos use PR_NOWAIT.
 1.101 22-Sep-2017  knakahara fix opencrypto(9) part of PR kern/52515

percpu data use pointers to TAILQ instead of TAILQ itself.
 1.100 31-Jul-2017  knakahara use pool_cache(9) instead of pool(9) as they can be called concurrently.
 1.99 31-Jul-2017  knakahara use kmem_alloc KPI instead of malloc KPI.
 1.98 31-Jul-2017  knakahara pack crypto_drivers variables to struct and add cacheline_aligned qualifier.
 1.97 31-Jul-2017  knakahara refactor: remove glue macros for FreeBSD code.
 1.96 26-Jul-2017  knakahara divide crp_ret_{,k}q by CPU to avoid reordering.

update locking note later.
 1.95 26-Jul-2017  knakahara make crp_{,k}q percpu to scale crypto_dispatch().

update locking note later.
 1.94 20-Jul-2017  knakahara fix typo
 1.93 20-Jul-2017  knakahara fix panic when using ecryption devices attached earlier than ipi_sysinit().

pointed out and tested by martin@n.o, thanks.
 1.92 18-Jul-2017  knakahara branches: 1.92.2;
make cryptoret() context softint to balance dequeuing crypto_ret_q with enqueuing it.
 1.91 26-Jun-2017  knakahara simplify mutex_enter/exit(crypto_q_mtx), and fix missing exit.
 1.90 15-Jun-2017  knakahara Reduce crypto_ret_q_mtx lock regions.

crypto.c does not access the members of crp when the crp is in crp_q or
crp_ret_q. Furthermore, crp_q and crp_ret_q are protected by each mutex,
so the members of crp is not shared. That means crp_flags is not required
mutex in crypto.c.
 1.89 14-Jun-2017  knakahara support multiple encryption drivers (port from FreeBSD).
 1.88 14-Jun-2017  knakahara refactor crypto_newsession() like FreeBSD.
 1.87 14-Jun-2017  knakahara must release cap->cc_lock before calling cap->cc_newsession() because of spinlock.
 1.86 08-Jun-2017  christos Put back crypto_checkdriver(); use it when we need to make sure that we
get back a cryptocap that has been initialized.
 1.85 06-Jun-2017  christos - acquire lock
- use c99 loop indexes
- initialize featp
 1.84 06-Jun-2017  knakahara apply the same fix as crypto.c:r1.83 for crypto_dispatch to crypto_kdispatch.
 1.83 06-Jun-2017  knakahara avoid crp_q reordering as hardware interrupts.

crypto_{,k}invoke() can be called with holding crp_q_mtx now.
 1.82 06-Jun-2017  knakahara restructure locks(2/2): crypto_q_mtx can be adaptive now.
 1.81 06-Jun-2017  knakahara restructure locks(1/2): make relation between lock and data explicit.

+ crypto_drv_mtx protects
- whole crypto_drivers
+ crypto_drivers[i].cc_lock (new) protects
- crypto_drivers[i] itself
- member of crypto_drivers[i]
+ crypto_q_mtx protects
- crp_q
- crp_kq
+ crypto_ret_q_mtx protects
- crp_ret_q
- crp_ret_kq
- crypto_exit_flag

I will add locking note later.
 1.80 05-Jun-2017  knakahara fix reading crp_q without holding crypto_q_mtx
 1.79 05-Jun-2017  knakahara use crypto_checkdriver_uninit() when it may touch uninitialized crypto_drivers.
 1.78 31-May-2017  knakahara branches: 1.78.2;
fix: crypto_unregister didn't work.
 1.77 29-May-2017  knakahara strictly use crypto_checkdriver(i) instead of using crypto_drivers[i] directly.
 1.76 25-May-2017  knakahara add cryptkop alloc/free KPI instead of manipulating cryptkop_pool directly.
 1.75 24-May-2017  knakahara make the default values of q_maxlen build parameters.
 1.74 24-May-2017  knakahara implement crypto_ret_q limitation. original code is implemented by hsuenaga@IIJ.
 1.73 24-May-2017  knakahara add crypto_ret_{,k}q length sysctl entries and statistics codes.
 1.72 24-May-2017  knakahara initialize sysctl in the same way regardless of module or not.
 1.71 17-May-2017  knakahara decrease the priority of batch crp even if there are more than one batch crp.
 1.70 17-May-2017  knakahara refactor cryptointr(), no functional changes.
 1.69 17-May-2017  knakahara fix cryptointr() can process unexpected request.

If migrate crp is linked after batch crp, "submit" is already set to
the batch crp. So, cryptointr() can process the batch crp instead of
the target migrate crp.
 1.68 17-May-2017  knakahara refactor crypto_kdispatch() in a similar way as crypto_dispatch().
 1.67 17-May-2017  knakahara refactor crypto_dispatch (3/3): do "blocked" operation previously
 1.66 17-May-2017  knakahara refactor crypto_dispatch (2/3): divide migrate operation
 1.65 17-May-2017  knakahara refactor crypto_dispatch (1/3): divide batch operation

Processing batch operation at first, crypto_q_mtx's lock region is reduced
because crp does not require crypto_q_mtx.
 1.64 17-May-2017  knakahara opencrypto: cleanup debug messages.
 1.63 10-May-2017  knakahara fix: crypto_drivers[hid].cc_process() could be called even if it was null.

If a crypto driver is unregistered before calling cyrptointr(),
the crypto_drivers[hid].cc_process is null in spite of the hid
is less than crypto_drivers_num.

reffered to FreeBSD code.
 1.62 10-May-2017  knakahara unify implementation of crypto_unregister() and crypto_unregister_all()
 1.61 10-May-2017  knakahara refactor crypto_unregister()

- separate logic to crypto_unregister_locked()
- refactor cryptocap cleanup condition
 1.60 10-May-2017  knakahara use macro instead of immediate value
 1.59 02-May-2017  knakahara add some assertion. tested by ATF net/ipsec/ and crypto/.
 1.58 26-Apr-2017  knakahara branches: 1.58.2;
When crypto request is deferred processing, opencrypto should not return error.

When the crypto device blocks a crypto request, opnecrypto enqueues the request,
that is, the request is just deferred and no error occurs.
The pseudo error causes problems, e.g. ipsec can send wrong ICMP host unreach.

contributed by hsuenaga@IIJ, thanks.
 1.57 24-Apr-2017  knakahara separate crypto_drv_mtx from crypto_mtx.

crypto_mtx is used only for cryptodev.c and ocryptodev.c now.
 1.56 24-Apr-2017  knakahara reduce crypto_q_mtx lock regions.
 1.55 24-Apr-2017  knakahara refactor crypto_unblock(). No functional change.
 1.54 07-Apr-2017  knakahara the processing said "ghastly hacks" is unnecessary now.
 1.53 06-Apr-2017  knakahara fix build failure ALL
 1.52 05-Apr-2017  knakahara fix processes accessing /dev/crypto stall when over three processes run with a hardware encryption driver

The process has stalled at cv_wait(&crp->crp_cv) because cryptodev_cb()
is not called as cryptoret() kthread keep waiting at cv_wait(&cryptoret_cv).
Previous opencrypto implementation assumes the thread from cryptodev.c
does all processing in the same context, so skips enqueueing and sending
cryptoret_cv. However, the context can be switched, e.g. when we use
a hardware encryption driver.

And add debug messages.
 1.51 29-Mar-2017  knakahara fix missing mutex_exit() in crypto_destroy().

crypto_destroy() is called only in error case or unloading module.
 1.50 16-Mar-2017  knakahara fix: remove unmatched mutex_exit/enter. must be forgetting to remove at crypto.c:r1.41
 1.49 09-Feb-2017  knakahara make some locks and condvar static
 1.48 07-Jul-2016  msaitoh branches: 1.48.2; 1.48.4;
KNF. Remove extra spaces. No functional change.
 1.47 28-Nov-2015  christos fix the build
 1.46 28-Nov-2015  pgoyette Re-work the module init and destroy code to allow it to be unloaded and
then reloaded.

Should fix PR kern/49842
 1.45 25-Feb-2014  pooka branches: 1.45.6;
Ensure that the top level sysctl nodes (kern, vfs, net, ...) exist before
the sysctl link sets are processed, and remove redundancy.

Shaves >13kB off of an amd64 GENERIC, not to mention >1k duplicate
lines of code.
 1.44 14-Jan-2014  pgoyette Clean-up module initialization
 1.43 13-Jan-2014  pgoyette When using modular opencrypto, make sure the sysctl variables get
created.
 1.42 01-Jan-2014  pgoyette Modularize the opencrypto components and link to the build
 1.41 09-Jun-2011  drochner branches: 1.41.2; 1.41.12; 1.41.16;
-if an opencrypto(9) session is allocated, the driver is refcounted
and can not disappear -- no need to hold crypto_mtx to check the
driver list
(the whole check is questionable)
-crp->crp_cv (the condition variable) is used by userland cryptodev
exclusively -- move its initialization there, no need to waste
cycles of in-kernel callers
-add a comment which members of "struct cryptop" are used
by opencrypto(9) and which by crypto(4)
(this should be split, no need to waste memory for in-kernel callers)
 1.40 16-May-2011  drochner branches: 1.40.2;
split the "crypto_mtx" spinlock into 3: one spinlock each for
the incoming and outgoing request queues (which can be dealt with
by hardware accelerators) and an adaptive lock for "all the rest"
(mostly driver configuration, but also some unrelated stuff in
cryptodev.c which should be revisited)
The latter one seems to be uneeded at many places, but for now I've
done simple replacements only, except minor fixes (where
softint_schedule() was called without the lock held)
 1.39 06-May-2011  drochner As a first step towards more fine-grained locking, don't require
crypto_{new.free}session() to be called with the "crypto_mtx"
spinlock held.
This doesn't change much for now because these functions acquire
the said mutex first on entry now, but at least it keeps the nasty
locks local to the opencrypto core.
 1.38 24-Feb-2011  drochner make the crypto softint MPSAFE -- I see no reason not to do it, and
it didn't cause trouble for me. (It doesn't give additional parallelization
as things look now, just saves the acquisition of KERNEL_LOCK.)
 1.37 26-Jan-2011  christos PR/44470: Dr. Wolfgang Stukenbrock: opencrypto kernel implementation may pass
outdated argument to worker
 1.36 11-Aug-2010  pgoyette branches: 1.36.2; 1.36.4;
Keep condvar wmesg within 8 char limit.
 1.35 02-Aug-2010  jakllsch Consistently use a single CRYPTO_SESID2HID-like macro.
Improve CRYPTO_DEBUG printing a bit:
print pointers with %p
print unsigned with %u rather than %d
use CRYPTO_SESID2LID instead of just casting to uint32_t
 1.34 18-Apr-2009  tsutsui branches: 1.34.2; 1.34.4;
Remove extra whitespace added by a stupid tool.
XXX: more in src/sys/arch
 1.33 25-Mar-2009  darran Fixes PR kern/41069 and PR kern/41070.

Extends the Opencrypto API to allow the destination buffer size to be
specified when its not the same size as the input buffer (i.e. for
operations like compress and decompress).
The crypto_op and crypt_n_op structures gain a u_int dst_len field.
The session_op structure gains a comp_alg field to specify a compression
algorithm.
Moved four ioctls to new ids; CIOCGSESSION, CIOCNGSESSION, CIOCCRYPT,
and CIOCNCRYPTM.
Added four backward compatible ioctls; OCIOCGSESSION, OCIOCNGSESSION,
OCIOCCRYPT, and OCIOCNCRYPTM.

Backward compatibility is maintained in ocryptodev.h and ocryptodev.c which
implement the original ioctls and set dst_len and comp_alg to 0.

Adds user-space access to compression features.

Adds software gzip support (CRYPTO_GZIP_COMP).

Adds the fast version of crc32 from zlib to libkern. This should be generally
useful and provide a place to start normalizing the various crc32 routines
in the kernel. The crc32 routine is used in this patch to support GZIP.

With input and support from tls@NetBSD.org.
 1.32 18-Mar-2009  cegger bcopy -> memcpy
 1.31 18-Mar-2009  cegger bzero -> memset
 1.30 18-Nov-2008  darran branches: 1.30.4;
Fix a race condition in opencrypto where the crypto request could be
completed by the crypto device, queued on the retq, but freed by the
ioctl lwp. The problem manifests as various panics relating to the
condvar inside the request. The problem can occur whenever the crypto
device completes the request immediately and the ioctl skips the cv_wait().

The problem can be reproduced by enabling cryptosoft and running an openssl
speed test. E.g.
sysctl -w kern.cryptodevallowsoft=-1
openssl speed -engine cryptodev -evp des-ede3-cbc -multi 64

Add a macro for TAILQ_FOREACH_REVERSE_SAFE() to queue.h, since this
was missing and the opencrypto code removes requests from a list while
iterating with TAILQ_FOREACH_REVERSE().

Add missing cv_destroy() calls for the key request cleanup.

Reviewed by Thor Lancelot Simon.
 1.29 03-Aug-2008  degroote branches: 1.29.2; 1.29.4;
In crypto_freereq, destroy explicitly the condvar

ok by tls@
 1.28 28-Apr-2008  martin branches: 1.28.2; 1.28.6;
Remove clause 3 and 4 from TNF licenses
 1.27 10-Apr-2008  tls branches: 1.27.2; 1.27.4;
Extend crypto.4 interface:

* Asynchronous operation with result retrieval via select/poll
* Mutliple-request submit/retrieve ioctls
* Mutliple-session create-destroy ioctls

Revise/rewrite crypto.4 manual page. It should now be much easier to write
new applications to this API.

Measured performance for trivial requests: 84,000 very short modular math
operations/sec, 120,000 very short md5 hashes per sec (with a hardware
accellerator of moderate performance but very low latency, whose driver
will be contributed at a later date).

Contributed to TNF by Coyote Point Systems, Inc.
 1.26 05-Feb-2008  ad branches: 1.26.6;
cryptoret: avoid a sleep/wakeup race.
 1.25 05-Feb-2008  tls The kthread is now MPSAFE.
 1.24 04-Feb-2008  tls Some locking fixes (double-release mutex in softintr wakeup case, which I
hadn't tested) and an uninitialized field in cse which Darran Hunt
found. Some more debugging printfs.

Turn on MPSAFE for the kthread. We're not sure it's safe for the softint
yet. Gives a little performance kick for swcrypto with many requests on
MP systems.
 1.23 04-Feb-2008  tls Rework opencrypto to use a spin mutex (crypto_mtx) instead of "splcrypto"
(actually splnet) and condvars instead of tsleep/wakeup. Fix a few
miscellaneous problems and add some debugging printfs while there.

Restore set of CRYPTO_F_DONE in crypto_done() which was lost at some
point after this code came from FreeBSD -- it made it impossible to wait
properly for a condition.

Add flags analogous to the "crp" flags to the key operation's krp struct.
Add a new flag, CRYPTO_F_ONRETQ which tells us a request finished before
the kthread had a chance to dequeue it and call its callback -- this was
letting requests stick on the queues before even though done and copied
out.

Callers of crypto_newsession() or crypto_freesession() must now take the
mutex. Change netipsec to do so. Dispatch takes the mutex itself as
needed.

This was tested fairly extensively with the cryptosoft backend and lightly
with a new hardware driver. It has not been tested with FAST_IPSEC; I am
unable to ascertain whether FAST_IPSEC currently works at all in our tree.

pjd@FreeBSD.ORG, ad@NetBSD.ORG, and darran@snark.us pointed me in the
right direction several times in the course of this. Remaining bugs
are mine alone.
 1.22 01-Feb-2008  tls This code never worked on a released version of FreeBSD in the form it's
been in in our tree, and certainly does not work on any version of FreeBSD
now. Run through unifdef -D__NetBSD__ -U__FreeBSD__ yielding a small
reduction of size and a dramatic improvement in readability.

No, this does not yield any meaningful decrease in patchability (unlike
mechanical changes that touch live source lines) -- try it and see.
 1.21 08-Oct-2007  ad branches: 1.21.4;
Use the softint API.
 1.20 09-Jul-2007  ad branches: 1.20.6; 1.20.8; 1.20.10;
Merge some of the less invasive changes from the vmlocking branch:

- kthread, callout, devsw API changes
- select()/poll() improvements
- miscellaneous MT safety improvements
 1.19 12-Mar-2007  ad branches: 1.19.2;
Pass an ipl argument to pool_init/POOL_INIT to be used when initializing
the pool's lock.
 1.18 12-Jan-2007  daniel branches: 1.18.2; 1.18.6;
crypto_init does not call crypto_init0 only once, because the marker
created with ONCE_DECL() is local. This results in reinitializing
the driver list when crypto_get_driverid() (and leaks memory). Fix
this by making the marker static.

Fixes PR/35412.

Ack freza@.
 1.17 16-Nov-2006  christos branches: 1.17.2;
__unused removal on arguments; approved by core.
 1.16 20-Oct-2006  mrg avoid yet another GCC uninitialised warning error that only comes
up with -O3.
 1.15 12-Oct-2006  christos - sprinkle __unused on function decls.
- fix a couple of unused bugs
- no more -Wno-unused for i386
 1.14 07-Jun-2006  kardel branches: 1.14.6; 1.14.8;
merge FreeBSD timecounters from branch simonb-timecounters
- struct timeval time is gone
time.tv_sec -> time_second
- struct timeval mono_time is gone
mono_time.tv_sec -> time_uptime
- access to time via
{get,}{micro,nano,bin}time()
get* versions are fast but less precise
- support NTP nanokernel implementation (NTP API 4)
- further reading:
Timecounter Paper: http://phk.freebsd.dk/pubs/timecounter.pdf
NTP Nanokernel: http://www.eecis.udel.edu/~mills/ntp/html/kern.html
 1.13 06-Mar-2006  christos branches: 1.13.6;
Add the 3 missing sysctl we are supposed to export.
 1.12 16-Jan-2006  yamt branches: 1.12.2; 1.12.4; 1.12.6;
- tweak RUN_ONCE api to allow init_func returns an error.
- physio: handle failure of workqueue_create.
 1.11 25-Nov-2005  thorpej branches: 1.11.2;
- De-couple the software crypto implementation from the rest of the
framework. There is no need to waste the space if you are only using
algoritms provided by hardware accelerators. To get the software
implementations, add "pseudo-device swcr" to your kernel config.
- Lazily initialize the opencrypto framework when crypto drivers
(either hardware or swcr) register themselves with the framework.
 1.10 26-Feb-2005  perry branches: 1.10.4; 1.10.10;
nuke trailing whitespace
 1.9 29-Apr-2004  jonathan branches: 1.9.2; 1.9.6; 1.9.8;
Change control knob for /dev/crypto to only allow requests which
will be hardware-accelerated. Avoids copyin()/copyout() overhead and
spending exceessive tie inside the kernel.

Pullup after: 24 hours, or confirmation by Jason Thorpe that this is the
consensus tech-kern agreed upon last summer.
 1.8 31-Dec-2003  jonathan branches: 1.8.2;
Split opencrypto configuration into an attribute, usable by inkernel
clients, and a pseudo-device for userspace access.

The attribute is named `opencrypto'. The pseudo-device is renamed to
"crypto", which has a dependency on "opencrypto". The sys/conf/majors
entry and pseudo-device attach entrypoint are updated to match the
new pseudo-device name.

Fast IPsec (sys/netipsec/files.ipsec) now lists a dependency on the
"opencrypto" attribute. Drivers for crypto accelerators (ubsec,
hifn775x) also pull in opencrypto, as providers of opencrypto transforms.
 1.7 19-Nov-2003  jonathan Fix typo.
 1.6 19-Nov-2003  jonathan Clean up userlevel access to software kernel transforms, in preparation
for using /dev/crypto for OpenSSL:

1. Add comments explaining crypto_devallowsoft, explaining the
OpenBSD-style three-way logic actully implemented in crypto_newsession().

2. Pass crypto_devallowsoft as the final argument to crypto_newsession(),
instead of a constant 0 value.

3. Set the default value of crypto_devallowsoft to 1, to allow
/dev/crypto access only for hardware-supported transforms.

Items 1-3 may be revised to match the FreeBSD two-way logic, if the
consensus is that there's no point to forcing software transforms.
But as a first step, let the description match what the code actually does.

GC unused variables usercrypto, userasmcrypto, cryptodevallowsoft from
cryptodev.c, in favour of variables crypto_usercrypto, crypto_userasmcrypto,
crypto_devallowsoft, which are used as well as defined in crypto.c.
 1.5 09-Nov-2003  scw Fix a genuine uninitialised variable.
 1.4 21-Sep-2003  lha Implement nanouptime as a function to avoid 'dereferencing type-punned pointer'.
No token after #endif
 1.3 31-Jul-2003  jonathan Remove bogus include which snuck into previous commit (from comparison
to Quentin Garnier's suggested patch).
 1.2 30-Jul-2003  jonathan Move the initialization of the crypto framework from the userland
pseudo-device to init_main(), so the framework is ready for
registration requests at autoconfiguration time.

Thanks to Quentin Garnier for confirming the change was required, and
for testing a similar fix.
 1.1 25-Jul-2003  jonathan Commit initial NetBSD port of the OpenCrypto Framework (OCF). This
code is derived from Sam Leffler's FreeBSD port of OCF, which is in
turn a port of Angelos Keromytis's OpenBSD work.
Credit to Sam and Angelos, any blame for the NetBSD port to me.
 1.8.2.1 30-Apr-2004  jmc Pullup rev 1.9 (requested by jonathan in ticket #234)

Change control knob for /dev/crypto to only allow requests which
will be hardware-accelerated. Avoids copyin()/copyout() overhead and
spending exceessive tie inside the kernel.
 1.9.8.1 19-Mar-2005  yamt sync with head. xen and whitespace. xen part is not finished.
 1.9.6.1 29-Apr-2005  kent sync with -current
 1.9.2.6 11-Dec-2005  christos Sync with head.
 1.9.2.5 04-Mar-2005  skrll Sync with HEAD.

Hi Perry!
 1.9.2.4 21-Sep-2004  skrll Fix the sync with head I botched.
 1.9.2.3 18-Sep-2004  skrll Sync with HEAD.
 1.9.2.2 03-Aug-2004  skrll Sync with HEAD
 1.9.2.1 29-Apr-2004  skrll file crypto.c was added on branch ktrace-lwp on 2004-08-03 10:56:25 +0000
 1.10.10.1 29-Nov-2005  yamt sync with head.
 1.10.4.7 11-Feb-2008  yamt sync with head.
 1.10.4.6 04-Feb-2008  yamt sync with head.
 1.10.4.5 27-Oct-2007  yamt sync with head.
 1.10.4.4 03-Sep-2007  yamt sync with head.
 1.10.4.3 26-Feb-2007  yamt sync with head.
 1.10.4.2 30-Dec-2006  yamt sync with head.
 1.10.4.1 21-Jun-2006  yamt sync with head.
 1.11.2.1 01-Feb-2006  yamt sync with head.
 1.12.6.2 26-Jun-2006  yamt sync with head.
 1.12.6.1 13-Mar-2006  yamt sync with head.
 1.12.4.2 22-Apr-2006  simonb Sync with head.
 1.12.4.1 04-Feb-2006  simonb We have nanouptime() now.
 1.12.2.1 09-Sep-2006  rpaulo sync with head
 1.13.6.1 19-Jun-2006  chap Sync with head.
 1.14.8.2 10-Dec-2006  yamt sync with head.
 1.14.8.1 22-Oct-2006  yamt sync with head
 1.14.6.2 01-Feb-2007  ad Sync with head.
 1.14.6.1 18-Nov-2006  ad Sync with head.
 1.17.2.1 20-Jan-2007  bouyer Pull up following revision(s) (requested by daniel in ticket #367):
sys/opencrypto/crypto.c: revision 1.18
Make the opencrypto framework work with more than one driver again.
Fixes PR/35412.
 1.18.6.5 17-Jun-2007  ad - Increase the number of thread priorities from 128 to 256. How the space
is set up is to be revisited.
- Implement soft interrupts as kernel threads. A generic implementation
is provided, with hooks for fast-path MD code that can run the interrupt
threads over the top of other threads executing in the kernel.
- Split vnode::v_flag into three fields, depending on how the flag is
locked (by the interlock, by the vnode lock, by the file system).
- Miscellaneous locking fixes and improvements.
 1.18.6.4 13-May-2007  ad - Pass the error number and residual count to biodone(), and let it handle
setting error indicators. Prepare to eliminate B_ERROR.
- Add a flag argument to brelse() to be set into the buf's flags, instead
of doing it directly. Typically used to set B_INVAL.
- Add a "struct cpu_info *" argument to kthread_create(), to be used to
create bound threads. Change "bool mpsafe" to "int flags".
- Allow exit of LWPs in the IDL state when (l != curlwp).
- More locking fixes & conversion to the new API.
 1.18.6.3 10-Apr-2007  ad Nuke the deferred kthread creation stuff, as it's no longer needed.
Pointed out by thorpej@.
 1.18.6.2 09-Apr-2007  ad - Add two new arguments to kthread_create1: pri_t pri, bool mpsafe.
- Fork kthreads off proc0 as new LWPs, not new processes.
 1.18.6.1 13-Mar-2007  ad Sync with head.
 1.18.2.1 24-Mar-2007  yamt sync with head.
 1.19.2.1 11-Jul-2007  mjf Sync with head.
 1.20.10.1 14-Oct-2007  yamt sync with head.
 1.20.8.2 23-Mar-2008  matt sync with HEAD
 1.20.8.1 06-Nov-2007  matt sync with HEAD
 1.20.6.1 26-Oct-2007  joerg Sync with HEAD.

Follow the merge of pmap.c on i386 and amd64 and move
pmap_init_tmp_pgtbl into arch/x86/x86/pmap.c. Modify the ACPI wakeup
code to restore CR4 before jumping back into kernel space as the large
page option might cover that.
 1.21.4.1 18-Feb-2008  mjf Sync with HEAD.
 1.26.6.3 17-Jan-2009  mjf Sync with HEAD.
 1.26.6.2 28-Sep-2008  mjf Sync with HEAD.
 1.26.6.1 02-Jun-2008  mjf Sync with HEAD.
 1.27.4.4 09-Oct-2010  yamt sync with head
 1.27.4.3 11-Aug-2010  yamt sync with head.
 1.27.4.2 04-May-2009  yamt sync with head.
 1.27.4.1 16-May-2008  yamt sync with head.
 1.27.2.1 18-May-2008  yamt sync with head.
 1.28.6.2 13-Dec-2008  haad Update haad-dm branch to haad-dm-base2.
 1.28.6.1 19-Oct-2008  haad Sync with HEAD.
 1.28.2.1 18-Sep-2008  wrstuden Sync with wrstuden-revivesa-base-2.
 1.29.4.2 03-May-2009  snj Pull up following revision(s) (requested by tls in ticket #611):
sys/lib/libkern/Makefile: patch
sys/lib/libkern/crc32.c: revision 1.1
sys/lib/libkern/crc32.h: revision 1.1
sys/lib/libkern/libkern.h: revision 1.89
sys/lib/libkern/arch/i386/Makefile.inc: revision 1.28
sys/net/zlib.h: revision 1.14 via patch
sys/opencrypto/crypto.c: revision 1.33
sys/opencrypto/cryptodev.c: revision 1.46
sys/opencrypto/cryptodev.h: revision 1.16
sys/opencrypto/cryptosoft.c: revision 1.24
sys/opencrypto/cryptosoft.h: revision 1.6
sys/opencrypto/deflate.h: revision 1.6
sys/opencrypto/cryptosoft_xform.c: revision 1.12
sys/opencrypto/deflate.c: revision 1.13
sys/opencrypto/files.opencrypto: revision 1.20
sys/opencrypto/ocryptodev.c: revision 1.1
sys/opencrypto/ocryptodev.h: revision 1.1
sys/opencrypto/xform.c: revision 1.18
sys/opencrypto/xform.h: revision 1.10
Fixes PR kern/41069 and PR kern/41070.

Extends the Opencrypto API to allow the destination buffer size to be
specified when its not the same size as the input buffer (i.e. for
operations like compress and decompress).
The crypto_op and crypt_n_op structures gain a u_int dst_len field.
The session_op structure gains a comp_alg field to specify a compression
algorithm.
Moved four ioctls to new ids; CIOCGSESSION, CIOCNGSESSION, CIOCCRYPT,
and CIOCNCRYPTM.
Added four backward compatible ioctls; OCIOCGSESSION, OCIOCNGSESSION,
OCIOCCRYPT, and OCIOCNCRYPTM.

Backward compatibility is maintained in ocryptodev.h and ocryptodev.c which
implement the original ioctls and set dst_len and comp_alg to 0.

Adds user-space access to compression features.

Adds software gzip support (CRYPTO_GZIP_COMP).

Adds the fast version of crc32 from zlib to libkern. This should be generally
useful and provide a place to start normalizing the various crc32 routines
in the kernel. The crc32 routine is used in this patch to support GZIP.

With input and support from tls@NetBSD.org.
 1.29.4.1 20-Nov-2008  snj branches: 1.29.4.1.4;
Pull up following revision(s) (requested by darran in ticket #92):
sys/opencrypto/cryptodev.c: revision 1.45
sys/sys/queue.h: revision 1.50
sys/opencrypto/cryptodev.h: revision 1.15
sys/opencrypto/crypto.c: revision 1.30
Fix a race condition in opencrypto where the crypto request could be
completed by the crypto device, queued on the retq, but freed by the
ioctl lwp. The problem manifests as various panics relating to the
condvar inside the request. The problem can occur whenever the crypto
device completes the request immediately and the ioctl skips the
cv_wait().
The problem can be reproduced by enabling cryptosoft and running an
openssl
speed test. E.g.
sysctl -w kern.cryptodevallowsoft=-1
openssl speed -engine cryptodev -evp des-ede3-cbc -multi 64
Add a macro for TAILQ_FOREACH_REVERSE_SAFE() to queue.h, since this
was missing and the opencrypto code removes requests from a list while
iterating with TAILQ_FOREACH_REVERSE().
Add missing cv_destroy() calls for the key request cleanup.
Reviewed by Thor Lancelot Simon.
 1.29.4.1.4.1 20-May-2011  matt bring matt-nb5-mips64 up to date with netbsd-5-1-RELEASE (except compat).
 1.29.2.2 28-Apr-2009  skrll Sync with HEAD.
 1.29.2.1 19-Jan-2009  skrll Sync with HEAD.
 1.30.4.1 13-May-2009  jym Sync with HEAD.

Commit is split, to avoid a "too many arguments" protocol error.
 1.34.4.3 12-Jun-2011  rmind sync with head
 1.34.4.2 31-May-2011  rmind sync with head
 1.34.4.1 05-Mar-2011  rmind sync with head
 1.34.2.1 17-Aug-2010  uebayasi Sync with HEAD.
 1.36.4.2 05-Mar-2011  bouyer Sync with HEAD
 1.36.4.1 08-Feb-2011  bouyer Sync with HEAD
 1.36.2.1 06-Jun-2011  jruoho Sync with HEAD.
 1.40.2.1 23-Jun-2011  cherry Catchup with rmind-uvmplock merge.
 1.41.16.1 18-May-2014  rmind sync with head
 1.41.12.2 03-Dec-2017  jdolecek update from HEAD
 1.41.12.1 20-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.41.2.1 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.45.6.3 28-Aug-2017  skrll Sync with HEAD
 1.45.6.2 09-Jul-2016  skrll Sync with HEAD
 1.45.6.1 27-Dec-2015  skrll Sync with HEAD (as of 26th Dec)
 1.48.4.1 21-Apr-2017  bouyer Sync with HEAD
 1.48.2.2 26-Apr-2017  pgoyette Sync with HEAD
 1.48.2.1 20-Mar-2017  pgoyette Sync with HEAD
 1.58.2.2 19-May-2017  pgoyette Resolve conflicts from previous merge (all resulting from $NetBSD
keywork expansion)
 1.58.2.1 11-May-2017  pgoyette Sync with HEAD
 1.78.2.5 27-Feb-2018  martin Pull up following revision(s) (requested by mrg in ticket #593):
sys/dev/marvell/mvxpsec.c: revision 1.2
sys/arch/m68k/m68k/pmap_motorola.c: revision 1.70
sys/opencrypto/crypto.c: revision 1.102
sys/arch/sparc64/sparc64/pmap.c: revision 1.308
sys/ufs/chfs/chfs_malloc.c: revision 1.5
sys/arch/powerpc/oea/pmap.c: revision 1.95
sys/sys/pool.h: revision 1.80,1.82
sys/kern/subr_pool.c: revision 1.209-1.216,1.219-1.220
sys/arch/alpha/alpha/pmap.c: revision 1.262
sys/kern/uipc_mbuf.c: revision 1.173
sys/uvm/uvm_fault.c: revision 1.202
sys/sys/mbuf.h: revision 1.172
sys/kern/subr_extent.c: revision 1.86
sys/arch/x86/x86/pmap.c: revision 1.266 (via patch)
sys/dev/dtv/dtv_scatter.c: revision 1.4

Allow only one pending call to a pool's backing allocator at a time.
Candidate fix for problems with hanging after kva fragmentation related
to PR kern/45718.

Proposed on tech-kern:
https://mail-index.NetBSD.org/tech-kern/2017/10/23/msg022472.html
Tested by bouyer@ on i386.

This makes one small change to the semantics of pool_prime and
pool_setlowat: they may fail with EWOULDBLOCK instead of ENOMEM, if
there is a pending call to the backing allocator in another thread but
we are not actually out of memory. That is unlikely because nearly
always these are used during initialization, when the pool is not in
use.

Define the new flag too for previous commit.

pool_grow can now fail even when sleeping is ok. Catch this case in pool_get
and retry.

Assert that pool_get failure happens only with PR_NOWAIT.
This would have caught the mistake I made last week leading to null
pointer dereferences all over the place, a mistake which I evidently
poorly scheduled alongside maxv's change to the panic message on x86
for null pointer dereferences.

Since pr_lock is now used to wait for two things now (PR_GROWING and
PR_WANTED) we need to loop for the condition we wanted.
make the KASSERTMSG/panic strings consistent as '%s: [%s], __func__, wchan'
Handle the ERESTART case from pool_grow()

don't pass 0 to the pool flags
Guess pool_cache_get(pc, 0) means PR_WAITOK here.
Earlier on in the same context we use kmem_alloc(sz, KM_SLEEP).

use PR_WAITOK everywhere.
use PR_NOWAIT.

Don't use 0 for PR_NOWAIT

use PR_NOWAIT instead of 0

panic ex nihilo -- PR_NOWAITing for zerot

Add assertions that either PR_WAITOK or PR_NOWAIT are set.
- fix an assert; we can reach there if we are nowait or limitfail.
- when priming the pool and failing with ERESTART, don't decrement the number
of pages; this avoids the issue of returning an ERESTART when we get to 0,
and is more correct.
- simplify the pool_grow code, and don't wakeup things if we ENOMEM.

In pmap_enter_ma(), only try to allocate pves if we might need them,
and even if that fails, only fail the operation if we later discover
that we really do need them. This implements the requirement that
pmap_enter(PMAP_CANFAIL) must not fail when replacing an existing
mapping with the first mapping of a new page, which is an unintended
consequence of the changes from the rmind-uvmplock branch in 2011.

The problem arises when pmap_enter(PMAP_CANFAIL) is used to replace an existing
pmap mapping with a mapping of a different page (eg. to resolve a copy-on-write).
If that fails and leaves the old pmap entry in place, then UVM won't hold
the right locks when it eventually retries. This entanglement of the UVM and
pmap locking was done in rmind-uvmplock in order to improve performance,
but it also means that the UVM state and pmap state need to be kept in sync
more than they did before. It would be possible to handle this in the UVM code
instead of in the pmap code, but these pmap changes improve the handling of
low memory situations in general, and handling this in UVM would be clunky,
so this seemed like the better way to go.

This somewhat indirectly fixes PR 52706, as well as the failing assertion
about "uvm_page_locked_p(old_pg)". (but only on x86, various other platforms
will need their own changes to handle this issue.)
In uvm_fault_upper_enter(), if pmap_enter(PMAP_CANFAIL) fails, assert that
the pmap did not leave around a now-stale pmap mapping for an old page.
If such a pmap mapping still existed after we unlocked the vm_map,
the UVM code would not know later that it would need to lock the
lower layer object while calling the pmap to remove or replace that
stale pmap mapping. See PR 52706 for further details.
hopefully workaround the irregularly "fork fails in init" problem.
if a pool is growing, and the grower is PR_NOWAIT, mark this.
if another caller wants to grow the pool and is also PR_NOWAIT,
busy-wait for the original caller, which should either succeed
or hard-fail fairly quickly.

implement the busy-wait by unlocking and relocking this pools
mutex and returning ERESTART. other methods (such as having
the caller do this) were significantly more code and this hack
is fairly localised.
ok chs@ riastradh@

Don't release the lock in the PR_NOWAIT allocation. Move flags setting
after the acquiring the mutex. (from Tobias Nygren)
apply the change from arch/x86/x86/pmap.c rev. 1.266 commitid vZRjvmxG7YTHLOfA:

In pmap_enter_ma(), only try to allocate pves if we might need them,
and even if that fails, only fail the operation if we later discover
that we really do need them. If we are replacing an existing mapping,
reuse the pv structure where possible.

This implements the requirement that pmap_enter(PMAP_CANFAIL) must not fail
when replacing an existing mapping with the first mapping of a new page,
which is an unintended consequence of the changes from the rmind-uvmplock
branch in 2011.

The problem arises when pmap_enter(PMAP_CANFAIL) is used to replace an existing
pmap mapping with a mapping of a different page (eg. to resolve a copy-on-write).
If that fails and leaves the old pmap entry in place, then UVM won't hold
the right locks when it eventually retries. This entanglement of the UVM and
pmap locking was done in rmind-uvmplock in order to improve performance,
but it also means that the UVM state and pmap state need to be kept in sync
more than they did before. It would be possible to handle this in the UVM code
instead of in the pmap code, but these pmap changes improve the handling of
low memory situations in general, and handling this in UVM would be clunky,
so this seemed like the better way to go.

This somewhat indirectly fixes PR 52706 on the remaining platforms where
this problem existed.
 1.78.2.4 02-Jan-2018  snj Pull up following revision(s) (requested by knakahara in ticket #460):
sys/opencrypto/crypto.c: revision 1.101
fix opencrypto(9) part of PR kern/52515
percpu data use pointers to TAILQ instead of TAILQ itself.
 1.78.2.3 05-Aug-2017  snj Pull up following revision(s) (requested by knakahara in ticket #178):
sys/opencrypto/crypto.c: 1.92-1.100
sys/opencrypto/cryptodev.h: 1.38-1.39
sys/opencrypto/ocryptodev.c: 1.9-1.11
sys/opencrypto/xform.c: revision 1.29
sys/opencrypto/xform.h: revision 1.20
KNF
--
Apply C99-style struct initialization to enc_xform, auth_hash and comp_algo
--
make cryptoret() context softint to balance dequeuing crypto_ret_q with enqueuing it.
--
fix panic when using ecryption devices attached earlier than ipi_sysinit().
pointed out and tested by martin@n.o, thanks.
--
fix typo
--
make crp_{,k}q percpu to scale crypto_dispatch().
update locking note later.
--
divide crp_ret_{,k}q by CPU to avoid reordering.
update locking note later.
--
update locking notes of opencrypto(9)
--
Don't disclose uninitialized 32-bit word if cryptodev_session fails.
From Ilja Van Sprundel.
--
Avert userland-controlled integer overflow.
From Ilja Van Sprundel.
--
Avoid another userland-controlled integer overflow.
From Ilja Van Sprundel.
--
refactor: remove glue macros for FreeBSD code.
--
pack crypto_drivers variables to struct and add cacheline_aligned qualifier.
--
use kmem_alloc KPI instead of malloc KPI.
--
use pool_cache(9) instead of pool(9) as they can be called concurrently.
 1.78.2.2 05-Jul-2017  snj Pull up following revision(s) (requested by knakahara in ticket #97):
sys/opencrypto/crypto.c: 1.87-1.91
sys/opencrypto/cryptodev.c: 1.93-1.95
sys/opencrypto/cryptodev.h: 1.37
sys/opencrypto/cryptosoft.c: 1.52
sys/rump/dev/lib/libopencrypto/opencrypto_component.c: 1.5
sanitize count used for kmem_alloc size.
Hmm, who uses CIOCNGSESSION, CIOCNFSESSION, CIOCNCRYPTM or CIOCNFKEYM?
--
sanitize in CIOCNCRYPTM and initialize comp_alg in CIOCNGSESSION
--
must release cap->cc_lock before calling cap->cc_newsession() because of spinlock.
--
refactor crypto_newsession() like FreeBSD.
--
support multiple encryption drivers (port from FreeBSD).
--
Divide crp_devflags from crp_flags to write exclusively.
CRYPTO_F_DQRETQ(new name is CRYPTODEV_F_RET) is used by cryptodev.c only.
It should be divided to other member.
--
Reduce crypto_ret_q_mtx lock regions.
crypto.c does not access the members of crp when the crp is in crp_q or
crp_ret_q. Furthermore, crp_q and crp_ret_q are protected by each mutex,
so the members of crp is not shared. That means crp_flags is not required
mutex in crypto.c.
--
fix cryptosoft.c:r1.51 mistake. swcrypto_attach() must not be called from module_init_class().
swcrypto_attach() will call softint_establish(), it must be called after cpus
attached. module_init_class() is too early to call softint_establish().
--
simplify mutex_enter/exit(crypto_q_mtx), and fix missing exit.
--
reduce rump waring message. pointed out by ozaki-r@n.o, thanks.
 1.78.2.1 22-Jun-2017  snj Pull up following revision(s) (requested by christos/knakahara in ticket #37):
sys/opencrypto/crypto.c: 1.79-1.86
sys/opencrypto/cryptodev.h: 1.35, 1.36
use crypto_checkdriver_uninit() when it may touch uninitialized crypto_drivers.
--
fix reading crp_q without holding crypto_q_mtx
--
restructure locks(1/2): make relation between lock and data explicit.
+ crypto_drv_mtx protects
- whole crypto_drivers
+ crypto_drivers[i].cc_lock (new) protects
- crypto_drivers[i] itself
- member of crypto_drivers[i]
+ crypto_q_mtx protects
- crp_q
- crp_kq
+ crypto_ret_q_mtx protects
- crp_ret_q
- crp_ret_kq
- crypto_exit_flag
I will add locking note later.
--
restructure locks(2/2): crypto_q_mtx can be adaptive now.
--
add locking notes.
--
avoid crp_q reordering as hardware interrupts.
crypto_{,k}invoke() can be called with holding crp_q_mtx now.
--
apply the same fix as crypto.c:r1.83 for crypto_dispatch to crypto_kdispatch.
--
- acquire lock
- use c99 loop indexes
- initialize featp
--
Put back crypto_checkdriver(); use it when we need to make sure that we
get back a cryptocap that has been initialized.
 1.92.2.2 18-Jul-2017  knakahara 3212654
 1.92.2.1 18-Jul-2017  knakahara file crypto.c was added on branch perseant-stdc-iso10646 on 2017-07-18 06:01:37 +0000
 1.105.2.1 25-Jun-2018  pgoyette Sync with HEAD
 1.106.2.1 13-Apr-2020  martin Mostly merge changes from HEAD upto 20200411
 1.110.2.1 29-Feb-2020  ad Sync with head.
 1.2 24-Aug-2003  thorpej crypto_mbuf.c is unused; remove it.
 1.1 25-Jul-2003  jonathan Commit initial NetBSD port of the OpenCrypto Framework (OCF). This
code is derived from Sam Leffler's FreeBSD port of OCF, which is in
turn a port of Angelos Keromytis's OpenBSD work.
Credit to Sam and Angelos, any blame for the NetBSD port to me.
 1.126 17-Apr-2025  riastradh opencrypto: Fix typo that somehow snuck past our resident typo-hunter.

crypto_userasymcrypto (as in asymmetric), not crypto_userasmcrypto
(as in assembly).
 1.125 10-Sep-2022  rillig branches: 1.125.10;
fix misspellings of 'available' and nearby typos
 1.124 22-May-2022  riastradh opencrypto: Prune dead code now that crypto_dispatch never fails.
 1.123 22-May-2022  riastradh opencrypto: crypto_dispatch never fails now. Make it return void.

Same with crypto_kdispatch.
 1.122 22-May-2022  riastradh opencrypto: Rip out EAGAIN logic when unregistering crypto drivers.

I'm pretty sure this never worked reliably based on code inspection,
and it's unlikely to have ever been tested because it only applies
when unregistering a driver -- but we have no crypto drivers for
removable devices, so it would only apply if we went out of our way
to trigger detach with drvctl.

Instead, just make the operation fail with ENODEV, and remove all the
callback logic to resubmit the request on EAGAIN. (Maybe this should
be ENXIO, but crypto_kdispatch already does ENODEV.)
 1.121 22-May-2022  riastradh crypto(4): Nix dead code now that crypto_freesession never fails.
 1.120 22-May-2022  riastradh opencrypto: Make crypto_freesession return void.

No callers use the return value. It is not sensible to allow this to
fail.
 1.119 22-May-2022  riastradh crypto(4): crypto_freesession should never fail here.

It can only fail if we pass it an invalid sid, which the logic to
maintain the user sessions should not do. So kassert error=0 here.
 1.118 22-May-2022  riastradh crypto(4): Refuse crypto operations with nothing in them earlier.

This way we avoid passing 0 to crypto_getreq -- makes it easier to
reason about everything downstream.
 1.117 22-May-2022  riastradh opencrypto: Make crp_callback, krp_callback return void.

Nothing uses the return values inside opencrypto, so let's stop
making users return them.
 1.116 22-May-2022  riastradh crypto(4): Fix possible use-after-free in race around detach.

This is extremely unlikely because I don't think we have any drivers
for removable crypto decelerators^Waccelerators...but if we were to
sprout one, and someone ran crypto_dispatch concurrently with
crypto_unregister, cryptodev_cb/mcb would enter with crp->crp_etype =
EAGAIN and with CRYPTO_F_DONE set in crp->crp_flags. In this case,
cryptodev_cb/mcb would issue crypto_dispatch but -- since nothing
clears CRYPTO_F_DONE -- it would _also_ consider the request done and
notify the ioctl thread of that.

With this change, we return early if crypto_dispatch succeeds. No
need to consult CRYPTO_F_DONE: if the callback is invoked it's done,
and if we try to redispatch it on EAGAIN but crypto_dispatch fails,
it's done. (Soon we'll get rid of the possibility of crypto_dispatch
failing synchronously, but not just yet.)

XXX This path could really use some testing!
 1.115 21-May-2022  riastradh crypto(4): Fix set-but-unused variable warning.

This deliberately ignores the error code returned by crypto_dispatch,
but that error code is fundamentally incoherent and the issue will be
mooted by subsequent changes to make it return void and always pass
the error through the callback, as well as subsequent changes to rip
out the EAGAIN logic anyway.
 1.114 21-May-2022  riastradh crypto(4): Don't signal the condvar for multi-operation completion.

The condvar may be destroyed by the time we got here, and nothing
waits on it anyway -- instead the caller is expected to select/poll
for completion in userland.

The bug was already here, but the recent change to eliminate
CRYPTO_F_CBIMM made it happen more often by causing the callback to
_always_ be run asynchronously instead of sometimes being run
synchronously.
 1.113 19-May-2022  riastradh opencrypto: Nix CRYPTO_F_USER, CRYPTO_F_CBIMM, CRYPTO_F_CBIFSYNC.

CRYPTO_F_USER is no longer needed. It was introduced in 2008 by
darran@ in crypto.c 1.30, cryptodev.c 1.45 in an attempt to avoid
double-free between the issuing thread and asynchronous callback.
But the `fix' didn't work. In 2017, knakahara@ fixed it properly in
cryptodev.c 1.87 by distinguishing `the crypto operation has
completed' (CRYPTO_F_DONE) from `the callback is done touching the
crp object' (CRYPTO_F_DQRETQ, now renamed to CRYPTODEV_F_RET).

CRYPTO_F_CBIMM formerly served to invoke the callback synchronously
from the driver's interrupt completion routine, to reduce contention
on what was once a single cryptoret thread. Now, there is a per-CPU
queue and softint for much cheaper processing, so there is less
motivation for this in the first place. So let's remove the
complicated logic. This means the callbacks never run in hard
interrupt context, which means we don't need to worry about recursion
into crypto_dispatch in hard interrupt context.
 1.112 18-May-2022  riastradh crypto(4): Simplify error test in cryptodev_op.

No functional change intended.
 1.111 18-May-2022  riastradh crypto(4): Narrow scope of cryptodev_mtx to cover wait.

No functional change intended -- this only removes an unnecessary
lock/unlock cycle in the error case.
 1.110 18-May-2022  riastradh crypto(4): Nix long-dead code and comments.
 1.109 18-May-2022  riastradh crypto(4): Use IPL_NONE, not IPL_NET, for /dev/crypto pools.

These are used (pool_get/put) only from thread context, never from
interrupt or even soft interrupt context.
 1.108 17-May-2022  riastradh opencrypto(9): Omit needless casts around callbacks.

Just declare the right types to begin with. No functional change
intended.
 1.107 31-Mar-2022  pgoyette For device modules that provide both auto-config and /dev/xxx
interfaces, make sure that initialization and destruction
follow the proper sequence. This is triggered by the recent
changes to the devsw stuff; per riastradh@ the required call
sequence is:

devsw_attach()
config_init_component() or config_cf*_attach()
...
config_fini_component() or config_cf*_detach()
devsw_detach()

While here, add a few missing calls to some of the detach
routines.

Testing of these changes has been limited to:
1. compile without build break
2. no related test failures from atf
3. modload/modunload work as well as
before.

No functional device testing done, since I don't have any
of these devices. Let me know of any damage I might cause
here!

XXX Some of the modules affected by this commit are already
XXX broken; see kern/56772. This commit does not break
any additional modules (as far as I know).
 1.106 30-Jun-2020  riastradh Rename enc_xform_rijndael128 -> enc_xform_aes.

Update netipsec dependency.
 1.105 13-Apr-2020  chs slightly change and fix the semantics of pool_set*wat(), pool_sethardlimit()
and pool_prime() (and their pool_cache_* counterparts):

- the pool_set*wat() APIs are supposed to specify thresholds for the count of
free items in the pool before pool pages are automatically allocated or freed
during pool_get() / pool_put(), whereas pool_sethardlimit() and pool_prime()
are supposed to specify minimum and maximum numbers of total items
in the pool (both free and allocated). these were somewhat conflated
in the existing code, so separate them as they were intended.

- change pool_prime() to take an absolute number of items to preallocate
rather than an increment over whatever was done before, and wait for
any memory allocations to succeed. since pool_prime() can no longer fail
after this, change its return value to void and adjust all callers.

- pool_setlowat() is documented as not immediately attempting to allocate
any memory, but it was changed some time ago to immediately try to allocate
up to the lowat level, so just fix the manpage to describe the current
behaviour.

- add a pool_cache_prime() to complete the API set.
 1.104 27-Jan-2020  pgoyette branches: 1.104.4;
If we get an error from devsw_attach(), don't destroy the error value,
since we need to return it to our caller. While we're here, improve
the value of the debug message by actually printing the error value.
 1.103 16-Jan-2020  christos Initialize the session variable to an impossible session to prevent compiler
warnings.
 1.102 29-Nov-2019  hikaru branches: 1.102.2;
crypto(4): accept CRYPTO_SHA2_384_HMAC and CRYPTO_SHA2_512_HMAC.
 1.101 13-Jun-2019  christos don't always panic when modunload crypto (int the pool destroy code, because
the pools are busy). XXX: this is still racy; we need to prevent creating
more sessions while destroying.
 1.100 01-Mar-2019  pgoyette Rename the MODULE_*_HOOK() macros to MODULE_HOOK_*() as briefly
discussed on irc.

NFCI intended.

Ride the earlier kernel bump - it;s getting crowded.
 1.99 27-Jan-2019  pgoyette Merge the [pgoyette-compat] branch
 1.98 08-Feb-2018  dholland branches: 1.98.2; 1.98.4;
Typos.
 1.97 30-Nov-2017  christos add fo_name so we can identify the fileops in a simple way.
 1.96 14-Nov-2017  christos check results of pool_prime.
 1.95 15-Jun-2017  knakahara Divide crp_devflags from crp_flags to write exclusively.

CRYPTO_F_DQRETQ(new name is CRYPTODEV_F_RET) is used by cryptodev.c only.
It should be divided to other member.
 1.94 08-Jun-2017  knakahara sanitize in CIOCNCRYPTM and initialize comp_alg in CIOCNGSESSION
 1.93 08-Jun-2017  knakahara sanitize count used for kmem_alloc size.

Hmm, who uses CIOCNGSESSION, CIOCNFSESSION, CIOCNCRYPTM or CIOCNFKEYM?
 1.92 02-Jun-2017  knakahara branches: 1.92.2;
rename crypto_mtx to cryptodev_mtx

It is used by cryptodev.c and ocryptodev.c only.
 1.91 25-May-2017  knakahara add cryptkop alloc/free KPI instead of manipulating cryptkop_pool directly.
 1.90 17-May-2017  knakahara opencrypto: cleanup debug messages.
 1.89 24-Apr-2017  knakahara branches: 1.89.2;
separate crypto_drv_mtx from crypto_mtx.

crypto_mtx is used only for cryptodev.c and ocryptodev.c now.
 1.88 07-Apr-2017  knakahara the processing said "ghastly hacks" is unnecessary now.
 1.87 07-Apr-2017  knakahara fix race among crypto_done(), cryptoret(), and {cryptodev_op(), cryptodev_key()}.

crypto_op() waited to be set CRYPTO_F_DONE with crp->crp_cv.
However, there is context switch chances between being set CRYPTO_F_DONE in
crypto_done() and done cv_signal(crp->crp_cv) in cryptodev_cb(), that is,
cryptodev_op() thread can run to cv_destroy(crp->crp_cv) before cryptoret()
thread is waken up. As a result, cryptodev_cb() can call invalid(destroyed)
cv_signal(crp->crp_cv).

Furthermore, below two implementations cause other races.
- waiting CRYPTO_F_DONE with crp->crp_cv
- context witch chances between set CRYPTO_F_DONE and cv_signal(crp->crp_cv)

So, use other flag(CRYPTO_F_DQRETQ) for cryptodev_op() and cryptodev_key(),
and then call cv_signal(crp->crp_cv) immediately after set CRYPTO_F_DQRETQ.

Tested concurrent over 20 processes with software and hardware drivers.
 1.86 05-Apr-2017  knakahara fix processes accessing /dev/crypto stall when over three processes run with a hardware encryption driver

The process has stalled at cv_wait(&crp->crp_cv) because cryptodev_cb()
is not called as cryptoret() kthread keep waiting at cv_wait(&cryptoret_cv).
Previous opencrypto implementation assumes the thread from cryptodev.c
does all processing in the same context, so skips enqueueing and sending
cryptoret_cv. However, the context can be switched, e.g. when we use
a hardware encryption driver.

And add debug messages.
 1.85 07-Jul-2016  msaitoh branches: 1.85.2; 1.85.4;
KNF. Remove extra spaces. No functional change.
 1.84 20-Aug-2015  christos include "ioconf.h" to get the 'void <driver>attach(int count);' prototype.
 1.83 26-Mar-2015  prlw1 Trivial printf format changes and typo fix
 1.82 27-Nov-2014  christos branches: 1.82.2;
Return ENOSPC instead of ENOMEM when there is no room in the buffer to
store results. ENOMEM in this subsystem means we cannot allocate more
requests or internal buffers for xforms.
 1.81 05-Sep-2014  matt Try not to use f_data, use f_fcrypt to get a correctly typed pointer.
 1.80 04-Aug-2014  skrll At least crypto_mtx needs initialisation here. Spotted during PR/49065
investigation.
 1.79 25-Jul-2014  dholland Add d_discard to all struct cdevsw instances I could find.

All have been set to "nodiscard"; some should get a real implementation.
 1.78 16-Mar-2014  dholland branches: 1.78.2;
Change (mostly mechanically) every cdevsw/bdevsw I can find to use
designated initializers.

I have not built every extant kernel so I have probably broken at
least one build; however I've also found and fixed some wrong
cdevsw/bdevsw entries so even if so I think we come out ahead.
 1.77 03-Feb-2014  pgoyette Undo previous - it still needs a lot more work.

For now, we'll use the hand-crafted cf* structures and directly
call all the config routines.
 1.76 31-Jan-2014  pgoyette Replace home-grown config with standardized calls to
config_{init,fini}_component()
 1.75 24-Jan-2014  pgoyette As requested by mrg@, since there is still a small window during which
the in-module ref-counting can fail, completely disable auto-unload.
 1.74 21-Jan-2014  pgoyette Implement in-module ref-counting, and do not allow auto-unload if there
are existing references.

Note that manual unloading is not prevented.

OK christos@

XXX Also note that there is still a small window where the ref-count can
XXX be decremented, and then the process/thread preempted. If auto-unload
XXX happens before that thread can return from the module's code, bad
XXX things (tm) could happen.
 1.73 21-Jan-2014  pgoyette knf: Blank line even if no variable declarations.
 1.72 19-Jan-2014  christos bail out unloading for now
 1.71 04-Jan-2014  pgoyette When crypto(4) is built-in, crypto_modcmd() doesn't need to handle all
the auto-config stuff.

While here, ensure that we depend on opencrypto.
 1.70 01-Jan-2014  pgoyette Modularize the opencrypto components and link to the build
 1.69 12-Sep-2013  martin Fix return value of cryptodev_msessionfin.
 1.68 04-Jul-2011  joerg branches: 1.68.2; 1.68.12; 1.68.16;
Fix memset usage.
 1.67 09-Jun-2011  drochner -if an opencrypto(9) session is allocated, the driver is refcounted
and can not disappear -- no need to hold crypto_mtx to check the
driver list
(the whole check is questionable)
-crp->crp_cv (the condition variable) is used by userland cryptodev
exclusively -- move its initialization there, no need to waste
cycles of in-kernel callers
-add a comment which members of "struct cryptop" are used
by opencrypto(9) and which by crypto(4)
(this should be split, no need to waste memory for in-kernel callers)
 1.66 27-May-2011  drochner branches: 1.66.2;
allow testing of GCM/GMAC code from userland
 1.65 26-May-2011  drochner fix building of a linked list if multiple algorithms are requested
in a session -- this just didn't work
 1.64 24-May-2011  drochner catch some corner cases of user input
 1.63 24-May-2011  drochner copy AES-XCBC-MAC support from KAME IPSEC to FAST_IPSEC
For this to fit, an API change in cryptosoft was adopted from OpenBSD
(addition of a "Setkey" method to hashes) which was done for GCM/GMAC
support there, so it might be useful in the future anyway.
tested against KAME IPSEC
AFAICT, FAST_IPSEC now supports as much as KAME.
 1.62 23-May-2011  drochner -remove references to crypto/arc4/arc4.* -- the code isn't used
anywhere afaics
(The confusion comes probably from use of arc4random() at various places,
but this lives in libkern and doesn't share code with the former.)
-g/c non-implementation of arc4 encryption in swcrypto(4)
-remove special casing of ARC4 in crypto(4) -- the point is that it
doesn't use an IV, and this fact is made explicit by the new "ivsize"
property of xforms
 1.61 23-May-2011  drochner If symmetric encryption is done from userland crypto(4) and no IV
is specified, the kernel gets one from the random generator. Make sure it
is copied out to the user, otherwise the result is quite useless.
 1.60 23-May-2011  drochner being here, export camellia-cbc through crypto(4) to allow userland tests
 1.59 23-May-2011  drochner add an AES-CTR xform, from OpenBSD
 1.58 23-May-2011  drochner -in the descriptor for encryption xforms, split the "blocksize" field
into "blocksize" and "IV size"
-add an "reinit" function pointer which, if set, means that the xform
does its IV handling itself and doesn't want the default CBC handling
by the framework (poor name, but left that way to avoid unecessary
differences)
This syncs with Open/FreeBSD, purpose is to allow non-CBC transforms.
Refer to ivsize instead of blocksize where appropriate.
(At this point, blocksize and ivsize are identical.)
 1.57 16-May-2011  drochner split the "crypto_mtx" spinlock into 3: one spinlock each for
the incoming and outgoing request queues (which can be dealt with
by hardware accelerators) and an adaptive lock for "all the rest"
(mostly driver configuration, but also some unrelated stuff in
cryptodev.c which should be revisited)
The latter one seems to be uneeded at many places, but for now I've
done simple replacements only, except minor fixes (where
softint_schedule() was called without the lock held)
 1.56 06-May-2011  drochner As a first step towards more fine-grained locking, don't require
crypto_{new.free}session() to be called with the "crypto_mtx"
spinlock held.
This doesn't change much for now because these functions acquire
the said mutex first on entry now, but at least it keeps the nasty
locks local to the opencrypto core.
 1.55 19-Feb-2011  drochner make the compatibility code conditional on COMPAT_50
 1.54 18-Feb-2011  drochner more "const"
 1.53 02-Aug-2010  jakllsch branches: 1.53.2; 1.53.4;
Consistently use a single CRYPTO_SESID2HID-like macro.
Improve CRYPTO_DEBUG printing a bit:
print pointers with %p
print unsigned with %u rather than %d
use CRYPTO_SESID2LID instead of just casting to uint32_t
 1.52 31-Jan-2010  hubertf branches: 1.52.2; 1.52.4;
Add missing "break" for CRYPTO_CAST_CBC, and some assorted comment fixes.
openssl(1) checks for CAST (and others) on ~every startup.
 1.51 20-Dec-2009  dsl If a multithreaded app closes an fd while another thread is blocked in
read/write/accept, then the expectation is that the blocked thread will
exit and the close complete.
Since only one fd is affected, but many fd can refer to the same file,
the close code can only request the fs code unblock with ERESTART.
Fixed for pipes and sockets, ERESTART will only be generated after such
a close - so there should be no change for other programs.
Also rename fo_abort() to fo_restart() (this used to be fo_drain()).
Fixes PR/26567
 1.50 09-Dec-2009  dsl Rename fo_drain() to fo_abort(), 'drain' is used to mean 'wait for output
do drain' in many places, whereas fo_drain() was called in order to force
blocking read()/write() etc calls to return to userspace so that a close()
call from a different thread can complete.
In the sockets code comment out the broken code in the inner function,
it was being called from compat code.
 1.49 11-Apr-2009  christos Fix locking as Andy explained. Also fill in uid and gid like sys_pipe did.
 1.48 11-Apr-2009  christos Fix PR/37878 and PR/37550: Provide stat(2) for all devices and don't use
fbadop_stat.
 1.47 04-Apr-2009  ad Add fileops::fo_drain(), to be called from fd_close() when there is more
than one active reference to a file descriptor. It should dislodge threads
sleeping while holding a reference to the descriptor. Implemented only for
sockets but should be extended to pipes, fifos, etc.

Fixes the case of a multithreaded process doing something like the
following, which would have hung until the process got a signal.

thr0 accept(fd, ...)
thr1 close(fd)
 1.46 25-Mar-2009  darran Fixes PR kern/41069 and PR kern/41070.

Extends the Opencrypto API to allow the destination buffer size to be
specified when its not the same size as the input buffer (i.e. for
operations like compress and decompress).
The crypto_op and crypt_n_op structures gain a u_int dst_len field.
The session_op structure gains a comp_alg field to specify a compression
algorithm.
Moved four ioctls to new ids; CIOCGSESSION, CIOCNGSESSION, CIOCCRYPT,
and CIOCNCRYPTM.
Added four backward compatible ioctls; OCIOCGSESSION, OCIOCNGSESSION,
OCIOCCRYPT, and OCIOCNCRYPTM.

Backward compatibility is maintained in ocryptodev.h and ocryptodev.c which
implement the original ioctls and set dst_len and comp_alg to 0.

Adds user-space access to compression features.

Adds software gzip support (CRYPTO_GZIP_COMP).

Adds the fast version of crc32 from zlib to libkern. This should be generally
useful and provide a place to start normalizing the various crc32 routines
in the kernel. The crc32 routine is used in this patch to support GZIP.

With input and support from tls@NetBSD.org.
 1.45 18-Nov-2008  darran branches: 1.45.4;
Fix a race condition in opencrypto where the crypto request could be
completed by the crypto device, queued on the retq, but freed by the
ioctl lwp. The problem manifests as various panics relating to the
condvar inside the request. The problem can occur whenever the crypto
device completes the request immediately and the ioctl skips the cv_wait().

The problem can be reproduced by enabling cryptosoft and running an openssl
speed test. E.g.
sysctl -w kern.cryptodevallowsoft=-1
openssl speed -engine cryptodev -evp des-ede3-cbc -multi 64

Add a macro for TAILQ_FOREACH_REVERSE_SAFE() to queue.h, since this
was missing and the opencrypto code removes requests from a list while
iterating with TAILQ_FOREACH_REVERSE().

Add missing cv_destroy() calls for the key request cleanup.

Reviewed by Thor Lancelot Simon.
 1.44 24-May-2008  christos branches: 1.44.4; 1.44.6; 1.44.8;
Coverity CID 5021: Check pointers before using.
 1.43 24-May-2008  christos Coverity CID 5027: Remove impossible test.
 1.42 24-May-2008  christos KNF, whitespace, b* -> mem*. No functional change.
 1.41 30-Apr-2008  ad branches: 1.41.2;
Make various bits of debug code compile again.
 1.40 28-Apr-2008  martin Remove clause 3 and 4 from TNF licenses
 1.39 21-Apr-2008  tls branches: 1.39.2;
As suggested by rmind, do not check return status of KM_SLEEP/PR_WAITOK
allocations. A little hair-raising but it does make the code easier to
read.
 1.38 11-Apr-2008  rmind branches: 1.38.2;
Protect selrecord/selnotify calls with crypto_mtx; few misc changes.
 1.37 11-Apr-2008  dogcow fix 64-bit b0rkenness.
 1.36 10-Apr-2008  tls Extend crypto.4 interface:

* Asynchronous operation with result retrieval via select/poll
* Mutliple-request submit/retrieve ioctls
* Mutliple-session create-destroy ioctls

Revise/rewrite crypto.4 manual page. It should now be much easier to write
new applications to this API.

Measured performance for trivial requests: 84,000 very short modular math
operations/sec, 120,000 very short md5 hashes per sec (with a hardware
accellerator of moderate performance but very low latency, whose driver
will be contributed at a later date).

Contributed to TNF by Coyote Point Systems, Inc.
 1.35 21-Mar-2008  ad Catch up with descriptor handling changes. See kern_descrip.c revision
1.173 for details.
 1.34 04-Feb-2008  tls branches: 1.34.6;
Some locking fixes (double-release mutex in softintr wakeup case, which I
hadn't tested) and an uninitialized field in cse which Darran Hunt
found. Some more debugging printfs.

Turn on MPSAFE for the kthread. We're not sure it's safe for the softint
yet. Gives a little performance kick for swcrypto with many requests on
MP systems.
 1.33 04-Feb-2008  tls Rework opencrypto to use a spin mutex (crypto_mtx) instead of "splcrypto"
(actually splnet) and condvars instead of tsleep/wakeup. Fix a few
miscellaneous problems and add some debugging printfs while there.

Restore set of CRYPTO_F_DONE in crypto_done() which was lost at some
point after this code came from FreeBSD -- it made it impossible to wait
properly for a condition.

Add flags analogous to the "crp" flags to the key operation's krp struct.
Add a new flag, CRYPTO_F_ONRETQ which tells us a request finished before
the kthread had a chance to dequeue it and call its callback -- this was
letting requests stick on the queues before even though done and copied
out.

Callers of crypto_newsession() or crypto_freesession() must now take the
mutex. Change netipsec to do so. Dispatch takes the mutex itself as
needed.

This was tested fairly extensively with the cryptosoft backend and lightly
with a new hardware driver. It has not been tested with FAST_IPSEC; I am
unable to ascertain whether FAST_IPSEC currently works at all in our tree.

pjd@FreeBSD.ORG, ad@NetBSD.ORG, and darran@snark.us pointed me in the
right direction several times in the course of this. Remaining bugs
are mine alone.
 1.32 02-Feb-2008  tls From Darran Hunt at Coyote Point: don't truncate HMAC to 96 bits unless
actually asked to.

Fixed in FreeBSD a while ago, discussed on tech-kern and tech-crypto.
 1.31 01-Feb-2008  tls This code never worked on a released version of FreeBSD in the form it's
been in in our tree, and certainly does not work on any version of FreeBSD
now. Run through unifdef -D__NetBSD__ -U__FreeBSD__ yielding a small
reduction of size and a dramatic improvement in readability.

No, this does not yield any meaningful decrease in patchability (unlike
mechanical changes that touch live source lines) -- try it and see.
 1.30 29-Jan-2008  tls Fix accidental checkin inverting the sense of cryptodev_allowsoft, which
is crazy but has always documented.
 1.29 26-Jan-2008  tls Make /dev/crypto properly cloning. Leave CRIOGET in place but note that
it is deprecated, no longer required, and will be removed in a future
release of NetBSD.

Dramatically reduce the size of the session structure by removing an
IOV_MAX array of iovecs where only the first was use. Saves an 8k
bzero on each session creation.

Convert fixed-size allocations in cryptodev.c to pools.
 1.28 25-Jan-2008  tls Some minor opencrypto fixes, one with a major performance impact for
OpenSSL:

1) Fix extremely misleading text in crypto.4 manual page so it does not
appear to claim that a new cloned file descriptor is required for every
session.

2) Fix severe performance problem (and fd leak!) in openssl cryptodev
engine resulting from misunderstanding probably caused by said manual
page text.

3) Check for session-ID wraparound in kernel cryptodev provider. Also,
start allocating sessions at 1, not 0 -- this will be necessary when
we add ioctls for the creation of multiple sessions at once, so we
can tell which if any creations failed.
 1.27 19-Jan-2008  tls Add constants for modular arithmetic operations other than exponentiation -- there's hardware out there which can do them.
 1.26 04-Mar-2007  christos branches: 1.26.16; 1.26.22; 1.26.28;
Kill caddr_t; there will be some MI fallout, but it will be fixed shortly.
 1.25 16-Nov-2006  christos branches: 1.25.4;
__unused removal on arguments; approved by core.
 1.24 12-Oct-2006  christos - sprinkle __unused on function decls.
- fix a couple of unused bugs
- no more -Wno-unused for i386
 1.23 29-Aug-2006  christos branches: 1.23.2; 1.23.4;
fix incomplete initializer
 1.22 23-Jul-2006  ad Use the LWP cached credentials where sane.
 1.21 14-May-2006  elad integrate kauth.
 1.20 04-Apr-2006  christos Coverity CID 1083: Avoid possible NULL pointer deref.
 1.19 17-Mar-2006  christos don't use MALLOC with a non-constant size; use malloc instead.
 1.18 06-Mar-2006  christos branches: 1.18.2; 1.18.4;
sprinkle DPRINTF()...
 1.17 01-Mar-2006  yamt branches: 1.17.2;
merge yamt-uio_vmspace branch.

- use vmspace rather than proc or lwp where appropriate.
the latter is more natural to specify an address space.
(and less likely to be abused for random purposes.)
- fix a swdmover race.
 1.16 11-Dec-2005  christos branches: 1.16.2; 1.16.4; 1.16.6;
merge ktrace-lwp.
 1.15 25-Nov-2005  thorpej - De-couple the software crypto implementation from the rest of the
framework. There is no need to waste the space if you are only using
algoritms provided by hardware accelerators. To get the software
implementations, add "pseudo-device swcr" to your kernel config.
- Lazily initialize the opencrypto framework when crypto drivers
(either hardware or swcr) register themselves with the framework.
 1.14 22-Aug-2005  jonathan branches: 1.14.6;
No change. Forced commit to record commit message for previous revision, viz:

Fix vulnerability to a denial-of-service attack which passes a
length-0 crypto op. Check for zero length and return EINVAL, taken from:

http://cvsweb.FreeBSD.org/src/sys/opencrypto/cryptodev.c.diff?r1=1.25&r2=1.26

Original FreeBSD log mesage:

Modified files:
sys/opencrypto cryptodev.c
Log:
Fix bogus check. It was possible to panic the kernel by giving 0 length.
This is actually a local DoS, as every user can use /dev/crypto if there
is crypto hardware in the system and cryptodev.ko is loaded (or compiled
into the kernel).

Reported by: Mike Tancsa <mike@sentex.net>


thanks to Sam Leffler for passing on a heads-up about this issue.
 1.13 22-Aug-2005  jonathan *** empty log message ***
 1.12 30-Nov-2004  christos branches: 1.12.12;
Cloning cleanup:
1. make fileops const
2. add 2 new negative errno's to `officially' support the cloning hack:
- EDUPFD (used to overload ENODEV)
- EMOVEFD (used to overload ENXIO)
3. Created an fdclone() function to encapsulate the operations needed for
EMOVEFD, and made all cloners use it.
4. Centralize the local noop/badop fileops functions to:
fnullop_fcntl, fnullop_poll, fnullop_kqfilter, fbadop_stat
 1.11 17-Sep-2004  skrll There's no need to pass a proc value when using UIO_SYSSPACE with
vn_rdwr(9) and uiomove(9).

OK'd by Jason Thorpe
 1.10 19-Nov-2003  jonathan branches: 1.10.4;
Wrap noisy pointless message about denied userspace requests with
`#ifdef CRYPTO_DEBUG', per Jason Thorpe's suggestion.
 1.9 19-Nov-2003  jonathan Clean up userlevel access to software kernel transforms, in preparation
for using /dev/crypto for OpenSSL:

1. Add comments explaining crypto_devallowsoft, explaining the
OpenBSD-style three-way logic actully implemented in crypto_newsession().

2. Pass crypto_devallowsoft as the final argument to crypto_newsession(),
instead of a constant 0 value.

3. Set the default value of crypto_devallowsoft to 1, to allow
/dev/crypto access only for hardware-supported transforms.

Items 1-3 may be revised to match the FreeBSD two-way logic, if the
consensus is that there's no point to forcing software transforms.
But as a first step, let the description match what the code actually does.

GC unused variables usercrypto, userasmcrypto, cryptodevallowsoft from
cryptodev.c, in favour of variables crypto_usercrypto, crypto_userasmcrypto,
crypto_devallowsoft, which are used as well as defined in crypto.c.
 1.8 16-Nov-2003  jonathan Remove '#ifdef notdef' around userspace ioctl() requests for
pure (non-HMAC) MD5 and SHA1.
 1.7 26-Aug-2003  thorpej Remove a bunch of unnecessary includes.
 1.6 25-Aug-2003  thorpej It's bad form to use the <opencrypto/rmd160.h> header file while
using the crypto/ripemd160/rmd160.c implementation. Remove the
opencrypto-local copies of these files entirely.
 1.5 22-Aug-2003  itojun on netbsd, major # for /dev/crypto depends on arch
 1.4 21-Aug-2003  jonathan Pull up `done' flag for crypto operations from FreeBSD. FreeBSD deltas:
cryptodev.c: 1.4.2.3 -> 1.4.2.4
cryptodev.h: 1.4.2.4 -> 1.4.2.5
 1.3 30-Jul-2003  jonathan Garbage-collect references to OpenBSD-only <dev/rndvar.h>.
 1.2 28-Jul-2003  jonathan Remove vestiges of OpenBSD <sys/md5k.h> header.
 1.1 25-Jul-2003  jonathan Commit initial NetBSD port of the OpenCrypto Framework (OCF). This
code is derived from Sam Leffler's FreeBSD port of OCF, which is in
turn a port of Angelos Keromytis's OpenBSD work.
Credit to Sam and Angelos, any blame for the NetBSD port to me.
 1.10.4.8 11-Dec-2005  christos Sync with head.
 1.10.4.7 10-Nov-2005  skrll Sync with HEAD. Here we go again...
 1.10.4.6 08-Mar-2005  skrll Sync with HEAD.
 1.10.4.5 18-Dec-2004  skrll Sync with HEAD.
 1.10.4.4 21-Sep-2004  skrll Fix the sync with head I botched.
 1.10.4.3 18-Sep-2004  skrll Sync with HEAD.
 1.10.4.2 03-Aug-2004  skrll Sync with HEAD
 1.10.4.1 19-Nov-2003  skrll file cryptodev.c was added on branch ktrace-lwp on 2004-08-03 10:56:25 +0000
 1.12.12.7 24-Mar-2008  yamt sync with head.
 1.12.12.6 11-Feb-2008  yamt sync with head.
 1.12.12.5 04-Feb-2008  yamt sync with head.
 1.12.12.4 21-Jan-2008  yamt sync with head
 1.12.12.3 03-Sep-2007  yamt sync with head.
 1.12.12.2 30-Dec-2006  yamt sync with head.
 1.12.12.1 21-Jun-2006  yamt sync with head.
 1.14.6.1 29-Nov-2005  yamt sync with head.
 1.16.6.2 01-Jun-2006  kardel Sync with head.
 1.16.6.1 22-Apr-2006  simonb Sync with head.
 1.16.4.1 09-Sep-2006  rpaulo sync with head
 1.16.2.1 02-Feb-2006  yamt adapt opencrypto.
 1.17.2.6 03-Sep-2006  yamt sync with head.
 1.17.2.5 11-Aug-2006  yamt sync with head
 1.17.2.4 24-May-2006  yamt sync with head.
 1.17.2.3 11-Apr-2006  yamt sync with head
 1.17.2.2 01-Apr-2006  yamt sync with head.
 1.17.2.1 13-Mar-2006  yamt sync with head.
 1.18.4.2 24-May-2006  tron Merge 2006-05-24 NetBSD-current into the "peter-altq" branch.
 1.18.4.1 28-Mar-2006  tron Merge 2006-03-28 NetBSD-current into the "peter-altq" branch.
 1.18.2.3 06-May-2006  christos - Move kauth_cred_t declaration to <sys/types.h>
- Cleanup struct ucred; forward declarations that are unused.
- Don't include <sys/kauth.h> in any header, but include it in the c files
that need it.

Approved by core.
 1.18.2.2 19-Apr-2006  elad sync with head.
 1.18.2.1 08-Mar-2006  elad Adapt to kernel authorization changes.
 1.23.4.2 10-Dec-2006  yamt sync with head.
 1.23.4.1 22-Oct-2006  yamt sync with head
 1.23.2.1 18-Nov-2006  ad Sync with head.
 1.25.4.1 12-Mar-2007  rmind Sync with HEAD.
 1.26.28.1 19-Jan-2008  bouyer Sync with HEAD
 1.26.22.1 18-Feb-2008  mjf Sync with HEAD.
 1.26.16.1 23-Mar-2008  matt sync with HEAD
 1.34.6.4 17-Jan-2009  mjf Sync with HEAD.
 1.34.6.3 02-Jun-2008  mjf Sync with HEAD.
 1.34.6.2 05-Apr-2008  mjf - add "file-system DEVFS" and "pseudo-device devfsctl" to conf/std seeing
as these are always needed.

- convert many, many drivers over to the New Devfs World Order. For a
list of device drivers yet to be converted see,
http://www.netbsd.org/~mjf/devfs-todo.html.

- add a new device_unregister_all(device_t) function to remove all device
names associated with a device_t, which saves us having to construct
device names when the driver is detached.

- add a DEV_AUDIO type for devices.
 1.34.6.1 03-Apr-2008  mjf Sync with HEAD.
 1.38.2.2 04-Jun-2008  yamt sync with head
 1.38.2.1 18-May-2008  yamt sync with head.
 1.39.2.4 11-Aug-2010  yamt sync with head.
 1.39.2.3 11-Mar-2010  yamt sync with head
 1.39.2.2 04-May-2009  yamt sync with head.
 1.39.2.1 16-May-2008  yamt sync with head.
 1.41.2.1 23-Jun-2008  wrstuden Sync w/ -current. 34 merge conflicts to follow.
 1.44.8.4 14-Feb-2010  bouyer Pull up following revision(s) (requested by hubertf in ticket #1291):
sys/opencrypto/cryptodev.c: revision 1.52
Add missing "break" for CRYPTO_CAST_CBC, and some assorted comment fixes.
openssl(1) checks for CAST (and others) on ~every startup.
 1.44.8.3 03-May-2009  snj Pull up following revision(s) (requested by tls in ticket #611):
sys/lib/libkern/Makefile: patch
sys/lib/libkern/crc32.c: revision 1.1
sys/lib/libkern/crc32.h: revision 1.1
sys/lib/libkern/libkern.h: revision 1.89
sys/lib/libkern/arch/i386/Makefile.inc: revision 1.28
sys/net/zlib.h: revision 1.14 via patch
sys/opencrypto/crypto.c: revision 1.33
sys/opencrypto/cryptodev.c: revision 1.46
sys/opencrypto/cryptodev.h: revision 1.16
sys/opencrypto/cryptosoft.c: revision 1.24
sys/opencrypto/cryptosoft.h: revision 1.6
sys/opencrypto/deflate.h: revision 1.6
sys/opencrypto/cryptosoft_xform.c: revision 1.12
sys/opencrypto/deflate.c: revision 1.13
sys/opencrypto/files.opencrypto: revision 1.20
sys/opencrypto/ocryptodev.c: revision 1.1
sys/opencrypto/ocryptodev.h: revision 1.1
sys/opencrypto/xform.c: revision 1.18
sys/opencrypto/xform.h: revision 1.10
Fixes PR kern/41069 and PR kern/41070.

Extends the Opencrypto API to allow the destination buffer size to be
specified when its not the same size as the input buffer (i.e. for
operations like compress and decompress).
The crypto_op and crypt_n_op structures gain a u_int dst_len field.
The session_op structure gains a comp_alg field to specify a compression
algorithm.
Moved four ioctls to new ids; CIOCGSESSION, CIOCNGSESSION, CIOCCRYPT,
and CIOCNCRYPTM.
Added four backward compatible ioctls; OCIOCGSESSION, OCIOCNGSESSION,
OCIOCCRYPT, and OCIOCNCRYPTM.

Backward compatibility is maintained in ocryptodev.h and ocryptodev.c which
implement the original ioctls and set dst_len and comp_alg to 0.

Adds user-space access to compression features.

Adds software gzip support (CRYPTO_GZIP_COMP).

Adds the fast version of crc32 from zlib to libkern. This should be generally
useful and provide a place to start normalizing the various crc32 routines
in the kernel. The crc32 routine is used in this patch to support GZIP.

With input and support from tls@NetBSD.org.
 1.44.8.2 04-Apr-2009  snj branches: 1.44.8.2.4;
Pull up following revision(s) (requested by ad in ticket #661):
sys/arch/xen/xen/xenevt.c: revision 1.32
sys/compat/svr4/svr4_net.c: revision 1.56
sys/compat/svr4_32/svr4_32_net.c: revision 1.19
sys/dev/dmover/dmover_io.c: revision 1.32
sys/dev/putter/putter.c: revision 1.21
sys/kern/kern_descrip.c: revision 1.190
sys/kern/kern_drvctl.c: revision 1.23
sys/kern/kern_event.c: revision 1.64
sys/kern/sys_mqueue.c: revision 1.14
sys/kern/sys_pipe.c: revision 1.109
sys/kern/sys_socket.c: revision 1.59
sys/kern/uipc_syscalls.c: revision 1.136
sys/kern/vfs_vnops.c: revision 1.164
sys/kern/uipc_socket.c: revision 1.188
sys/net/bpf.c: revision 1.144
sys/net/if_tap.c: revision 1.55
sys/opencrypto/cryptodev.c: revision 1.47
sys/sys/file.h: revision 1.67
sys/sys/param.h: patch
sys/sys/socketvar.h: revision 1.119
Add fileops::fo_drain(), to be called from fd_close() when there is more
than one active reference to a file descriptor. It should dislodge threads
sleeping while holding a reference to the descriptor. Implemented only for
sockets but should be extended to pipes, fifos, etc.
Fixes the case of a multithreaded process doing something like the
following, which would have hung until the process got a signal.
thr0 accept(fd, ...)
thr1 close(fd)
 1.44.8.1 20-Nov-2008  snj Pull up following revision(s) (requested by darran in ticket #92):
sys/opencrypto/cryptodev.c: revision 1.45
sys/sys/queue.h: revision 1.50
sys/opencrypto/cryptodev.h: revision 1.15
sys/opencrypto/crypto.c: revision 1.30
Fix a race condition in opencrypto where the crypto request could be
completed by the crypto device, queued on the retq, but freed by the
ioctl lwp. The problem manifests as various panics relating to the
condvar inside the request. The problem can occur whenever the crypto
device completes the request immediately and the ioctl skips the
cv_wait().
The problem can be reproduced by enabling cryptosoft and running an
openssl
speed test. E.g.
sysctl -w kern.cryptodevallowsoft=-1
openssl speed -engine cryptodev -evp des-ede3-cbc -multi 64
Add a macro for TAILQ_FOREACH_REVERSE_SAFE() to queue.h, since this
was missing and the opencrypto code removes requests from a list while
iterating with TAILQ_FOREACH_REVERSE().
Add missing cv_destroy() calls for the key request cleanup.
Reviewed by Thor Lancelot Simon.
 1.44.8.2.4.1 20-May-2011  matt bring matt-nb5-mips64 up to date with netbsd-5-1-RELEASE (except compat).
 1.44.6.2 28-Apr-2009  skrll Sync with HEAD.
 1.44.6.1 19-Jan-2009  skrll Sync with HEAD.
 1.44.4.1 13-Dec-2008  haad Update haad-dm branch to haad-dm-base2.
 1.45.4.1 13-May-2009  jym Sync with HEAD.

Commit is split, to avoid a "too many arguments" protocol error.
 1.52.4.3 12-Jun-2011  rmind sync with head
 1.52.4.2 31-May-2011  rmind sync with head
 1.52.4.1 05-Mar-2011  rmind sync with head
 1.52.2.1 17-Aug-2010  uebayasi Sync with HEAD.
 1.53.4.1 05-Mar-2011  bouyer Sync with HEAD
 1.53.2.1 06-Jun-2011  jruoho Sync with HEAD.
 1.66.2.1 23-Jun-2011  cherry Catchup with rmind-uvmplock merge.
 1.68.16.1 18-May-2014  rmind sync with head
 1.68.12.2 03-Dec-2017  jdolecek update from HEAD
 1.68.12.1 20-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.68.2.1 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.78.2.1 10-Aug-2014  tls Rebase.
 1.82.2.4 28-Aug-2017  skrll Sync with HEAD
 1.82.2.3 09-Jul-2016  skrll Sync with HEAD
 1.82.2.2 22-Sep-2015  skrll Sync with HEAD
 1.82.2.1 06-Apr-2015  skrll Sync with HEAD
 1.85.4.1 21-Apr-2017  bouyer Sync with HEAD
 1.85.2.4 26-Apr-2017  pgoyette Sync with HEAD
 1.85.2.3 26-Jul-2016  pgoyette Rename LOCALCOUNT_INITIALIZER to DEVSW_MODULE_INIT. This better describes
what we're doing, and why.
 1.85.2.2 19-Jul-2016  pgoyette Instead of repeatedly typing the conditional initialization of the
.d_localcount members in the various {b,c}devsw, define an initializer
macro and use it. This also removes the need for defining new symbols
for each 'struct localcount'.

As suggested by riastradh@
 1.85.2.1 18-Jul-2016  pgoyette Rump drivers are always installed via devsw_attach() so we need to
always allocate a 'struct localcount' for these drivers whenever they
are built as modules.
 1.89.2.4 19-May-2017  pgoyette Resolve conflicts from previous merge (all resulting from $NetBSD
keywork expansion)
 1.89.2.3 17-May-2017  pgoyette At suggestion of chuq@, modify config_attach_pseudo() to return with a
reference held on the device.

Adapt callers to expect the reference to exist, and to ensure that the
reference is released.
 1.89.2.2 29-Apr-2017  pgoyette Remove more unnecessary #include for sys/localcount.h
 1.89.2.1 27-Apr-2017  pgoyette Restore all work from the former pgoyette-localcount branch (which is
now abandoned doe to cvs merge botch).

The branch now builds, and installs via anita. There are still some
problems (cgd is non-functional and all atf tests time-out) but they
will get resolved soon.
 1.92.2.1 05-Jul-2017  snj Pull up following revision(s) (requested by knakahara in ticket #97):
sys/opencrypto/crypto.c: 1.87-1.91
sys/opencrypto/cryptodev.c: 1.93-1.95
sys/opencrypto/cryptodev.h: 1.37
sys/opencrypto/cryptosoft.c: 1.52
sys/rump/dev/lib/libopencrypto/opencrypto_component.c: 1.5
sanitize count used for kmem_alloc size.
Hmm, who uses CIOCNGSESSION, CIOCNFSESSION, CIOCNCRYPTM or CIOCNFKEYM?
--
sanitize in CIOCNCRYPTM and initialize comp_alg in CIOCNGSESSION
--
must release cap->cc_lock before calling cap->cc_newsession() because of spinlock.
--
refactor crypto_newsession() like FreeBSD.
--
support multiple encryption drivers (port from FreeBSD).
--
Divide crp_devflags from crp_flags to write exclusively.
CRYPTO_F_DQRETQ(new name is CRYPTODEV_F_RET) is used by cryptodev.c only.
It should be divided to other member.
--
Reduce crypto_ret_q_mtx lock regions.
crypto.c does not access the members of crp when the crp is in crp_q or
crp_ret_q. Furthermore, crp_q and crp_ret_q are protected by each mutex,
so the members of crp is not shared. That means crp_flags is not required
mutex in crypto.c.
--
fix cryptosoft.c:r1.51 mistake. swcrypto_attach() must not be called from module_init_class().
swcrypto_attach() will call softint_establish(), it must be called after cpus
attached. module_init_class() is too early to call softint_establish().
--
simplify mutex_enter/exit(crypto_q_mtx), and fix missing exit.
--
reduce rump waring message. pointed out by ozaki-r@n.o, thanks.
 1.98.4.4 21-Apr-2020  martin Sync with HEAD
 1.98.4.3 13-Apr-2020  martin Mostly merge changes from HEAD upto 20200411
 1.98.4.2 08-Apr-2020  martin Merge changes from current as of 20200406
 1.98.4.1 10-Jun-2019  christos Sync with HEAD
 1.98.2.11 22-Jan-2019  pgoyette Convert the MODULE_{,VOID_}HOOK_CALL macros to do everything in-line
rather than defining an intermediate hook##call function. Almost
all of the hooks are called only once, and although we lose the
ability of doing things like

if (MODULE_HOOK_CALL(...) == 0) ...

we simplify things quite a bit. With this change, we no longer need
to have both declaration and definition macros, and the definition
no longer needs to have both prototype argument list and a "real"
argument list.

FWIW, the above if now needs to written as

int ret;

MODULE_HOOK_CALL(..., ret);
if (ret == 0) ...

with appropriate use of braces {}.
 1.98.2.10 18-Jan-2019  pgoyette Don't restrict hooks to having only int or void types. Pass the hook's
type to the various macros, as needed.

Allows us to reduce diffs to original in at least one or two places (we
no longer have to provide an additional parameter to the hook routine
for returning a non-int return value).
 1.98.2.9 14-Jan-2019  pgoyette Create a variant of the HOOK macros that handles hook routines of
type void, and use them where appropriate.
 1.98.2.8 13-Jan-2019  pgoyette Remove the HOOK2 versions of the MODULE_HOOK macros. There were
only a few uses, and using them led to some lack of clarity in the
code. Instead, we now use two separate hooks, with names that
make it clear(er) what we're doing.

This also positions us to start unraveling some of the rtsock_50
mess, which will need (at least) five hooks.
 1.98.2.7 29-Sep-2018  pgoyette In MODULE_HOOK_CALL_DECL we don't need to provide the actual argument
list for calling the hook function, nor do we need to provide the
default value (for when the hook has not been set).
 1.98.2.6 23-Sep-2018  pgoyette Split the compat_crypto_50 from the rest of the crypto module

Cleanup some stuff left over from similar changes to raid modules.
 1.98.2.5 22-Sep-2018  pgoyette When the compat code needs to callback to the original code, we cannot
call directly via the routines' global symbols, since the original code
might not be built-in. So, the original code that calls compat code
needs to pass in the addresses of the callbacks. This allows for the
compat code to be built whether or not the original (calling) code is
included.

XXX Done for cryptodev, will need to do the same thing for ccd(4) and
XXX vnd(4)
 1.98.2.4 18-Sep-2018  pgoyette The COMPAT_HOOK macros were renamed to MODULE_HOOK, adjust all callers
 1.98.2.3 18-Sep-2018  pgoyette Split the COMPAT_CALL_HOOK to separate the declaration from the
implementation. Some hooks are called from multiple source files,
and the old method resulted in duplicate implementations.

Implement MP-safe hooks for the usb_subr_30 code. Pass the helper
functions as arguments to the compat code so it does not have to
determine if the kernel contains usb code.
 1.98.2.2 17-Sep-2018  pgoyette Adapt (most of) the indirect function pointers to the new MP-safe
mechanism. Still remaining are the compat_netbsd32 stuff, and
some usb subroutines.
 1.98.2.1 23-Mar-2018  pgoyette Handle the compat_50 stuff for opencrypto/cryptodev
 1.102.2.2 29-Feb-2020  ad Sync with head.
 1.102.2.1 17-Jan-2020  ad Sync with head.
 1.104.4.1 20-Apr-2020  bouyer Sync with HEAD
 1.125.10.1 02-Aug-2025  perseant Sync with HEAD
 1.51 11-Jul-2023  riastradh opencrypto/cryptodev.h: Fix includes.

- Move sys/condvar.h under #ifdef _KERNEL.
- Add some other necessary includes and forward declarations.
- Sort.
 1.50 22-May-2022  riastradh branches: 1.50.4;
opencrypto: crypto_dispatch never fails now. Make it return void.

Same with crypto_kdispatch.
 1.49 22-May-2022  riastradh opencrypto: Rip out EAGAIN logic when unregistering crypto drivers.

I'm pretty sure this never worked reliably based on code inspection,
and it's unlikely to have ever been tested because it only applies
when unregistering a driver -- but we have no crypto drivers for
removable devices, so it would only apply if we went out of our way
to trigger detach with drvctl.

Instead, just make the operation fail with ENODEV, and remove all the
callback logic to resubmit the request on EAGAIN. (Maybe this should
be ENXIO, but crypto_kdispatch already does ENODEV.)
 1.48 22-May-2022  riastradh opencrypto: Make crypto_freesession return void.

No callers use the return value. It is not sensible to allow this to
fail.
 1.47 22-May-2022  riastradh opencrypto: Make freesession callback return void.

No functional change intended: all drivers already return zero
unconditionally.
 1.46 22-May-2022  riastradh opencrypto: Make crp_callback, krp_callback return void.

Nothing uses the return values inside opencrypto, so let's stop
making users return them.
 1.45 22-May-2022  riastradh opencrypto: Nix CRYPTO_F_DONE.

Nothing uses it any more.
 1.44 22-May-2022  riastradh opencrypto: Make sid=0 always invalid, but OK to free.

Previously, crypto_newsession could sometimes return 0 as the
driver-specific part of the session id, and 0 as the hid, for sid=0.
But netipsec assumes that it is always safe to free sid=0 from
zero-initialized memory even if crypto_newsession has never
succeeded. So it was up to every driver in tree to gracefully handle
sid=0, if it happened to get assigned hid=0. And, as long as the
freesession callback was expected to just return an error code when
given a bogus session id, that worked out fine...because nothing ever
used the error code.

That was a terrible fragile system that should never have been
invented. Instead, let's just ensure that valid session ids are
nonzero, and make crypto_freesession with sid=0 be a no-op.
 1.43 19-May-2022  riastradh opencrypto: Nix CRYPTO_F_USER, CRYPTO_F_CBIMM, CRYPTO_F_CBIFSYNC.

CRYPTO_F_USER is no longer needed. It was introduced in 2008 by
darran@ in crypto.c 1.30, cryptodev.c 1.45 in an attempt to avoid
double-free between the issuing thread and asynchronous callback.
But the `fix' didn't work. In 2017, knakahara@ fixed it properly in
cryptodev.c 1.87 by distinguishing `the crypto operation has
completed' (CRYPTO_F_DONE) from `the callback is done touching the
crp object' (CRYPTO_F_DQRETQ, now renamed to CRYPTODEV_F_RET).

CRYPTO_F_CBIMM formerly served to invoke the callback synchronously
from the driver's interrupt completion routine, to reduce contention
on what was once a single cryptoret thread. Now, there is a per-CPU
queue and softint for much cheaper processing, so there is less
motivation for this in the first place. So let's remove the
complicated logic. This means the callbacks never run in hard
interrupt context, which means we don't need to worry about recursion
into crypto_dispatch in hard interrupt context.
 1.42 14-Aug-2021  andvar fix typo in CRK_ALGORITHM_MIN definition to match CRK_ALGORITHM_MAX one.
while here fix few typos in comments.
 1.41 09-Aug-2021  andvar fix typos in asymmetry, asymmetric(al), symmetrical.
 1.40 29-Nov-2019  hikaru HMAC-SHA-512 has 32 bytes MAC.
 1.39 26-Jul-2017  knakahara branches: 1.39.4;
update locking notes of opencrypto(9)
 1.38 18-Jul-2017  knakahara branches: 1.38.2;
make cryptoret() context softint to balance dequeuing crypto_ret_q with enqueuing it.
 1.37 15-Jun-2017  knakahara Divide crp_devflags from crp_flags to write exclusively.

CRYPTO_F_DQRETQ(new name is CRYPTODEV_F_RET) is used by cryptodev.c only.
It should be divided to other member.
 1.36 06-Jun-2017  knakahara add locking notes.
 1.35 06-Jun-2017  knakahara restructure locks(1/2): make relation between lock and data explicit.

+ crypto_drv_mtx protects
- whole crypto_drivers
+ crypto_drivers[i].cc_lock (new) protects
- crypto_drivers[i] itself
- member of crypto_drivers[i]
+ crypto_q_mtx protects
- crp_q
- crp_kq
+ crypto_ret_q_mtx protects
- crp_ret_q
- crp_ret_kq
- crypto_exit_flag

I will add locking note later.
 1.34 25-May-2017  knakahara branches: 1.34.2;
add cryptkop alloc/free KPI instead of manipulating cryptkop_pool directly.
 1.33 25-May-2017  knakahara remove obsoleted declarations.
 1.32 17-May-2017  knakahara opencrypto: cleanup debug messages.
 1.31 24-Apr-2017  knakahara branches: 1.31.2;
separate crypto_drv_mtx from crypto_mtx.

crypto_mtx is used only for cryptodev.c and ocryptodev.c now.
 1.30 07-Apr-2017  knakahara fix race among crypto_done(), cryptoret(), and {cryptodev_op(), cryptodev_key()}.

crypto_op() waited to be set CRYPTO_F_DONE with crp->crp_cv.
However, there is context switch chances between being set CRYPTO_F_DONE in
crypto_done() and done cv_signal(crp->crp_cv) in cryptodev_cb(), that is,
cryptodev_op() thread can run to cv_destroy(crp->crp_cv) before cryptoret()
thread is waken up. As a result, cryptodev_cb() can call invalid(destroyed)
cv_signal(crp->crp_cv).

Furthermore, below two implementations cause other races.
- waiting CRYPTO_F_DONE with crp->crp_cv
- context witch chances between set CRYPTO_F_DONE and cv_signal(crp->crp_cv)

So, use other flag(CRYPTO_F_DQRETQ) for cryptodev_op() and cryptodev_key(),
and then call cv_signal(crp->crp_cv) immediately after set CRYPTO_F_DQRETQ.

Tested concurrent over 20 processes with software and hardware drivers.
 1.29 06-Mar-2017  knakahara add sysctl to select software/hardware encryption driver. can enable CRYPTO_DEBUG.
 1.28 07-Jul-2016  msaitoh branches: 1.28.2; 1.28.4;
KNF. Remove extra spaces. No functional change.
 1.27 22-Jan-2016  dholland Needs sys/time.h for struct timespec.
 1.26 28-Nov-2015  pgoyette Re-work the module init and destroy code to allow it to be unloaded and
then reloaded.

Should fix PR kern/49842
 1.25 09-Jun-2011  drochner branches: 1.25.12; 1.25.30;
-if an opencrypto(9) session is allocated, the driver is refcounted
and can not disappear -- no need to hold crypto_mtx to check the
driver list
(the whole check is questionable)
-crp->crp_cv (the condition variable) is used by userland cryptodev
exclusively -- move its initialization there, no need to waste
cycles of in-kernel callers
-add a comment which members of "struct cryptop" are used
by opencrypto(9) and which by crypto(4)
(this should be split, no need to waste memory for in-kernel callers)
 1.24 26-May-2011  drochner branches: 1.24.2;
pull in AES-GCM/GMAC support from OpenBSD
This is still somewhat experimental. Tested between 2 similar boxes
so far. There is much potential for performance improvement. For now,
I've changed the gmac code to accept any data alignment, as the "char *"
pointer suggests. As the code is practically used, 32-bit alignment
can be assumed, at the cost of data copies. I don't know whether
bytewise access or copies are worse performance-wise. For efficient
implementations using SSE2 instructions on x86, even stricter
alignment requirements might arise.
 1.23 24-May-2011  drochner copy AES-XCBC-MAC support from KAME IPSEC to FAST_IPSEC
For this to fit, an API change in cryptosoft was adopted from OpenBSD
(addition of a "Setkey" method to hashes) which was done for GCM/GMAC
support there, so it might be useful in the future anyway.
tested against KAME IPSEC
AFAICT, FAST_IPSEC now supports as much as KAME.
 1.22 23-May-2011  drochner add an AES-CTR xform, from OpenBSD
 1.21 16-May-2011  drochner remove redundant declarations
 1.20 05-May-2011  drochner make camellia-cbc known to the opencrypto framework
 1.19 25-Feb-2011  drochner make the use of SHA2-HMAC by FAST_IPSEC compliant to current standards:
-RFC2104 says that the block size of the hash algorithm must be used
for key/ipad/opad calculations. While formerly all ciphers used a block
length of 64, SHA384 and SHA512 use 128 bytes. So we can't use the
HMAC_BLOCK_LEN constant anymore. Add a new field to "struct auth_hash"
for the per-cipher blocksize.
-Due to this, there can't be a single "CRYPTO_SHA2_HMAC" external name
anymore. Replace this by 3 for the 3 different keysizes.
This was done by Open/FreeBSD before.
-Also fix the number of authenticator bits used tor ESP and AH to
conform to RFC4868, and remove uses of AH_HMAC_HASHLEN which did
assume a fixed authenticator size of 12 bytes.

FAST_IPSEC will not interoperate with KAME IPSEC anymore if sha2 is used,
because the latter doesn't implement these standards. It should
interoperate with at least modern Free/OpenBSD now.
(I've only tested with NetBSD-current/FAST_IPSEC on both ends.)
 1.18 24-Feb-2011  drochner small modifications in dealing with the unknown result size of compression/
decompression:
-seperate the IPCOMP specific rule that compression must not grow the
data from general compression semantics: Introduce a special name
CRYPTO_DEFLATE_COMP_NOGROW/comp_algo_deflate_nogrow to describe
the IPCOMP semantics and use it there. (being here, fix the check
so that equal size is considered failure as well as required by
RFC2393)
Customers of CRYPTO_DEFLATE_COMP/comp_algo_deflate now always get
deflated data back, even if they are not smaller than the original.
-allow to pass a "size hint" to the DEFLATE decompression function
which is used for the initial buffer allocation. Due to the changes
done there, additional allocations and extra copies are avoided if the
initial allocation is sufficient. Set the size hint to MCLBYTES (=2k)
in IPCOMP which should be good for many use cases.
 1.17 04-Sep-2009  he branches: 1.17.4; 1.17.6; 1.17.8;
It is best to explicitly include <sys/condvar.h> if we're going to
use kcondvar_t.
 1.16 25-Mar-2009  darran Fixes PR kern/41069 and PR kern/41070.

Extends the Opencrypto API to allow the destination buffer size to be
specified when its not the same size as the input buffer (i.e. for
operations like compress and decompress).
The crypto_op and crypt_n_op structures gain a u_int dst_len field.
The session_op structure gains a comp_alg field to specify a compression
algorithm.
Moved four ioctls to new ids; CIOCGSESSION, CIOCNGSESSION, CIOCCRYPT,
and CIOCNCRYPTM.
Added four backward compatible ioctls; OCIOCGSESSION, OCIOCNGSESSION,
OCIOCCRYPT, and OCIOCNCRYPTM.

Backward compatibility is maintained in ocryptodev.h and ocryptodev.c which
implement the original ioctls and set dst_len and comp_alg to 0.

Adds user-space access to compression features.

Adds software gzip support (CRYPTO_GZIP_COMP).

Adds the fast version of crc32 from zlib to libkern. This should be generally
useful and provide a place to start normalizing the various crc32 routines
in the kernel. The crc32 routine is used in this patch to support GZIP.

With input and support from tls@NetBSD.org.
 1.15 18-Nov-2008  darran branches: 1.15.4;
Fix a race condition in opencrypto where the crypto request could be
completed by the crypto device, queued on the retq, but freed by the
ioctl lwp. The problem manifests as various panics relating to the
condvar inside the request. The problem can occur whenever the crypto
device completes the request immediately and the ioctl skips the cv_wait().

The problem can be reproduced by enabling cryptosoft and running an openssl
speed test. E.g.
sysctl -w kern.cryptodevallowsoft=-1
openssl speed -engine cryptodev -evp des-ede3-cbc -multi 64

Add a macro for TAILQ_FOREACH_REVERSE_SAFE() to queue.h, since this
was missing and the opencrypto code removes requests from a list while
iterating with TAILQ_FOREACH_REVERSE().

Add missing cv_destroy() calls for the key request cleanup.

Reviewed by Thor Lancelot Simon.
 1.14 28-Apr-2008  martin branches: 1.14.6; 1.14.8; 1.14.10;
Remove clause 3 and 4 from TNF licenses
 1.13 10-Apr-2008  tls branches: 1.13.2; 1.13.4;
Extend crypto.4 interface:

* Asynchronous operation with result retrieval via select/poll
* Mutliple-request submit/retrieve ioctls
* Mutliple-session create-destroy ioctls

Revise/rewrite crypto.4 manual page. It should now be much easier to write
new applications to this API.

Measured performance for trivial requests: 84,000 very short modular math
operations/sec, 120,000 very short md5 hashes per sec (with a hardware
accellerator of moderate performance but very low latency, whose driver
will be contributed at a later date).

Contributed to TNF by Coyote Point Systems, Inc.
 1.12 04-Feb-2008  tls branches: 1.12.6;
Rework opencrypto to use a spin mutex (crypto_mtx) instead of "splcrypto"
(actually splnet) and condvars instead of tsleep/wakeup. Fix a few
miscellaneous problems and add some debugging printfs while there.

Restore set of CRYPTO_F_DONE in crypto_done() which was lost at some
point after this code came from FreeBSD -- it made it impossible to wait
properly for a condition.

Add flags analogous to the "crp" flags to the key operation's krp struct.
Add a new flag, CRYPTO_F_ONRETQ which tells us a request finished before
the kthread had a chance to dequeue it and call its callback -- this was
letting requests stick on the queues before even though done and copied
out.

Callers of crypto_newsession() or crypto_freesession() must now take the
mutex. Change netipsec to do so. Dispatch takes the mutex itself as
needed.

This was tested fairly extensively with the cryptosoft backend and lightly
with a new hardware driver. It has not been tested with FAST_IPSEC; I am
unable to ascertain whether FAST_IPSEC currently works at all in our tree.

pjd@FreeBSD.ORG, ad@NetBSD.ORG, and darran@snark.us pointed me in the
right direction several times in the course of this. Remaining bugs
are mine alone.
 1.11 02-Feb-2008  tls From Darran Hunt at Coyote Point: don't truncate HMAC to 96 bits unless
actually asked to.

Fixed in FreeBSD a while ago, discussed on tech-kern and tech-crypto.
 1.10 01-Feb-2008  tls This code never worked on a released version of FreeBSD in the form it's
been in in our tree, and certainly does not work on any version of FreeBSD
now. Run through unifdef -D__NetBSD__ -U__FreeBSD__ yielding a small
reduction of size and a dramatic improvement in readability.

No, this does not yield any meaningful decrease in patchability (unlike
mechanical changes that touch live source lines) -- try it and see.
 1.9 19-Jan-2008  tls Add constants for modular arithmetic operations other than exponentiation -- there's hardware out there which can do them.
 1.8 04-Mar-2007  christos branches: 1.8.16; 1.8.22; 1.8.28;
Kill caddr_t; there will be some MI fallout, but it will be fixed shortly.
 1.7 25-Nov-2005  thorpej branches: 1.7.26;
- De-couple the software crypto implementation from the rest of the
framework. There is no need to waste the space if you are only using
algoritms provided by hardware accelerators. To get the software
implementations, add "pseudo-device swcr" to your kernel config.
- Lazily initialize the opencrypto framework when crypto drivers
(either hardware or swcr) register themselves with the framework.
 1.6 26-Feb-2005  perry branches: 1.6.4; 1.6.10;
nuke trailing whitespace
 1.5 21-Aug-2003  jonathan branches: 1.5.4; 1.5.10; 1.5.12;
Pull in FreeBSD sys/opencrypto/cryptodev.h, revision 1.2.2.5 -> 1.2.2.6.
Sam Leffler's FreeBSD commit message was
``to eliminate context switch when returning results from the
software crypto driver''
but the patch also contains the CRYPTO_SESID*() macros used in newer
ubsec and hifn drivers.
 1.4 21-Aug-2003  jonathan Pull up `done' flag for crypto operations from FreeBSD. FreeBSD deltas:
cryptodev.c: 1.4.2.3 -> 1.4.2.4
cryptodev.h: 1.4.2.4 -> 1.4.2.5
 1.3 30-Jul-2003  jonathan Move the initialization of the crypto framework from the userland
pseudo-device to init_main(), so the framework is ready for
registration requests at autoconfiguration time.

Thanks to Quentin Garnier for confirming the change was required, and
for testing a similar fix.
 1.2 27-Jul-2003  jonathan Cleanup traces of previous standalone m_apply()/m_getptr().
 1.1 25-Jul-2003  jonathan Commit initial NetBSD port of the OpenCrypto Framework (OCF). This
code is derived from Sam Leffler's FreeBSD port of OCF, which is in
turn a port of Angelos Keromytis's OpenBSD work.
Credit to Sam and Angelos, any blame for the NetBSD port to me.
 1.5.12.1 19-Mar-2005  yamt sync with head. xen and whitespace. xen part is not finished.
 1.5.10.1 29-Apr-2005  kent sync with -current
 1.5.4.6 11-Dec-2005  christos Sync with head.
 1.5.4.5 04-Mar-2005  skrll Sync with HEAD.

Hi Perry!
 1.5.4.4 21-Sep-2004  skrll Fix the sync with head I botched.
 1.5.4.3 18-Sep-2004  skrll Sync with HEAD.
 1.5.4.2 03-Aug-2004  skrll Sync with HEAD
 1.5.4.1 21-Aug-2003  skrll file cryptodev.h was added on branch ktrace-lwp on 2004-08-03 10:56:25 +0000
 1.6.10.1 29-Nov-2005  yamt sync with head.
 1.6.4.4 04-Feb-2008  yamt sync with head.
 1.6.4.3 21-Jan-2008  yamt sync with head
 1.6.4.2 03-Sep-2007  yamt sync with head.
 1.6.4.1 21-Jun-2006  yamt sync with head.
 1.7.26.1 12-Mar-2007  rmind Sync with HEAD.
 1.8.28.1 19-Jan-2008  bouyer Sync with HEAD
 1.8.22.1 18-Feb-2008  mjf Sync with HEAD.
 1.8.16.1 23-Mar-2008  matt sync with HEAD
 1.12.6.2 17-Jan-2009  mjf Sync with HEAD.
 1.12.6.1 02-Jun-2008  mjf Sync with HEAD.
 1.13.4.3 16-Sep-2009  yamt sync with head
 1.13.4.2 04-May-2009  yamt sync with head.
 1.13.4.1 16-May-2008  yamt sync with head.
 1.13.2.1 18-May-2008  yamt sync with head.
 1.14.10.2 03-May-2009  snj Pull up following revision(s) (requested by tls in ticket #611):
sys/lib/libkern/Makefile: patch
sys/lib/libkern/crc32.c: revision 1.1
sys/lib/libkern/crc32.h: revision 1.1
sys/lib/libkern/libkern.h: revision 1.89
sys/lib/libkern/arch/i386/Makefile.inc: revision 1.28
sys/net/zlib.h: revision 1.14 via patch
sys/opencrypto/crypto.c: revision 1.33
sys/opencrypto/cryptodev.c: revision 1.46
sys/opencrypto/cryptodev.h: revision 1.16
sys/opencrypto/cryptosoft.c: revision 1.24
sys/opencrypto/cryptosoft.h: revision 1.6
sys/opencrypto/deflate.h: revision 1.6
sys/opencrypto/cryptosoft_xform.c: revision 1.12
sys/opencrypto/deflate.c: revision 1.13
sys/opencrypto/files.opencrypto: revision 1.20
sys/opencrypto/ocryptodev.c: revision 1.1
sys/opencrypto/ocryptodev.h: revision 1.1
sys/opencrypto/xform.c: revision 1.18
sys/opencrypto/xform.h: revision 1.10
Fixes PR kern/41069 and PR kern/41070.

Extends the Opencrypto API to allow the destination buffer size to be
specified when its not the same size as the input buffer (i.e. for
operations like compress and decompress).
The crypto_op and crypt_n_op structures gain a u_int dst_len field.
The session_op structure gains a comp_alg field to specify a compression
algorithm.
Moved four ioctls to new ids; CIOCGSESSION, CIOCNGSESSION, CIOCCRYPT,
and CIOCNCRYPTM.
Added four backward compatible ioctls; OCIOCGSESSION, OCIOCNGSESSION,
OCIOCCRYPT, and OCIOCNCRYPTM.

Backward compatibility is maintained in ocryptodev.h and ocryptodev.c which
implement the original ioctls and set dst_len and comp_alg to 0.

Adds user-space access to compression features.

Adds software gzip support (CRYPTO_GZIP_COMP).

Adds the fast version of crc32 from zlib to libkern. This should be generally
useful and provide a place to start normalizing the various crc32 routines
in the kernel. The crc32 routine is used in this patch to support GZIP.

With input and support from tls@NetBSD.org.
 1.14.10.1 20-Nov-2008  snj branches: 1.14.10.1.4;
Pull up following revision(s) (requested by darran in ticket #92):
sys/opencrypto/cryptodev.c: revision 1.45
sys/sys/queue.h: revision 1.50
sys/opencrypto/cryptodev.h: revision 1.15
sys/opencrypto/crypto.c: revision 1.30
Fix a race condition in opencrypto where the crypto request could be
completed by the crypto device, queued on the retq, but freed by the
ioctl lwp. The problem manifests as various panics relating to the
condvar inside the request. The problem can occur whenever the crypto
device completes the request immediately and the ioctl skips the
cv_wait().
The problem can be reproduced by enabling cryptosoft and running an
openssl
speed test. E.g.
sysctl -w kern.cryptodevallowsoft=-1
openssl speed -engine cryptodev -evp des-ede3-cbc -multi 64
Add a macro for TAILQ_FOREACH_REVERSE_SAFE() to queue.h, since this
was missing and the opencrypto code removes requests from a list while
iterating with TAILQ_FOREACH_REVERSE().
Add missing cv_destroy() calls for the key request cleanup.
Reviewed by Thor Lancelot Simon.
 1.14.10.1.4.1 20-May-2011  matt bring matt-nb5-mips64 up to date with netbsd-5-1-RELEASE (except compat).
 1.14.8.2 28-Apr-2009  skrll Sync with HEAD.
 1.14.8.1 19-Jan-2009  skrll Sync with HEAD.
 1.14.6.1 13-Dec-2008  haad Update haad-dm branch to haad-dm-base2.
 1.15.4.1 13-May-2009  jym Sync with HEAD.

Commit is split, to avoid a "too many arguments" protocol error.
 1.17.8.1 05-Mar-2011  bouyer Sync with HEAD
 1.17.6.1 06-Jun-2011  jruoho Sync with HEAD.
 1.17.4.3 12-Jun-2011  rmind sync with head
 1.17.4.2 31-May-2011  rmind sync with head
 1.17.4.1 05-Mar-2011  rmind sync with head
 1.24.2.1 23-Jun-2011  cherry Catchup with rmind-uvmplock merge.
 1.25.30.4 28-Aug-2017  skrll Sync with HEAD
 1.25.30.3 09-Jul-2016  skrll Sync with HEAD
 1.25.30.2 19-Mar-2016  skrll Sync with HEAD
 1.25.30.1 27-Dec-2015  skrll Sync with HEAD (as of 26th Dec)
 1.25.12.1 03-Dec-2017  jdolecek update from HEAD
 1.28.4.1 21-Apr-2017  bouyer Sync with HEAD
 1.28.2.2 26-Apr-2017  pgoyette Sync with HEAD
 1.28.2.1 20-Mar-2017  pgoyette Sync with HEAD
 1.31.2.1 19-May-2017  pgoyette Resolve conflicts from previous merge (all resulting from $NetBSD
keywork expansion)
 1.34.2.3 05-Aug-2017  snj Pull up following revision(s) (requested by knakahara in ticket #178):
sys/opencrypto/crypto.c: 1.92-1.100
sys/opencrypto/cryptodev.h: 1.38-1.39
sys/opencrypto/ocryptodev.c: 1.9-1.11
sys/opencrypto/xform.c: revision 1.29
sys/opencrypto/xform.h: revision 1.20
KNF
--
Apply C99-style struct initialization to enc_xform, auth_hash and comp_algo
--
make cryptoret() context softint to balance dequeuing crypto_ret_q with enqueuing it.
--
fix panic when using ecryption devices attached earlier than ipi_sysinit().
pointed out and tested by martin@n.o, thanks.
--
fix typo
--
make crp_{,k}q percpu to scale crypto_dispatch().
update locking note later.
--
divide crp_ret_{,k}q by CPU to avoid reordering.
update locking note later.
--
update locking notes of opencrypto(9)
--
Don't disclose uninitialized 32-bit word if cryptodev_session fails.
From Ilja Van Sprundel.
--
Avert userland-controlled integer overflow.
From Ilja Van Sprundel.
--
Avoid another userland-controlled integer overflow.
From Ilja Van Sprundel.
--
refactor: remove glue macros for FreeBSD code.
--
pack crypto_drivers variables to struct and add cacheline_aligned qualifier.
--
use kmem_alloc KPI instead of malloc KPI.
--
use pool_cache(9) instead of pool(9) as they can be called concurrently.
 1.34.2.2 05-Jul-2017  snj Pull up following revision(s) (requested by knakahara in ticket #97):
sys/opencrypto/crypto.c: 1.87-1.91
sys/opencrypto/cryptodev.c: 1.93-1.95
sys/opencrypto/cryptodev.h: 1.37
sys/opencrypto/cryptosoft.c: 1.52
sys/rump/dev/lib/libopencrypto/opencrypto_component.c: 1.5
sanitize count used for kmem_alloc size.
Hmm, who uses CIOCNGSESSION, CIOCNFSESSION, CIOCNCRYPTM or CIOCNFKEYM?
--
sanitize in CIOCNCRYPTM and initialize comp_alg in CIOCNGSESSION
--
must release cap->cc_lock before calling cap->cc_newsession() because of spinlock.
--
refactor crypto_newsession() like FreeBSD.
--
support multiple encryption drivers (port from FreeBSD).
--
Divide crp_devflags from crp_flags to write exclusively.
CRYPTO_F_DQRETQ(new name is CRYPTODEV_F_RET) is used by cryptodev.c only.
It should be divided to other member.
--
Reduce crypto_ret_q_mtx lock regions.
crypto.c does not access the members of crp when the crp is in crp_q or
crp_ret_q. Furthermore, crp_q and crp_ret_q are protected by each mutex,
so the members of crp is not shared. That means crp_flags is not required
mutex in crypto.c.
--
fix cryptosoft.c:r1.51 mistake. swcrypto_attach() must not be called from module_init_class().
swcrypto_attach() will call softint_establish(), it must be called after cpus
attached. module_init_class() is too early to call softint_establish().
--
simplify mutex_enter/exit(crypto_q_mtx), and fix missing exit.
--
reduce rump waring message. pointed out by ozaki-r@n.o, thanks.
 1.34.2.1 22-Jun-2017  snj Pull up following revision(s) (requested by christos/knakahara in ticket #37):
sys/opencrypto/crypto.c: 1.79-1.86
sys/opencrypto/cryptodev.h: 1.35, 1.36
use crypto_checkdriver_uninit() when it may touch uninitialized crypto_drivers.
--
fix reading crp_q without holding crypto_q_mtx
--
restructure locks(1/2): make relation between lock and data explicit.
+ crypto_drv_mtx protects
- whole crypto_drivers
+ crypto_drivers[i].cc_lock (new) protects
- crypto_drivers[i] itself
- member of crypto_drivers[i]
+ crypto_q_mtx protects
- crp_q
- crp_kq
+ crypto_ret_q_mtx protects
- crp_ret_q
- crp_ret_kq
- crypto_exit_flag
I will add locking note later.
--
restructure locks(2/2): crypto_q_mtx can be adaptive now.
--
add locking notes.
--
avoid crp_q reordering as hardware interrupts.
crypto_{,k}invoke() can be called with holding crp_q_mtx now.
--
apply the same fix as crypto.c:r1.83 for crypto_dispatch to crypto_kdispatch.
--
- acquire lock
- use c99 loop indexes
- initialize featp
--
Put back crypto_checkdriver(); use it when we need to make sure that we
get back a cryptocap that has been initialized.
 1.38.2.2 18-Jul-2017  knakahara 3212654
 1.38.2.1 18-Jul-2017  knakahara file cryptodev.h was added on branch perseant-stdc-iso10646 on 2017-07-18 06:01:37 +0000
 1.39.4.1 08-Apr-2020  martin Merge changes from current as of 20200406
 1.50.4.1 09-Aug-2023  martin Pull up following revision(s) (requested by maya in ticket #316):

sys/arch/m68k/include/mutex.h: revision 1.13
sys/arch/arm/include/cpu.h: revision 1.125
sys/arch/sun68k/include/intr.h: revision 1.21
sys/arch/arm/include/mutex.h: revision 1.28
sys/sys/rwlock.h: revision 1.18
sys/arch/powerpc/include/mutex.h: revision 1.7
sys/arch/arm/include/mutex.h: revision 1.29
sys/arch/powerpc/include/mutex.h: revision 1.8
sys/uvm/uvm_param.h: revision 1.42
sys/sys/ksem.h: revision 1.16
sys/arch/x86/include/mutex.h: revision 1.10
sys/sys/proc.h: revision 1.372
sys/sys/ksem.h: revision 1.17
sys/arch/ia64/include/mutex.h: revision 1.8
sys/arch/evbarm/include/intr.h: revision 1.29
sys/sys/lua.h: revision 1.9
sys/arch/next68k/include/intr.h: revision 1.23
sys/arch/ia64/include/mutex.h: revision 1.9
sys/arch/hp300/include/intr.h: revision 1.35
sys/arch/hp300/include/intr.h: revision 1.36
sys/arch/sparc/include/cpu.h: revision 1.111
sys/arch/hppa/include/mutex.h: revision 1.16
sys/arch/vax/include/intr.h: revision 1.31
sys/arch/hppa/include/mutex.h: revision 1.17
sys/arch/news68k/include/intr.h: revision 1.28
sys/arch/hppa/include/mutex.h: revision 1.18
sys/arch/hppa/include/intr.h: revision 1.3
sys/arch/hppa/include/mutex.h: revision 1.19
sys/arch/hppa/include/intr.h: revision 1.4
sys/sys/sched.h: revision 1.92
sys/opencrypto/cryptodev.h: revision 1.51
sys/arch/vax/include/mutex.h: revision 1.20
sys/arch/sparc64/include/mutex.h: revision 1.10
sys/arch/ia64/include/sapicvar.h: revision 1.2
sys/arch/riscv/include/mutex.h: revision 1.5
sys/arch/amiga/dev/grfabs_cc.c: revision 1.39
sys/external/bsd/drm2/include/linux/idr.h: revision 1.11
sys/arch/riscv/include/mutex.h: revision 1.6
sys/ddb/files.ddb: revision 1.16
sys/arch/mac68k/include/intr.h: revision 1.32
share/man/man4/ddb.4: revision 1.203
sys/ddb/db_command.c: revision 1.183
sys/arch/mips/include/mutex.h: revision 1.10
sys/ddb/db_command.c: revision 1.184
sys/arch/x68k/include/intr.h: revision 1.22
sys/arch/sparc/include/psl.h: revision 1.51
sys/arch/or1k/include/mutex.h: revision 1.4
sys/arch/mips/include/mutex.h: revision 1.11
sys/arch/arm/xscale/pxa2x0_intr.h: revision 1.16
sys/arch/sparc64/include/cpu.h: revision 1.134
sys/arch/sparc/include/psl.h: revision 1.52
sys/arch/or1k/include/mutex.h: revision 1.5
sys/arch/mvme68k/include/intr.h: revision 1.22
sys/arch/luna68k/include/intr.h: revision 1.16
external/cddl/osnet/sys/sys/kcondvar.h: revision 1.6
sys/arch/sparc/include/mutex.h: revision 1.12
sys/arch/sparc/include/mutex.h: revision 1.13
sys/arch/usermode/include/mutex.h: revision 1.5
sys/arch/usermode/include/mutex.h: revision 1.6
sys/kern/kern_core.c: revision 1.38
usr.sbin/crash/Makefile: revision 1.49
sys/arch/amiga/include/intr.h: revision 1.23
sys/arch/alpha/include/mutex.h: revision 1.12
sys/arch/alpha/include/mutex.h: revision 1.13
sys/arch/evbarm/lubbock/sacc_obio.c: revision 1.16
sys/ddb/ddb.h: revision 1.6
sys/arch/sparc64/include/mutex.h: revision 1.8
sys/arch/sh3/include/mutex.h: revision 1.12
sys/arch/evbarm/lubbock/sacc_obio.c: revision 1.17
sys/ddb/db_syncobj.c: revision 1.1
sys/arch/vax/include/mutex.h: revision 1.18
sys/arch/sparc64/include/psl.h: revision 1.63
sys/arch/sparc64/include/mutex.h: revision 1.9
sys/arch/sh3/include/mutex.h: revision 1.13
sys/arch/evbarm/lubbock/obio.c: revision 1.13
sys/arch/atari/include/intr.h: revision 1.23
sys/ddb/db_syncobj.c: revision 1.2
sys/arch/vax/include/mutex.h: revision 1.19
sys/arch/evbarm/g42xxeb/obio.c: revision 1.14
sys/arch/evbarm/g42xxeb/obio.c: revision 1.15
sys/arch/cesfic/include/intr.h: revision 1.14
sys/ddb/db_syncobj.h: revision 1.1
sys/arch/x86/include/cpu.h: revision 1.134
sys/arch/evbarm/g42xxeb/obio.c: revision 1.16
sys/arch/cesfic/include/intr.h: revision 1.15
sys/arch/arm/xscale/pxa2x0_intr.c: revision 1.26
sys/sys/cpu_data.h: revision 1.54
sys/arch/m68k/include/mutex.h: revision 1.12
sys/arch/ia64/acpi/madt.c: revision 1.6

sys/rwlock.h: Make this more self-contained for bool.

machine/mutex.h: Sprinkle includes so this can be used by crash(8).

ddb: New `show all tstiles' command.
Shows who's waiting for which locks and what the owner is up to.

Include psl.h for ipl_cookie_t if __MUTEX_PRIVATE

sys: Rip <sys/resourcevar.h> out of <uvm/uvm_param.h>.

And thus out of <sys/param.h>, which is exceedingly overused and
fragile and delenda est.

Should fix (some) issues with the recent inclusion of machine/lock.h
in various machine/mutex.h files.

arm/mutex.h: Need machine/intr.h, machine/lock.h.

For ipl_cookie_t and __cpu_simple_lock_t.
evbarm/intr.h: Define ipl_cookie_t before including ARM_INTR_IMPL.

Otherwise arm/mutex.h doesn't work, due to a cyclic dependency which
should really be fixed.
opencrypto/cryptodev.h: Fix includes.
- Move sys/condvar.h under #ifdef _KERNEL.
- Add some other necessary includes and forward declarations.
- Sort.

hp300/intr.h: Fix missing includes.
linux/idr.h: Need <sys/mutex.h> for kmutex_t.
amiga/intr.h: Don't define spl*() functions if !_KERNEL.

This is used by crash(8) now, and what's important is ipl_cookie_t.
cesfic/intr.h: Expose ipl_cookie_t to userland for crash(8).
cesfic/intr.h: Expose ipl_cookie_t to userland only with _KMEMUSER.

Probably not necessary but let's be a little more cautious about
this.

atari/intr.h: Expose ipl_cookie_t with _KMEMUSER for crash(8).

arm/cpu.h: Need sys/param.h for COHERENCY_UNIT.

Nix machine/param.h -- not meant to be used directly, pulled in by
sys/param.h.

Move the definition of ipl_cookie_t out of the kernel-only sections,
some _KMEMUSER applications need it.

ddb: Cast pointer to uintptr_t first before db_expr_t.

hppa/intr.h: Expose ipl_cookie_t to _KMEMUSER for crash(8).

luna68k/intr.h: Expose ipl_cookie_t to _KMEMUSER for crash(8).

mvme68k/intr.h: Expose ipl_cookie_t to _KMEMUSER for crash(8).

news68k/intr.h: Fix includes. Put some definitions under _KERNEL.

next68k/intr.h: Expose ipl_cookie_t to _KMEMUSER for crash(8).

sys/ksem.h: Hack around fstat(8) abuse of _KERNEL.

sun68k/intr.h: Expose ipl_cookie_t to _KMEMUSER for crash(8).

vax/intr.h: Expose ipl_cookie_t to _KMEMUSER for crash(8).

x68k/intr.h: Put functions under _KERNEL so crash(8) can use this.

Make ipl_cookie_t visible for _KMEMUSER userland applications.

fix editor mishap in previous

Explicitly include <sys/mutex.h> for kmutex_t.

Replace kmutex_t * (which may be undefined here) with struct kmutex *,
suggested by Taylor.

hp300/intr.h: Put most of this under #ifdef _KERNEL.
Only ipl_cookie_t really needs to be exposed now, for crash(8).

mac68k/intr.h: Expose ipl_cookie_t to _KMEMUSER for crash(8).
Make inclusion of sys/intr.h explicit for spl*.

fix hppa and vax builds.

machine/lock.h isn't necessary for __cpu_simple_lock_t, it's in
sys/types.h. avoids cpu_data.h vs sched.h include order issues.

move the hppa ipl_t typedef with the moved usage of it.
machine/mutex.h: Sprinkle sys/types.h, omit machine/lock.h.

Turns out machine/lock.h is not needed for __cpu_simple_lock_t, which
always comes from sys/types.h. And, really, sys/types.h (or at least
sys/stdint.h) is needed for uintN_t and uintptr_t.

ddb: Cast pointer to uintptr_t, then to db_expr_t.
Avoids warnings about conversion between pointer and integer of
different size on some architectures.

re-fix hppa builds.

this file uses __cpu_simple_lock(), not just the underlying type,
so it does need machine/lock.h.

Break cycle by using `struct kmutex *' instead of `kmutex_t *'.
sys/sched.h included sys/mutex.h
which includes sys/intr.h
which includes machine/intr.h
which on cats includes arm/footbridge/footbridge_intr.h
which includes arm/cpu.h
which includes sys/cpu_data.h
which includes sys/sched.h

But there was never any real need for sys/mutex.h in sys/sched.h,
because it only uses pointers to the opaque struct kmutex. Cycle
broken by using `struct kmutex *' instead of pulling in sys/mutex.h
for the definition of kmutex_t.

Side effect: This revealed that sys/cpu_data.h needed sys/intr.h
(which was pulled in accidentally by sys/mutex.h via sys/sched.h) for
SOFTINT_COUNT. Also revealed some other machine/cpu.h header files
were missing includes of sys/mutex.h for kmutex_t.

ia64: Need sys/types.h for u_int, vaddr_t; sys/mutex.h for kmutex_t.

explicitly include no longer implicitly included sys/mutex.h.

arm/xscale: Use sys/bitops.h fls32 - 1 instead of 31 - __builtin_clz.
Sidesteps namespace collision with `#define bits ...' in net/zlib.c.

complete the previous - there were two calls to find_first_bit() to fix.

arm/xscale: Missed a spot with previous find_first_bit commit.

evbarm/g42xxeb: Fix off-by-one in previous.

The original find_first_bit(x) was 31 - __builtin_clz((uint32_t)x),
which is equivalent to fls32(x) - 1, not to fls32(x).

Note that fls32 is 1-based and returns 0 for x=0.
 1.3 02-Jun-2017  knakahara rename crypto_mtx to cryptodev_mtx

It is used by cryptodev.c and ocryptodev.c only.
 1.2 24-Apr-2017  knakahara separate crypto_drv_mtx from crypto_mtx.

crypto_mtx is used only for cryptodev.c and ocryptodev.c now.
 1.1 19-Feb-2011  drochner branches: 1.1.2; 1.1.4; 1.1.8; 1.1.20; 1.1.38; 1.1.42;
make the compatibility code conditional on COMPAT_50
 1.1.42.1 26-Apr-2017  pgoyette Sync with HEAD
 1.1.38.1 28-Aug-2017  skrll Sync with HEAD
 1.1.20.1 03-Dec-2017  jdolecek update from HEAD
 1.1.8.2 06-Jun-2011  jruoho Sync with HEAD.
 1.1.8.1 19-Feb-2011  jruoho file cryptodev_internal.h was added on branch jruoho-x86intr on 2011-06-06 09:10:03 +0000
 1.1.4.2 05-Mar-2011  rmind sync with head
 1.1.4.1 19-Feb-2011  rmind file cryptodev_internal.h was added on branch rmind-uvmplock on 2011-03-05 20:56:05 +0000
 1.1.2.2 05-Mar-2011  bouyer Sync with HEAD
 1.1.2.1 19-Feb-2011  bouyer file cryptodev_internal.h was added on branch bouyer-quota2 on 2011-03-05 15:10:48 +0000
 1.65 26-Jul-2025  perseant Correct a typo in rev 1.59, which results in a crash if the number of
sessions exceeds CRYPTO_SW_SESSIONS = 32.
 1.64 22-May-2022  riastradh branches: 1.64.4; 1.64.10;
opencrypto: Make freesession callback return void.

No functional change intended: all drivers already return zero
unconditionally.
 1.63 22-May-2022  riastradh cryptosoft(4): Prune dead branches. Assert session id validity.
 1.62 22-May-2022  riastradh cryptosoft(4): Rip out nonsense to quietly ignore sid=0.

This is no longer necessary because crypto_freesession no longer
calls into the driver for session ids that were never allocated in
the first place.
 1.61 06-Apr-2021  knakahara Fix ATF failures, sorry.
 1.60 05-Apr-2021  knakahara refactor: reduce access to swcr_sessions[i] directly
 1.59 05-Apr-2021  knakahara refactor: reduce changing swcr_sesnum
 1.58 05-Apr-2021  knakahara use kmem_{z,}alloc() instead of malloc()
 1.57 04-Jul-2020  riastradh branches: 1.57.4;
Fix kmem_free size in recent malloc->kmem conversion.

Should address this bracket report that has my name all over it:

https://mail-index.netbsd.org/current-users/2020/07/04/msg039059.html
 1.56 29-Jun-2020  riastradh opencrypto: Switch from legacy rijndael API to new aes API.

While here, apply various rijndael->aes renames, reduce the size
of aesxcbc_ctx by 480 bytes, and convert some malloc->kmem.

Leave in the symbol enc_xform_rijndael128 for now, though, so this
doesn't break any kernel ABI.
 1.55 14-Jun-2020  riastradh swcrypto(4): Simplify iv generation logic with cprng_fast.
 1.54 12-Oct-2019  christos add (void *) intermediate casts to elide gcc function cast warnings. This
is the simplest solution; choices:
- add pragmas, complex and ugly (need to be gcc-specific)
- add -Wno to COPTS. Needs to be done in many makefiles because of rump
- add intermediate functions: slows down things
 1.53 11-Jul-2019  christos Disable unloading until we keep track of references
 1.52 23-Jun-2017  knakahara branches: 1.52.6;
fix cryptosoft.c:r1.51 mistake. swcrypto_attach() must not be called from module_init_class().

swcrypto_attach() will call softint_establish(), it must be called after cpus
attached. module_init_class() is too early to call softint_establish().
 1.51 01-Jun-2017  knakahara branches: 1.51.2;
swcrypto0 was initialized twice. Fix like pseudo network interfaces.

ok by pgoyette@n.o.
 1.50 17-May-2017  knakahara opencrypto: cleanup debug messages.
 1.49 18-Apr-2017  maya branches: 1.49.2;
Remove duplicate assignment.
We assign the same value unconditionally just before.

from clang static analyzer

XXX surrounding code seems fishy
 1.48 13-Apr-2017  ozaki-r Fix usage of MD5Final/SHA1Final

Passing NULL as the digest parameter is wrong.
 1.47 20-Aug-2015  christos branches: 1.47.2; 1.47.4;
include "ioconf.h" to get the 'void <driver>attach(int count);' prototype.
 1.46 02-Jul-2014  riastradh branches: 1.46.4;
If we register with pmf on attach, deregister on detach.
 1.45 21-Jun-2014  christos register with pmf.
 1.44 01-Jan-2014  pgoyette branches: 1.44.2;
Modularize the opencrypto components and link to the build
 1.43 12-Sep-2013  martin Remove unused variable
 1.42 24-Jun-2013  riastradh branches: 1.42.2;
Replace consttime_bcmp/explicit_bzero by consttime_memequal/explicit_memset.

consttime_memequal is the same as the old consttime_bcmp.
explicit_memset is to memset as explicit_bzero was to bcmp.

Passes amd64 release and i386/ALL, but I'm sure I missed some spots,
so please let me know.
 1.41 02-Feb-2013  christos fix compilation
 1.40 30-Aug-2012  drochner branches: 1.40.2;
Add "consttime_bcmp" and "explicit_bzero" functions for both kernel
abd userland, as proposed on tech-security, with explicit_bzero using
a volatile function pointer as suggested by Alan Barrett.
Both do what the name says. For userland, both are prefixed by "__"
to keep them out of the user namespace.
Change some memset/memcmp uses to the new functions where it makes
sense -- these are just some examples, more to come.
 1.39 28-Nov-2011  tls Remove arc4random() and arc4randbytes() from the kernel API. Replace
arc4random() hacks in rump with stubs that call the host arc4random() to
get numbers that are hopefully actually random (arc4random() keyed with
stack junk is not). This should fix some of the currently failing anita
tests -- we should no longer generate duplicate "random" MAC addresses in
the test environment.
 1.38 07-Jun-2011  drochner branches: 1.38.2;
use a simple counter as IV for AES-GMAC as suggested in RFC4543
 1.37 26-May-2011  drochner branches: 1.37.2;
pull in AES-GCM/GMAC support from OpenBSD
This is still somewhat experimental. Tested between 2 similar boxes
so far. There is much potential for performance improvement. For now,
I've changed the gmac code to accept any data alignment, as the "char *"
pointer suggests. As the code is practically used, 32-bit alignment
can be assumed, at the cost of data copies. I don't know whether
bytewise access or copies are worse performance-wise. For efficient
implementations using SSE2 instructions on x86, even stricter
alignment requirements might arise.
 1.36 24-May-2011  drochner copy AES-XCBC-MAC support from KAME IPSEC to FAST_IPSEC
For this to fit, an API change in cryptosoft was adopted from OpenBSD
(addition of a "Setkey" method to hashes) which was done for GCM/GMAC
support there, so it might be useful in the future anyway.
tested against KAME IPSEC
AFAICT, FAST_IPSEC now supports as much as KAME.
 1.35 24-May-2011  drochner move the "context size" struct member (which is a pure software
implementation thing) from the abstract xform descriptor to
the cryptosoft implementation part -- for sanity, and now clients
of opencrypto don't depend on headers of cipher implementations anymore
 1.34 24-May-2011  drochner Change the way the IV is generated for AES-CTR: use a simple counter
instead of arc4random(). AES-CTR is sensitive against IV recurrence
(with the same key / nonce), and a random number doesn't give that
guarantee.
This needs a little API change in cryptosoft -- I've suggested it to
Open/FreeBSD, might change it depending on feedback.
Thanks to Steven Bellovin for hints.
 1.33 23-May-2011  drochner add an AES-CTR xform, from OpenBSD
 1.32 23-May-2011  drochner -in the descriptor for encryption xforms, split the "blocksize" field
into "blocksize" and "IV size"
-add an "reinit" function pointer which, if set, means that the xform
does its IV handling itself and doesn't want the default CBC handling
by the framework (poor name, but left that way to avoid unecessary
differences)
This syncs with Open/FreeBSD, purpose is to allow non-CBC transforms.
Refer to ivsize instead of blocksize where appropriate.
(At this point, blocksize and ivsize are identical.)
 1.31 21-May-2011  drochner fix a logics bug (which has been here from the beginning) which made
that only 96 random bits were used for IV generation,
this caused eg that the last 4 bytes of the IV in ESP/AES-CBC
were constant, leaking kernel memory
affects FAST_IPSEC only
 1.30 05-May-2011  drochner support camellia-cbc by swcrypt
 1.29 25-Feb-2011  drochner make the use of SHA2-HMAC by FAST_IPSEC compliant to current standards:
-RFC2104 says that the block size of the hash algorithm must be used
for key/ipad/opad calculations. While formerly all ciphers used a block
length of 64, SHA384 and SHA512 use 128 bytes. So we can't use the
HMAC_BLOCK_LEN constant anymore. Add a new field to "struct auth_hash"
for the per-cipher blocksize.
-Due to this, there can't be a single "CRYPTO_SHA2_HMAC" external name
anymore. Replace this by 3 for the 3 different keysizes.
This was done by Open/FreeBSD before.
-Also fix the number of authenticator bits used tor ESP and AH to
conform to RFC4868, and remove uses of AH_HMAC_HASHLEN which did
assume a fixed authenticator size of 12 bytes.

FAST_IPSEC will not interoperate with KAME IPSEC anymore if sha2 is used,
because the latter doesn't implement these standards. It should
interoperate with at least modern Free/OpenBSD now.
(I've only tested with NetBSD-current/FAST_IPSEC on both ends.)
 1.28 24-Feb-2011  drochner small modifications in dealing with the unknown result size of compression/
decompression:
-seperate the IPCOMP specific rule that compression must not grow the
data from general compression semantics: Introduce a special name
CRYPTO_DEFLATE_COMP_NOGROW/comp_algo_deflate_nogrow to describe
the IPCOMP semantics and use it there. (being here, fix the check
so that equal size is considered failure as well as required by
RFC2393)
Customers of CRYPTO_DEFLATE_COMP/comp_algo_deflate now always get
deflated data back, even if they are not smaller than the original.
-allow to pass a "size hint" to the DEFLATE decompression function
which is used for the initial buffer allocation. Due to the changes
done there, additional allocations and extra copies are avoided if the
initial allocation is sufficient. Set the size hint to MCLBYTES (=2k)
in IPCOMP which should be good for many use cases.
 1.27 10-Feb-2011  drochner Don't store temporary values in the opencrypto session data struct which
can be shared by multiple threads -- pass them on the stack instead.
Add some "const" to document this. (One _could_ use the session struct
for temporary stuff with proper locking, but it seems unnecessary here.)
Also remove the unused SW_crc member in the session struct.
From Wolfgang Stukenbrock per PR kern/44472.
 1.26 02-Aug-2010  jakllsch branches: 1.26.2; 1.26.4;
Consistently use a single CRYPTO_SESID2HID-like macro.
Improve CRYPTO_DEBUG printing a bit:
print pointers with %p
print unsigned with %u rather than %d
use CRYPTO_SESID2LID instead of just casting to uint32_t
 1.25 18-Apr-2009  tsutsui branches: 1.25.2; 1.25.4;
Remove extra whitespace added by a stupid tool.
XXX: more in src/sys/arch
 1.24 25-Mar-2009  darran Fixes PR kern/41069 and PR kern/41070.

Extends the Opencrypto API to allow the destination buffer size to be
specified when its not the same size as the input buffer (i.e. for
operations like compress and decompress).
The crypto_op and crypt_n_op structures gain a u_int dst_len field.
The session_op structure gains a comp_alg field to specify a compression
algorithm.
Moved four ioctls to new ids; CIOCGSESSION, CIOCNGSESSION, CIOCCRYPT,
and CIOCNCRYPTM.
Added four backward compatible ioctls; OCIOCGSESSION, OCIOCNGSESSION,
OCIOCCRYPT, and OCIOCNCRYPTM.

Backward compatibility is maintained in ocryptodev.h and ocryptodev.c which
implement the original ioctls and set dst_len and comp_alg to 0.

Adds user-space access to compression features.

Adds software gzip support (CRYPTO_GZIP_COMP).

Adds the fast version of crc32 from zlib to libkern. This should be generally
useful and provide a place to start normalizing the various crc32 routines
in the kernel. The crc32 routine is used in this patch to support GZIP.

With input and support from tls@NetBSD.org.
 1.23 18-Mar-2009  cegger bcopy -> memcpy
 1.22 18-Mar-2009  cegger bzero -> memset
 1.21 17-Dec-2008  cegger branches: 1.21.2;
kill MALLOC and FREE macros.
 1.20 04-Feb-2008  tls branches: 1.20.6; 1.20.10; 1.20.18; 1.20.20; 1.20.26;
Rework opencrypto to use a spin mutex (crypto_mtx) instead of "splcrypto"
(actually splnet) and condvars instead of tsleep/wakeup. Fix a few
miscellaneous problems and add some debugging printfs while there.

Restore set of CRYPTO_F_DONE in crypto_done() which was lost at some
point after this code came from FreeBSD -- it made it impossible to wait
properly for a condition.

Add flags analogous to the "crp" flags to the key operation's krp struct.
Add a new flag, CRYPTO_F_ONRETQ which tells us a request finished before
the kthread had a chance to dequeue it and call its callback -- this was
letting requests stick on the queues before even though done and copied
out.

Callers of crypto_newsession() or crypto_freesession() must now take the
mutex. Change netipsec to do so. Dispatch takes the mutex itself as
needed.

This was tested fairly extensively with the cryptosoft backend and lightly
with a new hardware driver. It has not been tested with FAST_IPSEC; I am
unable to ascertain whether FAST_IPSEC currently works at all in our tree.

pjd@FreeBSD.ORG, ad@NetBSD.ORG, and darran@snark.us pointed me in the
right direction several times in the course of this. Remaining bugs
are mine alone.
 1.19 02-Feb-2008  tls Add CRYPTO_*_HMAC_96 defines -- missed this file in previous commit.
 1.18 01-Feb-2008  tls This code never worked on a released version of FreeBSD in the form it's
been in in our tree, and certainly does not work on any version of FreeBSD
now. Run through unifdef -D__NetBSD__ -U__FreeBSD__ yielding a small
reduction of size and a dramatic improvement in readability.

No, this does not yield any meaningful decrease in patchability (unlike
mechanical changes that touch live source lines) -- try it and see.
 1.17 04-Mar-2007  christos branches: 1.17.16; 1.17.22;
Kill caddr_t; there will be some MI fallout, but it will be fixed shortly.
 1.16 17-Feb-2007  daniel branches: 1.16.2;
Add an opencrypto provider for the AES xcrypt instructions found on VIA
C5P and later cores (also known as 'ACE', which is part of the VIA PadLock
security engine). Ported from OpenBSD.

Reviewed on tech-crypto and port-i386, no objections to commiting this.
 1.15 16-Nov-2006  christos __unused removal on arguments; approved by core.
 1.14 12-Oct-2006  christos - sprinkle __unused on function decls.
- fix a couple of unused bugs
- no more -Wno-unused for i386
 1.13 02-Apr-2006  dsl branches: 1.13.8; 1.13.10;
malloc data the size the pointer points to, not the size of a pointer.
Maybe we get away with this (at least on 32bit archs) because the structure
is 24 bytes and I bet the minimum allocation size is 32.
Fixed coverty CIDs 2732 and 2733
 1.12 17-Mar-2006  christos don't use MALLOC with a non-constant size; use malloc instead.
 1.11 25-Nov-2005  thorpej branches: 1.11.4; 1.11.6; 1.11.8; 1.11.10; 1.11.12;
swcr -> swcrypto
 1.10 25-Nov-2005  thorpej - De-couple the software crypto implementation from the rest of the
framework. There is no need to waste the space if you are only using
algoritms provided by hardware accelerators. To get the software
implementations, add "pseudo-device swcr" to your kernel config.
- Lazily initialize the opencrypto framework when crypto drivers
(either hardware or swcr) register themselves with the framework.
 1.9 26-Feb-2005  perry branches: 1.9.4; 1.9.10;
nuke trailing whitespace
 1.8 27-Aug-2003  thorpej branches: 1.8.4; 1.8.10; 1.8.12;
Some const poisoning.
 1.7 26-Aug-2003  thorpej Remove a bunch of unnecessary includes.
 1.6 25-Aug-2003  thorpej It's bad form to use the <opencrypto/rmd160.h> header file while
using the crypto/ripemd160/rmd160.c implementation. Remove the
opencrypto-local copies of these files entirely.
 1.5 30-Jul-2003  jonathan Garbage-collect references to OpenBSD-only <dev/rndvar.h>.
 1.4 28-Jul-2003  jonathan Remove vestiges of OpenBSD <sys/md5k.h> header.
 1.3 27-Jul-2003  jonathan Cleanup traces of previous standalone m_apply()/m_getptr().
 1.2 26-Jul-2003  jonathan Fix authentication hashes requested via /dev/crypto. The handler for
userland hashes case was partly omitted in the OpenBSD -> FreeBSD port.
 1.1 25-Jul-2003  jonathan Commit initial NetBSD port of the OpenCrypto Framework (OCF). This
code is derived from Sam Leffler's FreeBSD port of OCF, which is in
turn a port of Angelos Keromytis's OpenBSD work.
Credit to Sam and Angelos, any blame for the NetBSD port to me.
 1.8.12.1 19-Mar-2005  yamt sync with head. xen and whitespace. xen part is not finished.
 1.8.10.1 29-Apr-2005  kent sync with -current
 1.8.4.6 11-Dec-2005  christos Sync with head.
 1.8.4.5 04-Mar-2005  skrll Sync with HEAD.

Hi Perry!
 1.8.4.4 21-Sep-2004  skrll Fix the sync with head I botched.
 1.8.4.3 18-Sep-2004  skrll Sync with HEAD.
 1.8.4.2 03-Aug-2004  skrll Sync with HEAD
 1.8.4.1 27-Aug-2003  skrll file cryptosoft.c was added on branch ktrace-lwp on 2004-08-03 10:56:25 +0000
 1.9.10.1 29-Nov-2005  yamt sync with head.
 1.9.4.4 04-Feb-2008  yamt sync with head.
 1.9.4.3 03-Sep-2007  yamt sync with head.
 1.9.4.2 26-Feb-2007  yamt sync with head.
 1.9.4.1 21-Jun-2006  yamt sync with head.
 1.11.12.2 24-May-2006  tron Merge 2006-05-24 NetBSD-current into the "peter-altq" branch.
 1.11.12.1 28-Mar-2006  tron Merge 2006-03-28 NetBSD-current into the "peter-altq" branch.
 1.11.10.1 19-Apr-2006  elad sync with head.
 1.11.8.2 11-Apr-2006  yamt sync with head
 1.11.8.1 01-Apr-2006  yamt sync with head.
 1.11.6.1 22-Apr-2006  simonb Sync with head.
 1.11.4.1 09-Sep-2006  rpaulo sync with head
 1.13.10.2 10-Dec-2006  yamt sync with head.
 1.13.10.1 22-Oct-2006  yamt sync with head
 1.13.8.1 18-Nov-2006  ad Sync with head.
 1.16.2.2 12-Mar-2007  rmind Sync with HEAD.
 1.16.2.1 17-Feb-2007  rmind file cryptosoft.c was added on branch yamt-idlelwp on 2007-03-12 06:00:50 +0000
 1.17.22.1 18-Feb-2008  mjf Sync with HEAD.
 1.17.16.1 23-Mar-2008  matt sync with HEAD
 1.20.26.1 20-May-2011  matt bring matt-nb5-mips64 up to date with netbsd-5-1-RELEASE (except compat).
 1.20.20.1 03-May-2009  snj Pull up following revision(s) (requested by tls in ticket #611):
sys/lib/libkern/Makefile: patch
sys/lib/libkern/crc32.c: revision 1.1
sys/lib/libkern/crc32.h: revision 1.1
sys/lib/libkern/libkern.h: revision 1.89
sys/lib/libkern/arch/i386/Makefile.inc: revision 1.28
sys/net/zlib.h: revision 1.14 via patch
sys/opencrypto/crypto.c: revision 1.33
sys/opencrypto/cryptodev.c: revision 1.46
sys/opencrypto/cryptodev.h: revision 1.16
sys/opencrypto/cryptosoft.c: revision 1.24
sys/opencrypto/cryptosoft.h: revision 1.6
sys/opencrypto/deflate.h: revision 1.6
sys/opencrypto/cryptosoft_xform.c: revision 1.12
sys/opencrypto/deflate.c: revision 1.13
sys/opencrypto/files.opencrypto: revision 1.20
sys/opencrypto/ocryptodev.c: revision 1.1
sys/opencrypto/ocryptodev.h: revision 1.1
sys/opencrypto/xform.c: revision 1.18
sys/opencrypto/xform.h: revision 1.10
Fixes PR kern/41069 and PR kern/41070.

Extends the Opencrypto API to allow the destination buffer size to be
specified when its not the same size as the input buffer (i.e. for
operations like compress and decompress).
The crypto_op and crypt_n_op structures gain a u_int dst_len field.
The session_op structure gains a comp_alg field to specify a compression
algorithm.
Moved four ioctls to new ids; CIOCGSESSION, CIOCNGSESSION, CIOCCRYPT,
and CIOCNCRYPTM.
Added four backward compatible ioctls; OCIOCGSESSION, OCIOCNGSESSION,
OCIOCCRYPT, and OCIOCNCRYPTM.

Backward compatibility is maintained in ocryptodev.h and ocryptodev.c which
implement the original ioctls and set dst_len and comp_alg to 0.

Adds user-space access to compression features.

Adds software gzip support (CRYPTO_GZIP_COMP).

Adds the fast version of crc32 from zlib to libkern. This should be generally
useful and provide a place to start normalizing the various crc32 routines
in the kernel. The crc32 routine is used in this patch to support GZIP.

With input and support from tls@NetBSD.org.
 1.20.18.2 28-Apr-2009  skrll Sync with HEAD.
 1.20.18.1 19-Jan-2009  skrll Sync with HEAD.
 1.20.10.2 11-Aug-2010  yamt sync with head.
 1.20.10.1 04-May-2009  yamt sync with head.
 1.20.6.1 17-Jan-2009  mjf Sync with HEAD.
 1.21.2.1 13-May-2009  jym Sync with HEAD.

Commit is split, to avoid a "too many arguments" protocol error.
 1.25.4.3 12-Jun-2011  rmind sync with head
 1.25.4.2 31-May-2011  rmind sync with head
 1.25.4.1 05-Mar-2011  rmind sync with head
 1.25.2.1 17-Aug-2010  uebayasi Sync with HEAD.
 1.26.4.2 05-Mar-2011  bouyer Sync with HEAD
 1.26.4.1 17-Feb-2011  bouyer Sync with HEAD
 1.26.2.1 06-Jun-2011  jruoho Sync with HEAD.
 1.37.2.1 23-Jun-2011  cherry Catchup with rmind-uvmplock merge.
 1.38.2.3 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.38.2.2 30-Oct-2012  yamt sync with head
 1.38.2.1 17-Apr-2012  yamt sync with head
 1.40.2.3 03-Dec-2017  jdolecek update from HEAD
 1.40.2.2 20-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.40.2.1 25-Feb-2013  tls resync with head
 1.42.2.1 18-May-2014  rmind sync with head
 1.44.2.1 10-Aug-2014  tls Rebase.
 1.46.4.2 28-Aug-2017  skrll Sync with HEAD
 1.46.4.1 22-Sep-2015  skrll Sync with HEAD
 1.47.4.1 21-Apr-2017  bouyer Sync with HEAD
 1.47.2.1 26-Apr-2017  pgoyette Sync with HEAD
 1.49.2.2 19-May-2017  pgoyette Resolve conflicts from previous merge (all resulting from $NetBSD
keywork expansion)
 1.49.2.1 17-May-2017  pgoyette At suggestion of chuq@, modify config_attach_pseudo() to return with a
reference held on the device.

Adapt callers to expect the reference to exist, and to ensure that the
reference is released.
 1.51.2.1 05-Jul-2017  snj Pull up following revision(s) (requested by knakahara in ticket #97):
sys/opencrypto/crypto.c: 1.87-1.91
sys/opencrypto/cryptodev.c: 1.93-1.95
sys/opencrypto/cryptodev.h: 1.37
sys/opencrypto/cryptosoft.c: 1.52
sys/rump/dev/lib/libopencrypto/opencrypto_component.c: 1.5
sanitize count used for kmem_alloc size.
Hmm, who uses CIOCNGSESSION, CIOCNFSESSION, CIOCNCRYPTM or CIOCNFKEYM?
--
sanitize in CIOCNCRYPTM and initialize comp_alg in CIOCNGSESSION
--
must release cap->cc_lock before calling cap->cc_newsession() because of spinlock.
--
refactor crypto_newsession() like FreeBSD.
--
support multiple encryption drivers (port from FreeBSD).
--
Divide crp_devflags from crp_flags to write exclusively.
CRYPTO_F_DQRETQ(new name is CRYPTODEV_F_RET) is used by cryptodev.c only.
It should be divided to other member.
--
Reduce crypto_ret_q_mtx lock regions.
crypto.c does not access the members of crp when the crp is in crp_q or
crp_ret_q. Furthermore, crp_q and crp_ret_q are protected by each mutex,
so the members of crp is not shared. That means crp_flags is not required
mutex in crypto.c.
--
fix cryptosoft.c:r1.51 mistake. swcrypto_attach() must not be called from module_init_class().
swcrypto_attach() will call softint_establish(), it must be called after cpus
attached. module_init_class() is too early to call softint_establish().
--
simplify mutex_enter/exit(crypto_q_mtx), and fix missing exit.
--
reduce rump waring message. pointed out by ozaki-r@n.o, thanks.
 1.52.6.1 13-Apr-2020  martin Mostly merge changes from HEAD upto 20200411
 1.57.4.1 17-Apr-2021  thorpej Sync with HEAD.
 1.64.10.1 02-Aug-2025  perseant Sync with HEAD
 1.64.4.1 27-Jul-2025  martin Pull up following revision(s) (requested by kim in ticket #1138):

sys/opencrypto/cryptosoft.c: revision 1.65

Correct a typo in rev 1.59, which results in a crash if the number of
sessions exceeds CRYPTO_SW_SESSIONS = 32.
 1.7 10-Feb-2011  drochner Don't store temporary values in the opencrypto session data struct which
can be shared by multiple threads -- pass them on the stack instead.
Add some "const" to document this. (One _could_ use the session struct
for temporary stuff with proper locking, but it seems unnecessary here.)
Also remove the unused SW_crc member in the session struct.
From Wolfgang Stukenbrock per PR kern/44472.
 1.6 25-Mar-2009  darran branches: 1.6.4; 1.6.6; 1.6.8;
Fixes PR kern/41069 and PR kern/41070.

Extends the Opencrypto API to allow the destination buffer size to be
specified when its not the same size as the input buffer (i.e. for
operations like compress and decompress).
The crypto_op and crypt_n_op structures gain a u_int dst_len field.
The session_op structure gains a comp_alg field to specify a compression
algorithm.
Moved four ioctls to new ids; CIOCGSESSION, CIOCNGSESSION, CIOCCRYPT,
and CIOCNCRYPTM.
Added four backward compatible ioctls; OCIOCGSESSION, OCIOCNGSESSION,
OCIOCCRYPT, and OCIOCNCRYPTM.

Backward compatibility is maintained in ocryptodev.h and ocryptodev.c which
implement the original ioctls and set dst_len and comp_alg to 0.

Adds user-space access to compression features.

Adds software gzip support (CRYPTO_GZIP_COMP).

Adds the fast version of crc32 from zlib to libkern. This should be generally
useful and provide a place to start normalizing the various crc32 routines
in the kernel. The crc32 routine is used in this patch to support GZIP.

With input and support from tls@NetBSD.org.
 1.5 04-Mar-2007  christos branches: 1.5.40; 1.5.50; 1.5.52; 1.5.56; 1.5.60;
Kill caddr_t; there will be some MI fallout, but it will be fixed shortly.
 1.4 17-Feb-2007  daniel branches: 1.4.2;
Add an opencrypto provider for the AES xcrypt instructions found on VIA
C5P and later cores (also known as 'ACE', which is part of the VIA PadLock
security engine). Ported from OpenBSD.

Reviewed on tech-crypto and port-i386, no objections to commiting this.
 1.3 25-Nov-2005  thorpej - De-couple the software crypto implementation from the rest of the
framework. There is no need to waste the space if you are only using
algoritms provided by hardware accelerators. To get the software
implementations, add "pseudo-device swcr" to your kernel config.
- Lazily initialize the opencrypto framework when crypto drivers
(either hardware or swcr) register themselves with the framework.
 1.2 27-Aug-2003  thorpej branches: 1.2.4; 1.2.18; 1.2.26;
Some const poisoning.
 1.1 25-Jul-2003  jonathan Commit initial NetBSD port of the OpenCrypto Framework (OCF). This
code is derived from Sam Leffler's FreeBSD port of OCF, which is in
turn a port of Angelos Keromytis's OpenBSD work.
Credit to Sam and Angelos, any blame for the NetBSD port to me.
 1.2.26.1 29-Nov-2005  yamt sync with head.
 1.2.18.3 03-Sep-2007  yamt sync with head.
 1.2.18.2 26-Feb-2007  yamt sync with head.
 1.2.18.1 21-Jun-2006  yamt sync with head.
 1.2.4.5 11-Dec-2005  christos Sync with head.
 1.2.4.4 21-Sep-2004  skrll Fix the sync with head I botched.
 1.2.4.3 18-Sep-2004  skrll Sync with HEAD.
 1.2.4.2 03-Aug-2004  skrll Sync with HEAD
 1.2.4.1 27-Aug-2003  skrll file cryptosoft.h was added on branch ktrace-lwp on 2004-08-03 10:56:25 +0000
 1.4.2.2 12-Mar-2007  rmind Sync with HEAD.
 1.4.2.1 17-Feb-2007  rmind file cryptosoft.h was added on branch yamt-idlelwp on 2007-03-12 06:00:50 +0000
 1.5.60.1 20-May-2011  matt bring matt-nb5-mips64 up to date with netbsd-5-1-RELEASE (except compat).
 1.5.56.1 13-May-2009  jym Sync with HEAD.

Commit is split, to avoid a "too many arguments" protocol error.
 1.5.52.1 03-May-2009  snj Pull up following revision(s) (requested by tls in ticket #611):
sys/lib/libkern/Makefile: patch
sys/lib/libkern/crc32.c: revision 1.1
sys/lib/libkern/crc32.h: revision 1.1
sys/lib/libkern/libkern.h: revision 1.89
sys/lib/libkern/arch/i386/Makefile.inc: revision 1.28
sys/net/zlib.h: revision 1.14 via patch
sys/opencrypto/crypto.c: revision 1.33
sys/opencrypto/cryptodev.c: revision 1.46
sys/opencrypto/cryptodev.h: revision 1.16
sys/opencrypto/cryptosoft.c: revision 1.24
sys/opencrypto/cryptosoft.h: revision 1.6
sys/opencrypto/deflate.h: revision 1.6
sys/opencrypto/cryptosoft_xform.c: revision 1.12
sys/opencrypto/deflate.c: revision 1.13
sys/opencrypto/files.opencrypto: revision 1.20
sys/opencrypto/ocryptodev.c: revision 1.1
sys/opencrypto/ocryptodev.h: revision 1.1
sys/opencrypto/xform.c: revision 1.18
sys/opencrypto/xform.h: revision 1.10
Fixes PR kern/41069 and PR kern/41070.

Extends the Opencrypto API to allow the destination buffer size to be
specified when its not the same size as the input buffer (i.e. for
operations like compress and decompress).
The crypto_op and crypt_n_op structures gain a u_int dst_len field.
The session_op structure gains a comp_alg field to specify a compression
algorithm.
Moved four ioctls to new ids; CIOCGSESSION, CIOCNGSESSION, CIOCCRYPT,
and CIOCNCRYPTM.
Added four backward compatible ioctls; OCIOCGSESSION, OCIOCNGSESSION,
OCIOCCRYPT, and OCIOCNCRYPTM.

Backward compatibility is maintained in ocryptodev.h and ocryptodev.c which
implement the original ioctls and set dst_len and comp_alg to 0.

Adds user-space access to compression features.

Adds software gzip support (CRYPTO_GZIP_COMP).

Adds the fast version of crc32 from zlib to libkern. This should be generally
useful and provide a place to start normalizing the various crc32 routines
in the kernel. The crc32 routine is used in this patch to support GZIP.

With input and support from tls@NetBSD.org.
 1.5.50.1 28-Apr-2009  skrll Sync with HEAD.
 1.5.40.1 04-May-2009  yamt sync with head.
 1.6.8.1 17-Feb-2011  bouyer Sync with HEAD
 1.6.6.1 06-Jun-2011  jruoho Sync with HEAD.
 1.6.4.1 05-Mar-2011  rmind sync with head
 1.30 30-Jun-2020  riastradh Rename enc_xform_rijndael128 -> enc_xform_aes.

Update netipsec dependency.
 1.29 29-Jun-2020  riastradh opencrypto: Switch from legacy rijndael API to new aes API.

While here, apply various rijndael->aes renames, reduce the size
of aesxcbc_ctx by 480 bytes, and convert some malloc->kmem.

Leave in the symbol enc_xform_rijndael128 for now, though, so this
doesn't break any kernel ABI.
 1.28 12-Oct-2019  christos add (void *) intermediate casts to elide gcc function cast warnings. This
is the simplest solution; choices:
- add pragmas, complex and ugly (need to be gcc-specific)
- add -Wno to COPTS. Needs to be done in many makefiles because of rump
- add intermediate functions: slows down things
 1.27 27-Nov-2014  christos branches: 1.27.20;
simplify, no functional change
 1.26 02-Feb-2013  christos fix compilation
 1.25 28-Nov-2011  tls branches: 1.25.8;
Remove arc4random() and arc4randbytes() from the kernel API. Replace
arc4random() hacks in rump with stubs that call the host arc4random() to
get numbers that are hopefully actually random (arc4random() keyed with
stack junk is not). This should fix some of the currently failing anita
tests -- we should no longer generate duplicate "random" MAC addresses in
the test environment.
 1.24 07-Jun-2011  drochner branches: 1.24.2;
use a simple counter as IV for AES-GMAC as suggested in RFC4543
 1.23 26-May-2011  drochner branches: 1.23.2;
pull in AES-GCM/GMAC support from OpenBSD
This is still somewhat experimental. Tested between 2 similar boxes
so far. There is much potential for performance improvement. For now,
I've changed the gmac code to accept any data alignment, as the "char *"
pointer suggests. As the code is practically used, 32-bit alignment
can be assumed, at the cost of data copies. I don't know whether
bytewise access or copies are worse performance-wise. For efficient
implementations using SSE2 instructions on x86, even stricter
alignment requirements might arise.
 1.22 24-May-2011  drochner copy AES-XCBC-MAC support from KAME IPSEC to FAST_IPSEC
For this to fit, an API change in cryptosoft was adopted from OpenBSD
(addition of a "Setkey" method to hashes) which was done for GCM/GMAC
support there, so it might be useful in the future anyway.
tested against KAME IPSEC
AFAICT, FAST_IPSEC now supports as much as KAME.
 1.21 24-May-2011  drochner move the "context size" struct member (which is a pure software
implementation thing) from the abstract xform descriptor to
the cryptosoft implementation part -- for sanity, and now clients
of opencrypto don't depend on headers of cipher implementations anymore
 1.20 24-May-2011  drochner Change the way the IV is generated for AES-CTR: use a simple counter
instead of arc4random(). AES-CTR is sensitive against IV recurrence
(with the same key / nonce), and a random number doesn't give that
guarantee.
This needs a little API change in cryptosoft -- I've suggested it to
Open/FreeBSD, might change it depending on feedback.
Thanks to Steven Bellovin for hints.
 1.19 23-May-2011  drochner -remove references to crypto/arc4/arc4.* -- the code isn't used
anywhere afaics
(The confusion comes probably from use of arc4random() at various places,
but this lives in libkern and doesn't share code with the former.)
-g/c non-implementation of arc4 encryption in swcrypto(4)
-remove special casing of ARC4 in crypto(4) -- the point is that it
doesn't use an IV, and this fact is made explicit by the new "ivsize"
property of xforms
 1.18 23-May-2011  drochner add an AES-CTR xform, from OpenBSD
 1.17 23-May-2011  drochner -in the descriptor for encryption xforms, split the "blocksize" field
into "blocksize" and "IV size"
-add an "reinit" function pointer which, if set, means that the xform
does its IV handling itself and doesn't want the default CBC handling
by the framework (poor name, but left that way to avoid unecessary
differences)
This syncs with Open/FreeBSD, purpose is to allow non-CBC transforms.
Refer to ivsize instead of blocksize where appropriate.
(At this point, blocksize and ivsize are identical.)
 1.16 21-May-2011  drochner check key size on initialization -- othwise the rijndael code
can fail silently
 1.15 05-May-2011  drochner support camellia-cbc by swcrypt
 1.14 24-Feb-2011  drochner small modifications in dealing with the unknown result size of compression/
decompression:
-seperate the IPCOMP specific rule that compression must not grow the
data from general compression semantics: Introduce a special name
CRYPTO_DEFLATE_COMP_NOGROW/comp_algo_deflate_nogrow to describe
the IPCOMP semantics and use it there. (being here, fix the check
so that equal size is considered failure as well as required by
RFC2393)
Customers of CRYPTO_DEFLATE_COMP/comp_algo_deflate now always get
deflated data back, even if they are not smaller than the original.
-allow to pass a "size hint" to the DEFLATE decompression function
which is used for the initial buffer allocation. Due to the changes
done there, additional allocations and extra copies are avoided if the
initial allocation is sufficient. Set the size hint to MCLBYTES (=2k)
in IPCOMP which should be good for many use cases.
 1.13 18-Feb-2011  drochner more "const"
 1.12 25-Mar-2009  darran branches: 1.12.4; 1.12.6; 1.12.8;
Fixes PR kern/41069 and PR kern/41070.

Extends the Opencrypto API to allow the destination buffer size to be
specified when its not the same size as the input buffer (i.e. for
operations like compress and decompress).
The crypto_op and crypt_n_op structures gain a u_int dst_len field.
The session_op structure gains a comp_alg field to specify a compression
algorithm.
Moved four ioctls to new ids; CIOCGSESSION, CIOCNGSESSION, CIOCCRYPT,
and CIOCNCRYPTM.
Added four backward compatible ioctls; OCIOCGSESSION, OCIOCNGSESSION,
OCIOCCRYPT, and OCIOCNCRYPTM.

Backward compatibility is maintained in ocryptodev.h and ocryptodev.c which
implement the original ioctls and set dst_len and comp_alg to 0.

Adds user-space access to compression features.

Adds software gzip support (CRYPTO_GZIP_COMP).

Adds the fast version of crc32 from zlib to libkern. This should be generally
useful and provide a place to start normalizing the various crc32 routines
in the kernel. The crc32 routine is used in this patch to support GZIP.

With input and support from tls@NetBSD.org.
 1.11 18-Mar-2009  cegger bzero -> memset
 1.10 14-Mar-2009  dsl Change about 4500 of the K&R function definitions to ANSI ones.
There are still about 1600 left, but they have ',' or /* ... */
in the actual variable definitions - which my awk script doesn't handle.
There are also many that need () -> (void).
(The script does handle misordered arguments.)
 1.9 17-Dec-2008  cegger branches: 1.9.2;
kill MALLOC and FREE macros.
 1.8 04-Feb-2008  tls branches: 1.8.6; 1.8.10; 1.8.18; 1.8.20; 1.8.26;
Rework opencrypto to use a spin mutex (crypto_mtx) instead of "splcrypto"
(actually splnet) and condvars instead of tsleep/wakeup. Fix a few
miscellaneous problems and add some debugging printfs while there.

Restore set of CRYPTO_F_DONE in crypto_done() which was lost at some
point after this code came from FreeBSD -- it made it impossible to wait
properly for a condition.

Add flags analogous to the "crp" flags to the key operation's krp struct.
Add a new flag, CRYPTO_F_ONRETQ which tells us a request finished before
the kthread had a chance to dequeue it and call its callback -- this was
letting requests stick on the queues before even though done and copied
out.

Callers of crypto_newsession() or crypto_freesession() must now take the
mutex. Change netipsec to do so. Dispatch takes the mutex itself as
needed.

This was tested fairly extensively with the cryptosoft backend and lightly
with a new hardware driver. It has not been tested with FAST_IPSEC; I am
unable to ascertain whether FAST_IPSEC currently works at all in our tree.

pjd@FreeBSD.ORG, ad@NetBSD.ORG, and darran@snark.us pointed me in the
right direction several times in the course of this. Remaining bugs
are mine alone.
 1.7 02-Feb-2008  tls From Darran Hunt at Coyote Point: don't truncate HMAC to 96 bits unless
actually asked to.

Fixed in FreeBSD a while ago, discussed on tech-kern and tech-crypto.
 1.6 01-Feb-2008  tls don't allocate AES keys with M_WAITOK!
 1.5 04-Mar-2007  christos branches: 1.5.16; 1.5.22;
Kill caddr_t; there will be some MI fallout, but it will be fixed shortly.
 1.4 16-Nov-2006  christos branches: 1.4.4;
__unused removal on arguments; approved by core.
 1.3 27-Oct-2006  christos Merge kernel and userland rmd160 and sha2 implementation.
XXX: We still install rmd160.h and sha2.h in /usr/include/crypto, unlike
the other hash functions which get installed in /usr/include for compatibility.
 1.2 12-Oct-2006  christos - sprinkle __unused on function decls.
- fix a couple of unused bugs
- no more -Wno-unused for i386
 1.1 25-Nov-2005  thorpej branches: 1.1.2; 1.1.4; 1.1.22; 1.1.26; 1.1.28;
- De-couple the software crypto implementation from the rest of the
framework. There is no need to waste the space if you are only using
algoritms provided by hardware accelerators. To get the software
implementations, add "pseudo-device swcr" to your kernel config.
- Lazily initialize the opencrypto framework when crypto drivers
(either hardware or swcr) register themselves with the framework.
 1.1.28.2 10-Dec-2006  yamt sync with head.
 1.1.28.1 22-Oct-2006  yamt sync with head
 1.1.26.1 18-Nov-2006  ad Sync with head.
 1.1.22.5 04-Feb-2008  yamt sync with head.
 1.1.22.4 03-Sep-2007  yamt sync with head.
 1.1.22.3 30-Dec-2006  yamt sync with head.
 1.1.22.2 21-Jun-2006  yamt sync with head.
 1.1.22.1 25-Nov-2005  yamt file cryptosoft_xform.c was added on branch yamt-lazymbuf on 2006-06-21 15:12:02 +0000
 1.1.4.2 11-Dec-2005  christos Sync with head.
 1.1.4.1 25-Nov-2005  christos file cryptosoft_xform.c was added on branch ktrace-lwp on 2005-12-11 10:29:36 +0000
 1.1.2.2 29-Nov-2005  yamt sync with head.
 1.1.2.1 25-Nov-2005  yamt file cryptosoft_xform.c was added on branch yamt-readahead on 2005-11-29 21:23:33 +0000
 1.4.4.1 12-Mar-2007  rmind Sync with HEAD.
 1.5.22.1 18-Feb-2008  mjf Sync with HEAD.
 1.5.16.1 23-Mar-2008  matt sync with HEAD
 1.8.26.1 20-May-2011  matt bring matt-nb5-mips64 up to date with netbsd-5-1-RELEASE (except compat).
 1.8.20.1 03-May-2009  snj Pull up following revision(s) (requested by tls in ticket #611):
sys/lib/libkern/Makefile: patch
sys/lib/libkern/crc32.c: revision 1.1
sys/lib/libkern/crc32.h: revision 1.1
sys/lib/libkern/libkern.h: revision 1.89
sys/lib/libkern/arch/i386/Makefile.inc: revision 1.28
sys/net/zlib.h: revision 1.14 via patch
sys/opencrypto/crypto.c: revision 1.33
sys/opencrypto/cryptodev.c: revision 1.46
sys/opencrypto/cryptodev.h: revision 1.16
sys/opencrypto/cryptosoft.c: revision 1.24
sys/opencrypto/cryptosoft.h: revision 1.6
sys/opencrypto/deflate.h: revision 1.6
sys/opencrypto/cryptosoft_xform.c: revision 1.12
sys/opencrypto/deflate.c: revision 1.13
sys/opencrypto/files.opencrypto: revision 1.20
sys/opencrypto/ocryptodev.c: revision 1.1
sys/opencrypto/ocryptodev.h: revision 1.1
sys/opencrypto/xform.c: revision 1.18
sys/opencrypto/xform.h: revision 1.10
Fixes PR kern/41069 and PR kern/41070.

Extends the Opencrypto API to allow the destination buffer size to be
specified when its not the same size as the input buffer (i.e. for
operations like compress and decompress).
The crypto_op and crypt_n_op structures gain a u_int dst_len field.
The session_op structure gains a comp_alg field to specify a compression
algorithm.
Moved four ioctls to new ids; CIOCGSESSION, CIOCNGSESSION, CIOCCRYPT,
and CIOCNCRYPTM.
Added four backward compatible ioctls; OCIOCGSESSION, OCIOCNGSESSION,
OCIOCCRYPT, and OCIOCNCRYPTM.

Backward compatibility is maintained in ocryptodev.h and ocryptodev.c which
implement the original ioctls and set dst_len and comp_alg to 0.

Adds user-space access to compression features.

Adds software gzip support (CRYPTO_GZIP_COMP).

Adds the fast version of crc32 from zlib to libkern. This should be generally
useful and provide a place to start normalizing the various crc32 routines
in the kernel. The crc32 routine is used in this patch to support GZIP.

With input and support from tls@NetBSD.org.
 1.8.18.2 28-Apr-2009  skrll Sync with HEAD.
 1.8.18.1 19-Jan-2009  skrll Sync with HEAD.
 1.8.10.1 04-May-2009  yamt sync with head.
 1.8.6.1 17-Jan-2009  mjf Sync with HEAD.
 1.9.2.1 13-May-2009  jym Sync with HEAD.

Commit is split, to avoid a "too many arguments" protocol error.
 1.12.8.1 05-Mar-2011  bouyer Sync with HEAD
 1.12.6.1 06-Jun-2011  jruoho Sync with HEAD.
 1.12.4.3 12-Jun-2011  rmind sync with head
 1.12.4.2 31-May-2011  rmind sync with head
 1.12.4.1 05-Mar-2011  rmind sync with head
 1.23.2.1 23-Jun-2011  cherry Catchup with rmind-uvmplock merge.
 1.24.2.2 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.24.2.1 17-Apr-2012  yamt sync with head
 1.25.8.2 03-Dec-2017  jdolecek update from HEAD
 1.25.8.1 25-Feb-2013  tls resync with head
 1.27.20.1 13-Apr-2020  martin Mostly merge changes from HEAD upto 20200411
 1.23 17-May-2017  knakahara opencrypto: cleanup debug messages.
 1.22 26-Mar-2015  prlw1 branches: 1.22.8;
Trivial printf format changes and typo fix
 1.21 03-Jul-2011  mrg branches: 1.21.12; 1.21.30;
avoid some uninitialised warning issues GCC 4.5 complains about.
 1.20 09-Mar-2011  drochner -start to make the GZIP code similar to DEFLATE: make error handling
work the same way, grow output buffer exponentially and kill
reallocation of metadata
-minor cleanup, make definitions private which are implementation
details of deflate.gzip
 1.19 24-Feb-2011  drochner small modifications in dealing with the unknown result size of compression/
decompression:
-seperate the IPCOMP specific rule that compression must not grow the
data from general compression semantics: Introduce a special name
CRYPTO_DEFLATE_COMP_NOGROW/comp_algo_deflate_nogrow to describe
the IPCOMP semantics and use it there. (being here, fix the check
so that equal size is considered failure as well as required by
RFC2393)
Customers of CRYPTO_DEFLATE_COMP/comp_algo_deflate now always get
deflated data back, even if they are not smaller than the original.
-allow to pass a "size hint" to the DEFLATE decompression function
which is used for the initial buffer allocation. Due to the changes
done there, additional allocations and extra copies are avoided if the
initial allocation is sufficient. Set the size hint to MCLBYTES (=2k)
in IPCOMP which should be good for many use cases.
 1.18 18-Feb-2011  drochner redo result buffer allocation, to avoid dynamic allocations:
-use exponentially growing buffer sizes instead of just linear extension
-drop the dynamic allocation of buffer metadata introduced in rev.1.8 --
if the initial array is not sufficient something is wrong
-apply some (arbitrary, heuristic) limit so that compressed data
which extract into insane amounts of constant data don't kill the system
This addresses PR kern/36864 by Wolfgang Stukenbrock. Some tuning
might be useful, but hopefully this is an improvement already.
 1.17 18-Feb-2011  drochner address the obvious byteorder and alignment problems in gzip size/crc
tail, should fix PR kern/44210 by Wolfgang Stukenbrock
being here, fix a bug in crc calculation of decompressed data, and
actually verify the crc
 1.16 17-Feb-2011  drochner The decompressor in sys/net/zlib.c has a bug: It returns Z_BUF_ERROR after
a successful decompression in rare cases. A necessary but not sufficient
condition seems to be that the decompressed data end exactly at the end
of an allocated output buffer. (I can reproduce this reliably with
a userland program built against kernel zlib. Userland libz is much
newer and not affected.)
Since kernel zlib is based on an old version and heavily modified, I don't
dare to touch it. So catch this case in the wrapper.
Being here, reorder deflate/inflate error handling and add comments
to make understandable what is tested and why.
 1.15 16-Feb-2011  drochner -avoid allocation of an extra result buffer and data copy in case
the DEFLATE complssion/decompression result is within a single
buffer already
-simplify bookkeeping of allocated buffers (and don't waste the
last member of the metadata array)
from Wolfgang Stukenbrock per PR kern/36865 (with some cleanup
of error handling by me)
The Gzip compression case can be improved too, but for now I've applied
the buffer bookkeeping changes.

tested with IP4 IPCOMP
 1.14 10-Feb-2011  drochner whan compressing, set the Z_FINISH flag to zlib to tell that
the data chunk is the final one, which makes that zlib issues the
proper termination marker
(KAME IPSEC does this, but doesn't check eagerly in the receive
path, so the missing termination didn't cause problems so far)
closes my PR kern/44539
being here, replace the Z_PARTIAL_FLUSH flag which is marked
deprecated by zlib by Z_SYNC_FLUSH in the decompression path
(tested with IPv4 IPCOMP on i386)
 1.13 25-Mar-2009  darran branches: 1.13.4; 1.13.6; 1.13.8;
Fixes PR kern/41069 and PR kern/41070.

Extends the Opencrypto API to allow the destination buffer size to be
specified when its not the same size as the input buffer (i.e. for
operations like compress and decompress).
The crypto_op and crypt_n_op structures gain a u_int dst_len field.
The session_op structure gains a comp_alg field to specify a compression
algorithm.
Moved four ioctls to new ids; CIOCGSESSION, CIOCNGSESSION, CIOCCRYPT,
and CIOCNCRYPTM.
Added four backward compatible ioctls; OCIOCGSESSION, OCIOCNGSESSION,
OCIOCCRYPT, and OCIOCNCRYPTM.

Backward compatibility is maintained in ocryptodev.h and ocryptodev.c which
implement the original ioctls and set dst_len and comp_alg to 0.

Adds user-space access to compression features.

Adds software gzip support (CRYPTO_GZIP_COMP).

Adds the fast version of crc32 from zlib to libkern. This should be generally
useful and provide a place to start normalizing the various crc32 routines
in the kernel. The crc32 routine is used in this patch to support GZIP.

With input and support from tls@NetBSD.org.
 1.12 14-Mar-2009  dsl Change about 4500 of the K&R function definitions to ANSI ones.
There are still about 1600 left, but they have ',' or /* ... */
in the actual variable definitions - which my awk script doesn't handle.
There are also many that need () -> (void).
(The script does handle misordered arguments.)
 1.11 05-May-2008  ad branches: 1.11.8; 1.11.10; 1.11.14; 1.11.18;
Back out previous. It broke the build.
 1.10 04-May-2008  ad Move zlib out of net/ and into kern/. It would probably be better to use
the reachover Makefiles and libz, but this is already here and it works.
 1.9 22-Sep-2007  degroote branches: 1.9.24;
Fix a stupid bogus bcopy -> memcpy conversion.
I will write it 1000 times : when I convert bcopy to memcpy, don't forget to
swap first and secund args.
 1.8 21-May-2007  degroote branches: 1.8.6; 1.8.8;
Use dynamic array instead of an static array to decompress. It lets us to
decompress any data, whatever is the radio decompressed data / compressed
data.

It fixes the last issues with fast_ipsec and ipcomp.

While here, bzero -> memset, bcopy -> memcpy, FREE -> free

Reviewed a long time ago by sam@
 1.7 16-Nov-2006  christos branches: 1.7.2; 1.7.8; 1.7.10; 1.7.16;
__unused removal on arguments; approved by core.
 1.6 12-Oct-2006  christos - sprinkle __unused on function decls.
- fix a couple of unused bugs
- no more -Wno-unused for i386
 1.5 17-Mar-2006  christos branches: 1.5.8; 1.5.10;
don't use MALLOC with a non-constant size; use malloc instead.
 1.4 11-Dec-2005  christos branches: 1.4.4; 1.4.6; 1.4.8; 1.4.10; 1.4.12;
merge ktrace-lwp.
 1.3 26-Feb-2005  perry branches: 1.3.4;
nuke trailing whitespace
 1.2 27-Aug-2003  thorpej branches: 1.2.4; 1.2.10; 1.2.12;
Tidy up the namespace of this a little.
 1.1 25-Jul-2003  jonathan Commit initial NetBSD port of the OpenCrypto Framework (OCF). This
code is derived from Sam Leffler's FreeBSD port of OCF, which is in
turn a port of Angelos Keromytis's OpenBSD work.
Credit to Sam and Angelos, any blame for the NetBSD port to me.
 1.2.12.1 19-Mar-2005  yamt sync with head. xen and whitespace. xen part is not finished.
 1.2.10.1 29-Apr-2005  kent sync with -current
 1.2.4.5 04-Mar-2005  skrll Sync with HEAD.

Hi Perry!
 1.2.4.4 21-Sep-2004  skrll Fix the sync with head I botched.
 1.2.4.3 18-Sep-2004  skrll Sync with HEAD.
 1.2.4.2 03-Aug-2004  skrll Sync with HEAD
 1.2.4.1 27-Aug-2003  skrll file deflate.c was added on branch ktrace-lwp on 2004-08-03 10:56:25 +0000
 1.3.4.3 27-Oct-2007  yamt sync with head.
 1.3.4.2 03-Sep-2007  yamt sync with head.
 1.3.4.1 21-Jun-2006  yamt sync with head.
 1.4.12.1 28-Mar-2006  tron Merge 2006-03-28 NetBSD-current into the "peter-altq" branch.
 1.4.10.1 19-Apr-2006  elad sync with head.
 1.4.8.1 01-Apr-2006  yamt sync with head.
 1.4.6.1 22-Apr-2006  simonb Sync with head.
 1.4.4.1 09-Sep-2006  rpaulo sync with head
 1.5.10.2 10-Dec-2006  yamt sync with head.
 1.5.10.1 22-Oct-2006  yamt sync with head
 1.5.8.1 18-Nov-2006  ad Sync with head.
 1.7.16.2 30-Sep-2007  wrstuden Catch up on netbsd-4 as of a few days ago.
 1.7.16.1 04-Jun-2007  wrstuden Update to today's netbsd-4.
 1.7.10.1 11-Jul-2007  mjf Sync with head.
 1.7.8.2 09-Oct-2007  ad Sync with head.
 1.7.8.1 08-Jun-2007  ad Sync with head.
 1.7.2.2 25-Sep-2007  xtraeme Pull up following revision(s) (requested by degroote in ticket #897):
sys/opencrypto/deflate.c: revision 1.9

Fix a stupid bogus bcopy -> memcpy conversion.
I will write it 1000 times : when I convert bcopy to memcpy, don't forget to
swap first and secund args.
 1.7.2.1 24-May-2007  pavel Pull up following revision(s) (requested by degroote in ticket #667):
sys/netinet/tcp_input.c: revision 1.260
sys/netinet/tcp_output.c: revision 1.154
sys/netinet/tcp_subr.c: revision 1.210
sys/netinet6/icmp6.c: revision 1.129
sys/netinet6/in6_proto.c: revision 1.70
sys/netinet6/ip6_forward.c: revision 1.54
sys/netinet6/ip6_input.c: revision 1.94
sys/netinet6/ip6_output.c: revision 1.114
sys/netinet6/raw_ip6.c: revision 1.81
sys/netipsec/ipcomp_var.h: revision 1.4
sys/netipsec/ipsec.c: revision 1.26 via patch,1.31-1.32
sys/netipsec/ipsec6.h: revision 1.5
sys/netipsec/ipsec_input.c: revision 1.14
sys/netipsec/ipsec_netbsd.c: revision 1.18,1.26
sys/netipsec/ipsec_output.c: revision 1.21 via patch
sys/netipsec/key.c: revision 1.33,1.44
sys/netipsec/xform_ipcomp.c: revision 1.9
sys/netipsec/xform_ipip.c: revision 1.15
sys/opencrypto/deflate.c: revision 1.8
Commit my SoC work
Add ipv6 support for fast_ipsec
Note that currently, packet with extensions headers are not correctly
supported
Change the ipcomp logic

Add sysctl tree to modify the fast_ipsec options related to ipv6. Similar
to the sysctl kame interface.

Choose the good default policy, depending of the adress family of the
desired policy

Increase the refcount for the default ipv6 policy so nobody can reclaim it

Always compute the sp index even if we don't have any sp in spd. It will
let us to choose the right default policy (based on the adress family
requested).
While here, fix an error message

Use dynamic array instead of an static array to decompress. It lets us to
decompress any data, whatever is the radio decompressed data / compressed
data.
It fixes the last issues with fast_ipsec and ipcomp.
While here, bzero -> memset, bcopy -> memcpy, FREE -> free
Reviewed a long time ago by sam@
 1.8.8.1 06-Nov-2007  matt sync with HEAD
 1.8.6.1 02-Oct-2007  joerg Sync with HEAD.
 1.9.24.1 04-May-2009  yamt sync with head.
 1.11.18.1 20-May-2011  matt bring matt-nb5-mips64 up to date with netbsd-5-1-RELEASE (except compat).
 1.11.14.1 13-May-2009  jym Sync with HEAD.

Commit is split, to avoid a "too many arguments" protocol error.
 1.11.10.1 03-May-2009  snj Pull up following revision(s) (requested by tls in ticket #611):
sys/lib/libkern/Makefile: patch
sys/lib/libkern/crc32.c: revision 1.1
sys/lib/libkern/crc32.h: revision 1.1
sys/lib/libkern/libkern.h: revision 1.89
sys/lib/libkern/arch/i386/Makefile.inc: revision 1.28
sys/net/zlib.h: revision 1.14 via patch
sys/opencrypto/crypto.c: revision 1.33
sys/opencrypto/cryptodev.c: revision 1.46
sys/opencrypto/cryptodev.h: revision 1.16
sys/opencrypto/cryptosoft.c: revision 1.24
sys/opencrypto/cryptosoft.h: revision 1.6
sys/opencrypto/deflate.h: revision 1.6
sys/opencrypto/cryptosoft_xform.c: revision 1.12
sys/opencrypto/deflate.c: revision 1.13
sys/opencrypto/files.opencrypto: revision 1.20
sys/opencrypto/ocryptodev.c: revision 1.1
sys/opencrypto/ocryptodev.h: revision 1.1
sys/opencrypto/xform.c: revision 1.18
sys/opencrypto/xform.h: revision 1.10
Fixes PR kern/41069 and PR kern/41070.

Extends the Opencrypto API to allow the destination buffer size to be
specified when its not the same size as the input buffer (i.e. for
operations like compress and decompress).
The crypto_op and crypt_n_op structures gain a u_int dst_len field.
The session_op structure gains a comp_alg field to specify a compression
algorithm.
Moved four ioctls to new ids; CIOCGSESSION, CIOCNGSESSION, CIOCCRYPT,
and CIOCNCRYPTM.
Added four backward compatible ioctls; OCIOCGSESSION, OCIOCNGSESSION,
OCIOCCRYPT, and OCIOCNCRYPTM.

Backward compatibility is maintained in ocryptodev.h and ocryptodev.c which
implement the original ioctls and set dst_len and comp_alg to 0.

Adds user-space access to compression features.

Adds software gzip support (CRYPTO_GZIP_COMP).

Adds the fast version of crc32 from zlib to libkern. This should be generally
useful and provide a place to start normalizing the various crc32 routines
in the kernel. The crc32 routine is used in this patch to support GZIP.

With input and support from tls@NetBSD.org.
 1.11.8.1 28-Apr-2009  skrll Sync with HEAD.
 1.13.8.2 05-Mar-2011  bouyer Sync with HEAD
 1.13.8.1 17-Feb-2011  bouyer Sync with HEAD
 1.13.6.1 06-Jun-2011  jruoho Sync with HEAD.
 1.13.4.2 21-Apr-2011  rmind sync with head
 1.13.4.1 05-Mar-2011  rmind sync with head
 1.21.30.2 28-Aug-2017  skrll Sync with HEAD
 1.21.30.1 06-Apr-2015  skrll Sync with HEAD
 1.21.12.1 03-Dec-2017  jdolecek update from HEAD
 1.22.8.1 19-May-2017  pgoyette Resolve conflicts from previous merge (all resulting from $NetBSD
keywork expansion)
 1.9 09-Mar-2011  drochner -start to make the GZIP code similar to DEFLATE: make error handling
work the same way, grow output buffer exponentially and kill
reallocation of metadata
-minor cleanup, make definitions private which are implementation
details of deflate.gzip
 1.8 24-Feb-2011  drochner small modifications in dealing with the unknown result size of compression/
decompression:
-seperate the IPCOMP specific rule that compression must not grow the
data from general compression semantics: Introduce a special name
CRYPTO_DEFLATE_COMP_NOGROW/comp_algo_deflate_nogrow to describe
the IPCOMP semantics and use it there. (being here, fix the check
so that equal size is considered failure as well as required by
RFC2393)
Customers of CRYPTO_DEFLATE_COMP/comp_algo_deflate now always get
deflated data back, even if they are not smaller than the original.
-allow to pass a "size hint" to the DEFLATE decompression function
which is used for the initial buffer allocation. Due to the changes
done there, additional allocations and extra copies are avoided if the
initial allocation is sufficient. Set the size hint to MCLBYTES (=2k)
in IPCOMP which should be good for many use cases.
 1.7 16-Feb-2011  drochner -avoid allocation of an extra result buffer and data copy in case
the DEFLATE complssion/decompression result is within a single
buffer already
-simplify bookkeeping of allocated buffers (and don't waste the
last member of the metadata array)
from Wolfgang Stukenbrock per PR kern/36865 (with some cleanup
of error handling by me)
The Gzip compression case can be improved too, but for now I've applied
the buffer bookkeeping changes.

tested with IP4 IPCOMP
 1.6 25-Mar-2009  darran branches: 1.6.4; 1.6.6; 1.6.8;
Fixes PR kern/41069 and PR kern/41070.

Extends the Opencrypto API to allow the destination buffer size to be
specified when its not the same size as the input buffer (i.e. for
operations like compress and decompress).
The crypto_op and crypt_n_op structures gain a u_int dst_len field.
The session_op structure gains a comp_alg field to specify a compression
algorithm.
Moved four ioctls to new ids; CIOCGSESSION, CIOCNGSESSION, CIOCCRYPT,
and CIOCNCRYPTM.
Added four backward compatible ioctls; OCIOCGSESSION, OCIOCNGSESSION,
OCIOCCRYPT, and OCIOCNCRYPTM.

Backward compatibility is maintained in ocryptodev.h and ocryptodev.c which
implement the original ioctls and set dst_len and comp_alg to 0.

Adds user-space access to compression features.

Adds software gzip support (CRYPTO_GZIP_COMP).

Adds the fast version of crc32 from zlib to libkern. This should be generally
useful and provide a place to start normalizing the various crc32 routines
in the kernel. The crc32 routine is used in this patch to support GZIP.

With input and support from tls@NetBSD.org.
 1.5 05-May-2008  ad branches: 1.5.8; 1.5.10; 1.5.14; 1.5.18;
Back out previous. It broke the build.
 1.4 04-May-2008  ad Move zlib out of net/ and into kern/. It would probably be better to use
the reachover Makefiles and libz, but this is already here and it works.
 1.3 11-Dec-2005  christos branches: 1.3.74;
merge ktrace-lwp.
 1.2 26-Feb-2005  perry nuke trailing whitespace
 1.1 25-Jul-2003  jonathan branches: 1.1.4; 1.1.10; 1.1.12;
Commit initial NetBSD port of the OpenCrypto Framework (OCF). This
code is derived from Sam Leffler's FreeBSD port of OCF, which is in
turn a port of Angelos Keromytis's OpenBSD work.
Credit to Sam and Angelos, any blame for the NetBSD port to me.
 1.1.12.1 19-Mar-2005  yamt sync with head. xen and whitespace. xen part is not finished.
 1.1.10.1 29-Apr-2005  kent sync with -current
 1.1.4.5 04-Mar-2005  skrll Sync with HEAD.

Hi Perry!
 1.1.4.4 21-Sep-2004  skrll Fix the sync with head I botched.
 1.1.4.3 18-Sep-2004  skrll Sync with HEAD.
 1.1.4.2 03-Aug-2004  skrll Sync with HEAD
 1.1.4.1 25-Jul-2003  skrll file deflate.h was added on branch ktrace-lwp on 2004-08-03 10:56:25 +0000
 1.3.74.1 04-May-2009  yamt sync with head.
 1.5.18.1 20-May-2011  matt bring matt-nb5-mips64 up to date with netbsd-5-1-RELEASE (except compat).
 1.5.14.1 13-May-2009  jym Sync with HEAD.

Commit is split, to avoid a "too many arguments" protocol error.
 1.5.10.1 03-May-2009  snj Pull up following revision(s) (requested by tls in ticket #611):
sys/lib/libkern/Makefile: patch
sys/lib/libkern/crc32.c: revision 1.1
sys/lib/libkern/crc32.h: revision 1.1
sys/lib/libkern/libkern.h: revision 1.89
sys/lib/libkern/arch/i386/Makefile.inc: revision 1.28
sys/net/zlib.h: revision 1.14 via patch
sys/opencrypto/crypto.c: revision 1.33
sys/opencrypto/cryptodev.c: revision 1.46
sys/opencrypto/cryptodev.h: revision 1.16
sys/opencrypto/cryptosoft.c: revision 1.24
sys/opencrypto/cryptosoft.h: revision 1.6
sys/opencrypto/deflate.h: revision 1.6
sys/opencrypto/cryptosoft_xform.c: revision 1.12
sys/opencrypto/deflate.c: revision 1.13
sys/opencrypto/files.opencrypto: revision 1.20
sys/opencrypto/ocryptodev.c: revision 1.1
sys/opencrypto/ocryptodev.h: revision 1.1
sys/opencrypto/xform.c: revision 1.18
sys/opencrypto/xform.h: revision 1.10
Fixes PR kern/41069 and PR kern/41070.

Extends the Opencrypto API to allow the destination buffer size to be
specified when its not the same size as the input buffer (i.e. for
operations like compress and decompress).
The crypto_op and crypt_n_op structures gain a u_int dst_len field.
The session_op structure gains a comp_alg field to specify a compression
algorithm.
Moved four ioctls to new ids; CIOCGSESSION, CIOCNGSESSION, CIOCCRYPT,
and CIOCNCRYPTM.
Added four backward compatible ioctls; OCIOCGSESSION, OCIOCNGSESSION,
OCIOCCRYPT, and OCIOCNCRYPTM.

Backward compatibility is maintained in ocryptodev.h and ocryptodev.c which
implement the original ioctls and set dst_len and comp_alg to 0.

Adds user-space access to compression features.

Adds software gzip support (CRYPTO_GZIP_COMP).

Adds the fast version of crc32 from zlib to libkern. This should be generally
useful and provide a place to start normalizing the various crc32 routines
in the kernel. The crc32 routine is used in this patch to support GZIP.

With input and support from tls@NetBSD.org.
 1.5.8.1 28-Apr-2009  skrll Sync with HEAD.
 1.6.8.2 05-Mar-2011  bouyer Sync with HEAD
 1.6.8.1 17-Feb-2011  bouyer Sync with HEAD
 1.6.6.1 06-Jun-2011  jruoho Sync with HEAD.
 1.6.4.2 21-Apr-2011  rmind sync with head
 1.6.4.1 05-Mar-2011  rmind sync with head
 1.2 25-Aug-2003  thorpej These are unused; remove them.
 1.1 25-Jul-2003  jonathan Commit initial NetBSD port of the OpenCrypto Framework (OCF). This
code is derived from Sam Leffler's FreeBSD port of OCF, which is in
turn a port of Angelos Keromytis's OpenBSD work.
Credit to Sam and Angelos, any blame for the NetBSD port to me.
 1.2 25-Aug-2003  thorpej These are unused; remove them.
 1.1 25-Jul-2003  jonathan Commit initial NetBSD port of the OpenCrypto Framework (OCF). This
code is derived from Sam Leffler's FreeBSD port of OCF, which is in
turn a port of Angelos Keromytis's OpenBSD work.
Credit to Sam and Angelos, any blame for the NetBSD port to me.
 1.30 29-Jun-2020  riastradh opencrypto: Switch from legacy rijndael API to new aes API.

While here, apply various rijndael->aes renames, reduce the size
of aesxcbc_ctx by 480 bytes, and convert some malloc->kmem.

Leave in the symbol enc_xform_rijndael128 for now, though, so this
doesn't break any kernel ABI.
 1.29 22-Apr-2020  rin Make crypto/rijindael optional again as cprng_strong does no longer
depend on it. Dependency is explicitly declared in files.foo if a
component requires it.
 1.28 27-Jan-2020  pgoyette branches: 1.28.4;
Split the module glue out from the rest of opencrypto/ocryptodev to
make rump happy.

Rump doesn't have compat modules (the compat code is included in the
relevant librump*.so), so there's no module compat_50 listed in
link_set_modules, and thus ocryptodev's MODULE(...) can't "require"
it.

This fixes the problem of "built-in module compat_50 not found" when
starting up rump_allserver (or rump_server with -l rumpdev_opencrypto).

XXX This does not resolve the long-standing "crypto: unable to
XXX register devsw, error 17" message noted at line 78 of
XXX sys/rump/dev/lib/libopencrypto/opencrypto_component.c
 1.27 27-Jan-2019  pgoyette branches: 1.27.6;
Merge the [pgoyette-compat] branch
 1.26 24-May-2017  knakahara branches: 1.26.8; 1.26.10;
make the default values of q_maxlen build parameters.
 1.25 19-Nov-2011  tls branches: 1.25.8; 1.25.26;
First step of random number subsystem rework described in
<20111022023242.BA26F14A158@mail.netbsd.org>. This change includes
the following:

An initial cleanup and minor reorganization of the entropy pool
code in sys/dev/rnd.c and sys/dev/rndpool.c. Several bugs are
fixed. Some effort is made to accumulate entropy more quickly at
boot time.

A generic interface, "rndsink", is added, for stream generators to
request that they be re-keyed with good quality entropy from the pool
as soon as it is available.

The arc4random()/arc4randbytes() implementation in libkern is
adjusted to use the rndsink interface for rekeying, which helps
address the problem of low-quality keys at boot time.

An implementation of the FIPS 140-2 statistical tests for random
number generator quality is provided (libkern/rngtest.c). This
is based on Greg Rose's implementation from Qualcomm.

A new random stream generator, nist_ctr_drbg, is provided. It is
based on an implementation of the NIST SP800-90 CTR_DRBG by
Henric Jungheim. This generator users AES in a modified counter
mode to generate a backtracking-resistant random stream.

An abstraction layer, "cprng", is provided for in-kernel consumers
of randomness. The arc4random/arc4randbytes API is deprecated for
in-kernel use. It is replaced by "cprng_strong". The current
cprng_fast implementation wraps the existing arc4random
implementation. The current cprng_strong implementation wraps the
new CTR_DRBG implementation. Both interfaces are rekeyed from
the entropy pool automatically at intervals justifiable from best
current cryptographic practice.

In some quick tests, cprng_fast() is about the same speed as
the old arc4randbytes(), and cprng_strong() is about 20% faster
than rnd_extract_data(). Performance is expected to improve.

The AES code in src/crypto/rijndael is no longer an optional
kernel component, as it is required by cprng_strong, which is
not an optional kernel component.

The entropy pool output is subjected to the rngtest tests at
startup time; if it fails, the system will reboot. There is
approximately a 3/10000 chance of a false positive from these
tests. Entropy pool _input_ from hardware random numbers is
subjected to the rngtest tests at attach time, as well as the
FIPS continuous-output test, to detect bad or stuck hardware
RNGs; if any are detected, they are detached, but the system
continues to run.

A problem with rndctl(8) is fixed -- datastructures with
pointers in arrays are no longer passed to userspace (this
was not a security problem, but rather a major issue for
compat32). A new kernel will require a new rndctl.

The sysctl kern.arandom() and kern.urandom() nodes are hooked
up to the new generators, but the /dev/*random pseudodevices
are not, yet.

Manual pages for the new kernel interfaces are forthcoming.
 1.24 26-May-2011  drochner branches: 1.24.4;
pull in AES-GCM/GMAC support from OpenBSD
This is still somewhat experimental. Tested between 2 similar boxes
so far. There is much potential for performance improvement. For now,
I've changed the gmac code to accept any data alignment, as the "char *"
pointer suggests. As the code is practically used, 32-bit alignment
can be assumed, at the cost of data copies. I don't know whether
bytewise access or copies are worse performance-wise. For efficient
implementations using SSE2 instructions on x86, even stricter
alignment requirements might arise.
 1.23 24-May-2011  drochner copy AES-XCBC-MAC support from KAME IPSEC to FAST_IPSEC
For this to fit, an API change in cryptosoft was adopted from OpenBSD
(addition of a "Setkey" method to hashes) which was done for GCM/GMAC
support there, so it might be useful in the future anyway.
tested against KAME IPSEC
AFAICT, FAST_IPSEC now supports as much as KAME.
 1.22 05-May-2011  drochner support camellia-cbc by swcrypt
 1.21 19-Feb-2011  drochner make the compatibility code conditional on COMPAT_50
 1.20 25-Mar-2009  darran branches: 1.20.4; 1.20.6; 1.20.8;
Fixes PR kern/41069 and PR kern/41070.

Extends the Opencrypto API to allow the destination buffer size to be
specified when its not the same size as the input buffer (i.e. for
operations like compress and decompress).
The crypto_op and crypt_n_op structures gain a u_int dst_len field.
The session_op structure gains a comp_alg field to specify a compression
algorithm.
Moved four ioctls to new ids; CIOCGSESSION, CIOCNGSESSION, CIOCCRYPT,
and CIOCNCRYPTM.
Added four backward compatible ioctls; OCIOCGSESSION, OCIOCNGSESSION,
OCIOCCRYPT, and OCIOCNCRYPTM.

Backward compatibility is maintained in ocryptodev.h and ocryptodev.c which
implement the original ioctls and set dst_len and comp_alg to 0.

Adds user-space access to compression features.

Adds software gzip support (CRYPTO_GZIP_COMP).

Adds the fast version of crc32 from zlib to libkern. This should be generally
useful and provide a place to start normalizing the various crc32 routines
in the kernel. The crc32 routine is used in this patch to support GZIP.

With input and support from tls@NetBSD.org.
 1.19 04-Feb-2008  tls branches: 1.19.10; 1.19.18; 1.19.20; 1.19.24; 1.19.28;
Rework opencrypto to use a spin mutex (crypto_mtx) instead of "splcrypto"
(actually splnet) and condvars instead of tsleep/wakeup. Fix a few
miscellaneous problems and add some debugging printfs while there.

Restore set of CRYPTO_F_DONE in crypto_done() which was lost at some
point after this code came from FreeBSD -- it made it impossible to wait
properly for a condition.

Add flags analogous to the "crp" flags to the key operation's krp struct.
Add a new flag, CRYPTO_F_ONRETQ which tells us a request finished before
the kthread had a chance to dequeue it and call its callback -- this was
letting requests stick on the queues before even though done and copied
out.

Callers of crypto_newsession() or crypto_freesession() must now take the
mutex. Change netipsec to do so. Dispatch takes the mutex itself as
needed.

This was tested fairly extensively with the cryptosoft backend and lightly
with a new hardware driver. It has not been tested with FAST_IPSEC; I am
unable to ascertain whether FAST_IPSEC currently works at all in our tree.

pjd@FreeBSD.ORG, ad@NetBSD.ORG, and darran@snark.us pointed me in the
right direction several times in the course of this. Remaining bugs
are mine alone.
 1.18 27-Oct-2006  christos branches: 1.18.24; 1.18.30;
Merge kernel and userland rmd160 and sha2 implementation.
XXX: We still install rmd160.h and sha2.h in /usr/include/crypto, unlike
the other hash functions which get installed in /usr/include for compatibility.
 1.17 25-Nov-2005  thorpej branches: 1.17.20; 1.17.22;
swcr -> swcrypto
 1.16 25-Nov-2005  thorpej - De-couple the software crypto implementation from the rest of the
framework. There is no need to waste the space if you are only using
algoritms provided by hardware accelerators. To get the software
implementations, add "pseudo-device swcr" to your kernel config.
- Lazily initialize the opencrypto framework when crypto drivers
(either hardware or swcr) register themselves with the framework.
 1.15 26-Feb-2005  perry branches: 1.15.4; 1.15.10;
nuke trailing whitespace
 1.14 31-Dec-2003  jonathan branches: 1.14.4; 1.14.10; 1.14.12;
Split opencrypto configuration into an attribute, usable by inkernel
clients, and a pseudo-device for userspace access.

The attribute is named `opencrypto'. The pseudo-device is renamed to
"crypto", which has a dependency on "opencrypto". The sys/conf/majors
entry and pseudo-device attach entrypoint are updated to match the
new pseudo-device name.

Fast IPsec (sys/netipsec/files.ipsec) now lists a dependency on the
"opencrypto" attribute. Drivers for crypto accelerators (ubsec,
hifn775x) also pull in opencrypto, as providers of opencrypto transforms.
 1.13 16-Nov-2003  tls Move the Skipjack algorithm from sys/opencrypto to sys/crypto/skipjack.
There are now no cryptographic algorithms in sys/opencrypto, which,
according to the comment formerly in files.opencrypto, was the original
intent.
 1.12 27-Aug-2003  thorpej Use the Blowfish in crypto/blowfish, which has hooks for using tuned
assembly for the transform.
 1.11 26-Aug-2003  thorpej Move the opencrypto CAST-128 implementation to crypto/cast128, removing
the old one. Rename the functions/structures from cast_* to cast128_*.
Adapt the KAME IPsec to use the new CAST-128 code, which has a simpler
API and smaller footprint.
 1.10 26-Aug-2003  thorpej G/C opencrypto's local copy of rijndael. Adapt the extant rijndael
code for opencrypto's use by adding the simplified API that opencrypto
expects.
 1.9 24-Aug-2003  thorpej crypto_mbuf.c is unused; remove it.
 1.8 24-Aug-2003  thorpej G/C some unused config definitions.
 1.7 24-Aug-2003  thorpej Make opencrypto depend on the "ripemd160" and "sha2" attributes, rather
than polluting the crypto algorithm config info with opencrypto knowledge.
 1.6 24-Aug-2003  thorpej Move the opencrypto defpseudo into files.opencrypto.
 1.5 30-Jul-2003  jonathan Add ``needs-flag'' to opencrypto/crypto.c, so that init_main() can do
a feature-test and if present, initialize the crypto framework before
configuring hardware devices, so the crypto framework is ready to accept
attachment requests when crypto hardware are auto-configured and
try to register themselves with the framework.

Thanks to Quentin Garnier who, given a heads-up about the problem,
tested the same basic fix with a hardware device.
 1.4 28-Jul-2003  jonathan Clean up dependencies between opencrypto framework and sys/crypto,
by making the sys/crypto transforms depend on
(per-transform-config-keyword|opencrypto)

Thanks to Martin Husemann for double-checking the changes.
 1.3 27-Jul-2003  jonathan Cleanup traces of previous standalone m_apply()/m_getptr().
 1.2 26-Jul-2003  he Don't declare the sha2.c file twice, instead add opencrypto flag to
existing declaration, and leave behind a comment in files.opencrypto.
 1.1 25-Jul-2003  jonathan Commit initial NetBSD port of the OpenCrypto Framework (OCF). This
code is derived from Sam Leffler's FreeBSD port of OCF, which is in
turn a port of Angelos Keromytis's OpenBSD work.
Credit to Sam and Angelos, any blame for the NetBSD port to me.
 1.14.12.1 19-Mar-2005  yamt sync with head. xen and whitespace. xen part is not finished.
 1.14.10.1 29-Apr-2005  kent sync with -current
 1.14.4.6 11-Dec-2005  christos Sync with head.
 1.14.4.5 04-Mar-2005  skrll Sync with HEAD.

Hi Perry!
 1.14.4.4 21-Sep-2004  skrll Fix the sync with head I botched.
 1.14.4.3 18-Sep-2004  skrll Sync with HEAD.
 1.14.4.2 03-Aug-2004  skrll Sync with HEAD
 1.14.4.1 31-Dec-2003  skrll file files.opencrypto was added on branch ktrace-lwp on 2004-08-03 10:56:25 +0000
 1.15.10.1 29-Nov-2005  yamt sync with head.
 1.15.4.3 04-Feb-2008  yamt sync with head.
 1.15.4.2 30-Dec-2006  yamt sync with head.
 1.15.4.1 21-Jun-2006  yamt sync with head.
 1.17.22.1 10-Dec-2006  yamt sync with head.
 1.17.20.1 18-Nov-2006  ad Sync with head.
 1.18.30.1 18-Feb-2008  mjf Sync with HEAD.
 1.18.24.1 23-Mar-2008  matt sync with HEAD
 1.19.28.1 20-May-2011  matt bring matt-nb5-mips64 up to date with netbsd-5-1-RELEASE (except compat).
 1.19.24.1 13-May-2009  jym Sync with HEAD.

Commit is split, to avoid a "too many arguments" protocol error.
 1.19.20.1 03-May-2009  snj Pull up following revision(s) (requested by tls in ticket #611):
sys/lib/libkern/Makefile: patch
sys/lib/libkern/crc32.c: revision 1.1
sys/lib/libkern/crc32.h: revision 1.1
sys/lib/libkern/libkern.h: revision 1.89
sys/lib/libkern/arch/i386/Makefile.inc: revision 1.28
sys/net/zlib.h: revision 1.14 via patch
sys/opencrypto/crypto.c: revision 1.33
sys/opencrypto/cryptodev.c: revision 1.46
sys/opencrypto/cryptodev.h: revision 1.16
sys/opencrypto/cryptosoft.c: revision 1.24
sys/opencrypto/cryptosoft.h: revision 1.6
sys/opencrypto/deflate.h: revision 1.6
sys/opencrypto/cryptosoft_xform.c: revision 1.12
sys/opencrypto/deflate.c: revision 1.13
sys/opencrypto/files.opencrypto: revision 1.20
sys/opencrypto/ocryptodev.c: revision 1.1
sys/opencrypto/ocryptodev.h: revision 1.1
sys/opencrypto/xform.c: revision 1.18
sys/opencrypto/xform.h: revision 1.10
Fixes PR kern/41069 and PR kern/41070.

Extends the Opencrypto API to allow the destination buffer size to be
specified when its not the same size as the input buffer (i.e. for
operations like compress and decompress).
The crypto_op and crypt_n_op structures gain a u_int dst_len field.
The session_op structure gains a comp_alg field to specify a compression
algorithm.
Moved four ioctls to new ids; CIOCGSESSION, CIOCNGSESSION, CIOCCRYPT,
and CIOCNCRYPTM.
Added four backward compatible ioctls; OCIOCGSESSION, OCIOCNGSESSION,
OCIOCCRYPT, and OCIOCNCRYPTM.

Backward compatibility is maintained in ocryptodev.h and ocryptodev.c which
implement the original ioctls and set dst_len and comp_alg to 0.

Adds user-space access to compression features.

Adds software gzip support (CRYPTO_GZIP_COMP).

Adds the fast version of crc32 from zlib to libkern. This should be generally
useful and provide a place to start normalizing the various crc32 routines
in the kernel. The crc32 routine is used in this patch to support GZIP.

With input and support from tls@NetBSD.org.
 1.19.18.1 28-Apr-2009  skrll Sync with HEAD.
 1.19.10.1 04-May-2009  yamt sync with head.
 1.20.8.1 05-Mar-2011  bouyer Sync with HEAD
 1.20.6.1 06-Jun-2011  jruoho Sync with HEAD.
 1.20.4.2 31-May-2011  rmind sync with head
 1.20.4.1 05-Mar-2011  rmind sync with head
 1.24.4.1 17-Apr-2012  yamt sync with head
 1.25.26.1 28-Aug-2017  skrll Sync with HEAD
 1.25.8.1 03-Dec-2017  jdolecek update from HEAD
 1.26.10.2 08-Apr-2020  martin Merge changes from current as of 20200406
 1.26.10.1 10-Jun-2019  christos Sync with HEAD
 1.26.8.2 23-Sep-2018  pgoyette Split the compat_crypto_50 from the rest of the crypto module

Cleanup some stuff left over from similar changes to raid modules.
 1.26.8.1 22-Sep-2018  pgoyette Include the compat code whether or not the calling device or filesystem
exists.
 1.27.6.1 29-Feb-2020  ad Sync with head.
 1.28.4.1 25-Apr-2020  bouyer Sync with bouyer-xenpvh-base2 (HEAD)
 1.4 29-Jun-2020  riastradh opencrypto: Switch from legacy rijndael API to new aes API.

While here, apply various rijndael->aes renames, reduce the size
of aesxcbc_ctx by 480 bytes, and convert some malloc->kmem.

Leave in the symbol enc_xform_rijndael128 for now, though, so this
doesn't break any kernel ABI.
 1.3 09-Jun-2011  drochner use 64-bit integers for GF128 multiplication on LP64 CPUs
 1.2 08-Jun-2011  drochner reduce typecasts and byte swapping
 1.1 26-May-2011  drochner branches: 1.1.2; 1.1.4; 1.1.6;
pull in AES-GCM/GMAC support from OpenBSD
This is still somewhat experimental. Tested between 2 similar boxes
so far. There is much potential for performance improvement. For now,
I've changed the gmac code to accept any data alignment, as the "char *"
pointer suggests. As the code is practically used, 32-bit alignment
can be assumed, at the cost of data copies. I don't know whether
bytewise access or copies are worse performance-wise. For efficient
implementations using SSE2 instructions on x86, even stricter
alignment requirements might arise.
 1.1.6.2 06-Jun-2011  jruoho Sync with HEAD.
 1.1.6.1 26-May-2011  jruoho file gmac.c was added on branch jruoho-x86intr on 2011-06-06 09:10:04 +0000
 1.1.4.1 23-Jun-2011  cherry Catchup with rmind-uvmplock merge.
 1.1.2.3 12-Jun-2011  rmind sync with head
 1.1.2.2 31-May-2011  rmind sync with head
 1.1.2.1 26-May-2011  rmind file gmac.c was added on branch rmind-uvmplock on 2011-05-31 03:05:10 +0000
 1.3 29-Jun-2020  riastradh opencrypto: Switch from legacy rijndael API to new aes API.

While here, apply various rijndael->aes renames, reduce the size
of aesxcbc_ctx by 480 bytes, and convert some malloc->kmem.

Leave in the symbol enc_xform_rijndael128 for now, though, so this
doesn't break any kernel ABI.
 1.2 09-Jun-2011  drochner use 64-bit integers for GF128 multiplication on LP64 CPUs
 1.1 26-May-2011  drochner branches: 1.1.2; 1.1.4; 1.1.6;
pull in AES-GCM/GMAC support from OpenBSD
This is still somewhat experimental. Tested between 2 similar boxes
so far. There is much potential for performance improvement. For now,
I've changed the gmac code to accept any data alignment, as the "char *"
pointer suggests. As the code is practically used, 32-bit alignment
can be assumed, at the cost of data copies. I don't know whether
bytewise access or copies are worse performance-wise. For efficient
implementations using SSE2 instructions on x86, even stricter
alignment requirements might arise.
 1.1.6.2 06-Jun-2011  jruoho Sync with HEAD.
 1.1.6.1 26-May-2011  jruoho file gmac.h was added on branch jruoho-x86intr on 2011-06-06 09:10:04 +0000
 1.1.4.1 23-Jun-2011  cherry Catchup with rmind-uvmplock merge.
 1.1.2.3 12-Jun-2011  rmind sync with head
 1.1.2.2 31-May-2011  rmind sync with head
 1.1.2.1 26-May-2011  rmind file gmac.h was added on branch rmind-uvmplock on 2011-05-31 03:05:10 +0000
 1.17 12-Mar-2022  riastradh crypto(4): Refuse count>1 for old CIOCNCRYPTM.

This hasn't worked since it was written in 2009; if anyone cared
surely they would have fixed it by now!

(Fixing this properly -- and putting a more reasonable upper bound
than the maximum that size_t arithmetic allows -- left as an exercise
or the reader.)

Reported-by: syzbot+798d4a16bc15ae88526e@syzkaller.appspotmail.com
 1.16 27-Jan-2020  pgoyette Split the module glue out from the rest of opencrypto/ocryptodev to
make rump happy.

Rump doesn't have compat modules (the compat code is included in the
relevant librump*.so), so there's no module compat_50 listed in
link_set_modules, and thus ocryptodev's MODULE(...) can't "require"
it.

This fixes the problem of "built-in module compat_50 not found" when
starting up rump_allserver (or rump_server with -l rumpdev_opencrypto).

XXX This does not resolve the long-standing "crypto: unable to
XXX register devsw, error 17" message noted at line 78 of
XXX sys/rump/dev/lib/libopencrypto/opencrypto_component.c
 1.15 16-Jan-2020  christos Initialize the session variable to an impossible session to prevent compiler
warnings.
 1.14 12-Dec-2019  pgoyette branches: 1.14.2;
Rather than keeping a separate mutex, condvar, and pserialize for each
module hook, we can share a common set of synchronization structures.
This cuts the amount of cacheline_aligned data for these structures by
50%.

Note that we still have a per-hook localcount, since we need to count
individual references.

As discussed with riastradh@

Welcome to 9.99.22 !
 1.13 01-Mar-2019  pgoyette Rename the MODULE_*_HOOK() macros to MODULE_HOOK_*() as briefly
discussed on irc.

NFCI intended.

Ride the earlier kernel bump - it;s getting crowded.
 1.12 27-Jan-2019  pgoyette Merge the [pgoyette-compat] branch
 1.11 28-Jul-2017  riastradh branches: 1.11.2; 1.11.4;
Avoid another userland-controlled integer overflow.

From Ilja Van Sprundel.
 1.10 28-Jul-2017  riastradh Avert userland-controlled integer overflow.

From Ilja Van Sprundel.
 1.9 28-Jul-2017  riastradh Don't disclose uninitialized 32-bit word if cryptodev_session fails.

From Ilja Van Sprundel.
 1.8 02-Jun-2017  knakahara branches: 1.8.2;
rename crypto_mtx to cryptodev_mtx

It is used by cryptodev.c and ocryptodev.c only.
 1.7 17-May-2017  knakahara opencrypto: cleanup debug messages.
 1.6 05-Sep-2014  matt branches: 1.6.2; 1.6.10;
Try not to use f_data, use f_fcrypt to get a correctly typed pointer.
 1.5 01-Jan-2014  pgoyette Modularize the opencrypto components and link to the build
 1.4 16-May-2011  drochner branches: 1.4.2; 1.4.6; 1.4.16; 1.4.20;
split the "crypto_mtx" spinlock into 3: one spinlock each for
the incoming and outgoing request queues (which can be dealt with
by hardware accelerators) and an adaptive lock for "all the rest"
(mostly driver configuration, but also some unrelated stuff in
cryptodev.c which should be revisited)
The latter one seems to be uneeded at many places, but for now I've
done simple replacements only, except minor fixes (where
softint_schedule() was called without the lock held)
 1.3 19-Feb-2011  drochner make the compatibility code conditional on COMPAT_50
 1.2 26-Mar-2009  mrg branches: 1.2.2; 1.2.4; 1.2.6; 1.2.8; 1.2.12; 1.2.14; 1.2.16;
avoid a useless uninitialised use, picked up by gcc -O3.
 1.1 25-Mar-2009  darran Fixes PR kern/41069 and PR kern/41070.

Extends the Opencrypto API to allow the destination buffer size to be
specified when its not the same size as the input buffer (i.e. for
operations like compress and decompress).
The crypto_op and crypt_n_op structures gain a u_int dst_len field.
The session_op structure gains a comp_alg field to specify a compression
algorithm.
Moved four ioctls to new ids; CIOCGSESSION, CIOCNGSESSION, CIOCCRYPT,
and CIOCNCRYPTM.
Added four backward compatible ioctls; OCIOCGSESSION, OCIOCNGSESSION,
OCIOCCRYPT, and OCIOCNCRYPTM.

Backward compatibility is maintained in ocryptodev.h and ocryptodev.c which
implement the original ioctls and set dst_len and comp_alg to 0.

Adds user-space access to compression features.

Adds software gzip support (CRYPTO_GZIP_COMP).

Adds the fast version of crc32 from zlib to libkern. This should be generally
useful and provide a place to start normalizing the various crc32 routines
in the kernel. The crc32 routine is used in this patch to support GZIP.

With input and support from tls@NetBSD.org.
 1.2.16.1 05-Mar-2011  bouyer Sync with HEAD
 1.2.14.1 06-Jun-2011  jruoho Sync with HEAD.
 1.2.12.2 31-May-2011  rmind sync with head
 1.2.12.1 05-Mar-2011  rmind sync with head
 1.2.8.2 13-May-2009  jym Sync with HEAD.

Commit is split, to avoid a "too many arguments" protocol error.
 1.2.8.1 26-Mar-2009  jym file ocryptodev.c was added on branch jym-xensuspend on 2009-05-13 17:22:56 +0000
 1.2.6.2 04-May-2009  yamt sync with head.
 1.2.6.1 26-Mar-2009  yamt file ocryptodev.c was added on branch yamt-nfs-mp on 2009-05-04 08:14:24 +0000
 1.2.4.2 03-May-2009  snj Pull up following revision(s) (requested by tls in ticket #611):
sys/lib/libkern/Makefile: patch
sys/lib/libkern/crc32.c: revision 1.1
sys/lib/libkern/crc32.h: revision 1.1
sys/lib/libkern/libkern.h: revision 1.89
sys/lib/libkern/arch/i386/Makefile.inc: revision 1.28
sys/net/zlib.h: revision 1.14 via patch
sys/opencrypto/crypto.c: revision 1.33
sys/opencrypto/cryptodev.c: revision 1.46
sys/opencrypto/cryptodev.h: revision 1.16
sys/opencrypto/cryptosoft.c: revision 1.24
sys/opencrypto/cryptosoft.h: revision 1.6
sys/opencrypto/deflate.h: revision 1.6
sys/opencrypto/cryptosoft_xform.c: revision 1.12
sys/opencrypto/deflate.c: revision 1.13
sys/opencrypto/files.opencrypto: revision 1.20
sys/opencrypto/ocryptodev.c: revision 1.1
sys/opencrypto/ocryptodev.h: revision 1.1
sys/opencrypto/xform.c: revision 1.18
sys/opencrypto/xform.h: revision 1.10
Fixes PR kern/41069 and PR kern/41070.

Extends the Opencrypto API to allow the destination buffer size to be
specified when its not the same size as the input buffer (i.e. for
operations like compress and decompress).
The crypto_op and crypt_n_op structures gain a u_int dst_len field.
The session_op structure gains a comp_alg field to specify a compression
algorithm.
Moved four ioctls to new ids; CIOCGSESSION, CIOCNGSESSION, CIOCCRYPT,
and CIOCNCRYPTM.
Added four backward compatible ioctls; OCIOCGSESSION, OCIOCNGSESSION,
OCIOCCRYPT, and OCIOCNCRYPTM.

Backward compatibility is maintained in ocryptodev.h and ocryptodev.c which
implement the original ioctls and set dst_len and comp_alg to 0.

Adds user-space access to compression features.

Adds software gzip support (CRYPTO_GZIP_COMP).

Adds the fast version of crc32 from zlib to libkern. This should be generally
useful and provide a place to start normalizing the various crc32 routines
in the kernel. The crc32 routine is used in this patch to support GZIP.

With input and support from tls@NetBSD.org.
 1.2.4.1 26-Mar-2009  snj file ocryptodev.c was added on branch netbsd-5 on 2009-05-03 17:24:45 +0000
 1.2.2.2 28-Apr-2009  skrll Sync with HEAD.
 1.2.2.1 26-Mar-2009  skrll file ocryptodev.c was added on branch nick-hppapmap on 2009-04-28 07:37:50 +0000
 1.4.20.1 18-May-2014  rmind sync with head
 1.4.16.2 03-Dec-2017  jdolecek update from HEAD
 1.4.16.1 20-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.4.6.1 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.4.2.2 20-May-2011  matt bring matt-nb5-mips64 up to date with netbsd-5-1-RELEASE (except compat).
 1.4.2.1 16-May-2011  matt file ocryptodev.c was added on branch matt-nb5-mips64 on 2011-05-20 08:11:32 +0000
 1.6.10.1 19-May-2017  pgoyette Resolve conflicts from previous merge (all resulting from $NetBSD
keywork expansion)
 1.6.2.1 28-Aug-2017  skrll Sync with HEAD
 1.8.2.1 05-Aug-2017  snj Pull up following revision(s) (requested by knakahara in ticket #178):
sys/opencrypto/crypto.c: 1.92-1.100
sys/opencrypto/cryptodev.h: 1.38-1.39
sys/opencrypto/ocryptodev.c: 1.9-1.11
sys/opencrypto/xform.c: revision 1.29
sys/opencrypto/xform.h: revision 1.20
KNF
--
Apply C99-style struct initialization to enc_xform, auth_hash and comp_algo
--
make cryptoret() context softint to balance dequeuing crypto_ret_q with enqueuing it.
--
fix panic when using ecryption devices attached earlier than ipi_sysinit().
pointed out and tested by martin@n.o, thanks.
--
fix typo
--
make crp_{,k}q percpu to scale crypto_dispatch().
update locking note later.
--
divide crp_ret_{,k}q by CPU to avoid reordering.
update locking note later.
--
update locking notes of opencrypto(9)
--
Don't disclose uninitialized 32-bit word if cryptodev_session fails.
From Ilja Van Sprundel.
--
Avert userland-controlled integer overflow.
From Ilja Van Sprundel.
--
Avoid another userland-controlled integer overflow.
From Ilja Van Sprundel.
--
refactor: remove glue macros for FreeBSD code.
--
pack crypto_drivers variables to struct and add cacheline_aligned qualifier.
--
use kmem_alloc KPI instead of malloc KPI.
--
use pool_cache(9) instead of pool(9) as they can be called concurrently.
 1.11.4.2 08-Apr-2020  martin Merge changes from current as of 20200406
 1.11.4.1 10-Jun-2019  christos Sync with HEAD
 1.11.2.8 23-Jan-2019  pgoyette Convert the macros for setting and unsetting a hook to generate
in-line code rather than using an intermediary hook##set routine.
Hooks are set and unset only in one place, so the intermediary
routine provides no benefit. IMHO using the macro at the point-
of-call is more readable than using it elsewhere in the code and
then calling the generated intermediary routine (for which you
won't even find its declaration or definition unless you remember
to search for the HOOK_SET macro instead).

NFC intended, will verify with a bulk build and an atf test run.
 1.11.2.7 23-Sep-2018  pgoyette Fix some thinkos/pastos/typos
 1.11.2.6 23-Sep-2018  pgoyette Split the compat_crypto_50 from the rest of the crypto module

Cleanup some stuff left over from similar changes to raid modules.
 1.11.2.5 22-Sep-2018  pgoyette Since the caller passes the address of our mutex, use it rather than
trying to reference it via global symbol (which might not exist).
 1.11.2.4 22-Sep-2018  pgoyette When the compat code needs to callback to the original code, we cannot
call directly via the routines' global symbols, since the original code
might not be built-in. So, the original code that calls compat code
needs to pass in the addresses of the callbacks. This allows for the
compat code to be built whether or not the original (calling) code is
included.

XXX Done for cryptodev, will need to do the same thing for ccd(4) and
XXX vnd(4)
 1.11.2.3 18-Sep-2018  pgoyette The COMPAT_HOOK macros were renamed to MODULE_HOOK, adjust all callers
 1.11.2.2 17-Sep-2018  pgoyette Adapt (most of) the indirect function pointers to the new MP-safe
mechanism. Still remaining are the compat_netbsd32 stuff, and
some usb subroutines.
 1.11.2.1 23-Mar-2018  pgoyette Handle the compat_50 stuff for opencrypto/cryptodev
 1.14.2.2 29-Feb-2020  ad Sync with head.
 1.14.2.1 17-Jan-2020  ad Sync with head.
 1.4 27-Jan-2019  pgoyette Merge the [pgoyette-compat] branch
 1.3 06-Sep-2015  dholland branches: 1.3.16; 1.3.18;
More on PR 41200: headers that declare ioctls should include sys/ioccom.h.
This covers (I think) all the MI headers outside of external/ (and dist/).
 1.2 19-Feb-2011  drochner branches: 1.2.2; 1.2.16; 1.2.34;
make the compatibility code conditional on COMPAT_50
 1.1 25-Mar-2009  darran branches: 1.1.2; 1.1.4; 1.1.6; 1.1.8; 1.1.12; 1.1.14; 1.1.16;
Fixes PR kern/41069 and PR kern/41070.

Extends the Opencrypto API to allow the destination buffer size to be
specified when its not the same size as the input buffer (i.e. for
operations like compress and decompress).
The crypto_op and crypt_n_op structures gain a u_int dst_len field.
The session_op structure gains a comp_alg field to specify a compression
algorithm.
Moved four ioctls to new ids; CIOCGSESSION, CIOCNGSESSION, CIOCCRYPT,
and CIOCNCRYPTM.
Added four backward compatible ioctls; OCIOCGSESSION, OCIOCNGSESSION,
OCIOCCRYPT, and OCIOCNCRYPTM.

Backward compatibility is maintained in ocryptodev.h and ocryptodev.c which
implement the original ioctls and set dst_len and comp_alg to 0.

Adds user-space access to compression features.

Adds software gzip support (CRYPTO_GZIP_COMP).

Adds the fast version of crc32 from zlib to libkern. This should be generally
useful and provide a place to start normalizing the various crc32 routines
in the kernel. The crc32 routine is used in this patch to support GZIP.

With input and support from tls@NetBSD.org.
 1.1.16.1 05-Mar-2011  bouyer Sync with HEAD
 1.1.14.1 06-Jun-2011  jruoho Sync with HEAD.
 1.1.12.1 05-Mar-2011  rmind sync with head
 1.1.8.2 13-May-2009  jym Sync with HEAD.

Commit is split, to avoid a "too many arguments" protocol error.
 1.1.8.1 25-Mar-2009  jym file ocryptodev.h was added on branch jym-xensuspend on 2009-05-13 17:22:56 +0000
 1.1.6.2 04-May-2009  yamt sync with head.
 1.1.6.1 25-Mar-2009  yamt file ocryptodev.h was added on branch yamt-nfs-mp on 2009-05-04 08:14:24 +0000
 1.1.4.2 03-May-2009  snj Pull up following revision(s) (requested by tls in ticket #611):
sys/lib/libkern/Makefile: patch
sys/lib/libkern/crc32.c: revision 1.1
sys/lib/libkern/crc32.h: revision 1.1
sys/lib/libkern/libkern.h: revision 1.89
sys/lib/libkern/arch/i386/Makefile.inc: revision 1.28
sys/net/zlib.h: revision 1.14 via patch
sys/opencrypto/crypto.c: revision 1.33
sys/opencrypto/cryptodev.c: revision 1.46
sys/opencrypto/cryptodev.h: revision 1.16
sys/opencrypto/cryptosoft.c: revision 1.24
sys/opencrypto/cryptosoft.h: revision 1.6
sys/opencrypto/deflate.h: revision 1.6
sys/opencrypto/cryptosoft_xform.c: revision 1.12
sys/opencrypto/deflate.c: revision 1.13
sys/opencrypto/files.opencrypto: revision 1.20
sys/opencrypto/ocryptodev.c: revision 1.1
sys/opencrypto/ocryptodev.h: revision 1.1
sys/opencrypto/xform.c: revision 1.18
sys/opencrypto/xform.h: revision 1.10
Fixes PR kern/41069 and PR kern/41070.

Extends the Opencrypto API to allow the destination buffer size to be
specified when its not the same size as the input buffer (i.e. for
operations like compress and decompress).
The crypto_op and crypt_n_op structures gain a u_int dst_len field.
The session_op structure gains a comp_alg field to specify a compression
algorithm.
Moved four ioctls to new ids; CIOCGSESSION, CIOCNGSESSION, CIOCCRYPT,
and CIOCNCRYPTM.
Added four backward compatible ioctls; OCIOCGSESSION, OCIOCNGSESSION,
OCIOCCRYPT, and OCIOCNCRYPTM.

Backward compatibility is maintained in ocryptodev.h and ocryptodev.c which
implement the original ioctls and set dst_len and comp_alg to 0.

Adds user-space access to compression features.

Adds software gzip support (CRYPTO_GZIP_COMP).

Adds the fast version of crc32 from zlib to libkern. This should be generally
useful and provide a place to start normalizing the various crc32 routines
in the kernel. The crc32 routine is used in this patch to support GZIP.

With input and support from tls@NetBSD.org.
 1.1.4.1 25-Mar-2009  snj file ocryptodev.h was added on branch netbsd-5 on 2009-05-03 17:24:45 +0000
 1.1.2.2 28-Apr-2009  skrll Sync with HEAD.
 1.1.2.1 25-Mar-2009  skrll file ocryptodev.h was added on branch nick-hppapmap on 2009-04-28 07:37:50 +0000
 1.2.34.1 22-Sep-2015  skrll Sync with HEAD
 1.2.16.1 03-Dec-2017  jdolecek update from HEAD
 1.2.2.2 20-May-2011  matt bring matt-nb5-mips64 up to date with netbsd-5-1-RELEASE (except compat).
 1.2.2.1 19-Feb-2011  matt file ocryptodev.h was added on branch matt-nb5-mips64 on 2011-05-20 08:11:33 +0000
 1.3.18.1 10-Jun-2019  christos Sync with HEAD
 1.3.16.5 23-Sep-2018  pgoyette No need to declare the crypto_50 init/fini routines. The compat_50
module no longer needs them, and they're declared static in the
ocryptodev.c code
 1.3.16.4 23-Sep-2018  pgoyette Fix some thinkos/pastos/typos
 1.3.16.3 22-Sep-2018  pgoyette Forward-declare some structs so we can use pointers to them in
function declarations.
 1.3.16.2 22-Sep-2018  pgoyette When the compat code needs to callback to the original code, we cannot
call directly via the routines' global symbols, since the original code
might not be built-in. So, the original code that calls compat code
needs to pass in the addresses of the callbacks. This allows for the
compat code to be built whether or not the original (calling) code is
included.

XXX Done for cryptodev, will need to do the same thing for ccd(4) and
XXX vnd(4)
 1.3.16.1 23-Mar-2018  pgoyette Handle the compat_50 stuff for opencrypto/cryptodev
 1.2 26-Aug-2003  thorpej G/C opencrypto's local copy of rijndael. Adapt the extant rijndael
code for opencrypto's use by adding the simplified API that opencrypto
expects.
 1.1 25-Jul-2003  jonathan Commit initial NetBSD port of the OpenCrypto Framework (OCF). This
code is derived from Sam Leffler's FreeBSD port of OCF, which is in
turn a port of Angelos Keromytis's OpenBSD work.
Credit to Sam and Angelos, any blame for the NetBSD port to me.
 1.2 26-Aug-2003  thorpej G/C opencrypto's local copy of rijndael. Adapt the extant rijndael
code for opencrypto's use by adding the simplified API that opencrypto
expects.
 1.1 25-Jul-2003  jonathan Commit initial NetBSD port of the OpenCrypto Framework (OCF). This
code is derived from Sam Leffler's FreeBSD port of OCF, which is in
turn a port of Angelos Keromytis's OpenBSD work.
Credit to Sam and Angelos, any blame for the NetBSD port to me.
 1.2 25-Aug-2003  thorpej It's bad form to use the <opencrypto/rmd160.h> header file while
using the crypto/ripemd160/rmd160.c implementation. Remove the
opencrypto-local copies of these files entirely.
 1.1 25-Jul-2003  jonathan Commit initial NetBSD port of the OpenCrypto Framework (OCF). This
code is derived from Sam Leffler's FreeBSD port of OCF, which is in
turn a port of Angelos Keromytis's OpenBSD work.
Credit to Sam and Angelos, any blame for the NetBSD port to me.
 1.2 25-Aug-2003  thorpej It's bad form to use the <opencrypto/rmd160.h> header file while
using the crypto/ripemd160/rmd160.c implementation. Remove the
opencrypto-local copies of these files entirely.
 1.1 25-Jul-2003  jonathan Commit initial NetBSD port of the OpenCrypto Framework (OCF). This
code is derived from Sam Leffler's FreeBSD port of OCF, which is in
turn a port of Angelos Keromytis's OpenBSD work.
Credit to Sam and Angelos, any blame for the NetBSD port to me.
 1.3 16-Nov-2003  tls Move the Skipjack algorithm from sys/opencrypto to sys/crypto/skipjack.
There are now no cryptographic algorithms in sys/opencrypto, which,
according to the comment formerly in files.opencrypto, was the original
intent.
 1.2 27-Aug-2003  thorpej Some const poisoning.
 1.1 25-Jul-2003  jonathan Commit initial NetBSD port of the OpenCrypto Framework (OCF). This
code is derived from Sam Leffler's FreeBSD port of OCF, which is in
turn a port of Angelos Keromytis's OpenBSD work.
Credit to Sam and Angelos, any blame for the NetBSD port to me.
 1.3 16-Nov-2003  tls Move the Skipjack algorithm from sys/opencrypto to sys/crypto/skipjack.
There are now no cryptographic algorithms in sys/opencrypto, which,
according to the comment formerly in files.opencrypto, was the original
intent.
 1.2 27-Aug-2003  thorpej Some const poisoning.
 1.1 25-Jul-2003  jonathan Commit initial NetBSD port of the OpenCrypto Framework (OCF). This
code is derived from Sam Leffler's FreeBSD port of OCF, which is in
turn a port of Angelos Keromytis's OpenBSD work.
Credit to Sam and Angelos, any blame for the NetBSD port to me.
 1.31 30-Jun-2020  riastradh Rename enc_xform_rijndael128 -> enc_xform_aes.

Update netipsec dependency.
 1.30 29-Jun-2020  riastradh opencrypto: Switch from legacy rijndael API to new aes API.

While here, apply various rijndael->aes renames, reduce the size
of aesxcbc_ctx by 480 bytes, and convert some malloc->kmem.

Leave in the symbol enc_xform_rijndael128 for now, though, so this
doesn't break any kernel ABI.
 1.29 06-Jul-2017  ozaki-r Apply C99-style struct initialization to enc_xform, auth_hash and comp_algo
 1.28 26-May-2011  drochner branches: 1.28.14; 1.28.32; 1.28.48;
pull in AES-GCM/GMAC support from OpenBSD
This is still somewhat experimental. Tested between 2 similar boxes
so far. There is much potential for performance improvement. For now,
I've changed the gmac code to accept any data alignment, as the "char *"
pointer suggests. As the code is practically used, 32-bit alignment
can be assumed, at the cost of data copies. I don't know whether
bytewise access or copies are worse performance-wise. For efficient
implementations using SSE2 instructions on x86, even stricter
alignment requirements might arise.
 1.27 24-May-2011  drochner copy AES-XCBC-MAC support from KAME IPSEC to FAST_IPSEC
For this to fit, an API change in cryptosoft was adopted from OpenBSD
(addition of a "Setkey" method to hashes) which was done for GCM/GMAC
support there, so it might be useful in the future anyway.
tested against KAME IPSEC
AFAICT, FAST_IPSEC now supports as much as KAME.
 1.26 24-May-2011  drochner move the "context size" struct member (which is a pure software
implementation thing) from the abstract xform descriptor to
the cryptosoft implementation part -- for sanity, and now clients
of opencrypto don't depend on headers of cipher implementations anymore
 1.25 23-May-2011  drochner add an AES-CTR xform, from OpenBSD
 1.24 23-May-2011  drochner -in the descriptor for encryption xforms, split the "blocksize" field
into "blocksize" and "IV size"
-add an "reinit" function pointer which, if set, means that the xform
does its IV handling itself and doesn't want the default CBC handling
by the framework (poor name, but left that way to avoid unecessary
differences)
This syncs with Open/FreeBSD, purpose is to allow non-CBC transforms.
Refer to ivsize instead of blocksize where appropriate.
(At this point, blocksize and ivsize are identical.)
 1.23 21-May-2011  drochner sync minimum key size for AES with reality
 1.22 05-May-2011  drochner make camellia-cbc known to the opencrypto framework
 1.21 09-Mar-2011  drochner const'fy algorithm descriptors
 1.20 25-Feb-2011  drochner make the use of SHA2-HMAC by FAST_IPSEC compliant to current standards:
-RFC2104 says that the block size of the hash algorithm must be used
for key/ipad/opad calculations. While formerly all ciphers used a block
length of 64, SHA384 and SHA512 use 128 bytes. So we can't use the
HMAC_BLOCK_LEN constant anymore. Add a new field to "struct auth_hash"
for the per-cipher blocksize.
-Due to this, there can't be a single "CRYPTO_SHA2_HMAC" external name
anymore. Replace this by 3 for the 3 different keysizes.
This was done by Open/FreeBSD before.
-Also fix the number of authenticator bits used tor ESP and AH to
conform to RFC4868, and remove uses of AH_HMAC_HASHLEN which did
assume a fixed authenticator size of 12 bytes.

FAST_IPSEC will not interoperate with KAME IPSEC anymore if sha2 is used,
because the latter doesn't implement these standards. It should
interoperate with at least modern Free/OpenBSD now.
(I've only tested with NetBSD-current/FAST_IPSEC on both ends.)
 1.19 24-Feb-2011  drochner small modifications in dealing with the unknown result size of compression/
decompression:
-seperate the IPCOMP specific rule that compression must not grow the
data from general compression semantics: Introduce a special name
CRYPTO_DEFLATE_COMP_NOGROW/comp_algo_deflate_nogrow to describe
the IPCOMP semantics and use it there. (being here, fix the check
so that equal size is considered failure as well as required by
RFC2393)
Customers of CRYPTO_DEFLATE_COMP/comp_algo_deflate now always get
deflated data back, even if they are not smaller than the original.
-allow to pass a "size hint" to the DEFLATE decompression function
which is used for the initial buffer allocation. Due to the changes
done there, additional allocations and extra copies are avoided if the
initial allocation is sufficient. Set the size hint to MCLBYTES (=2k)
in IPCOMP which should be good for many use cases.
 1.18 25-Mar-2009  darran branches: 1.18.4; 1.18.6; 1.18.8;
Fixes PR kern/41069 and PR kern/41070.

Extends the Opencrypto API to allow the destination buffer size to be
specified when its not the same size as the input buffer (i.e. for
operations like compress and decompress).
The crypto_op and crypt_n_op structures gain a u_int dst_len field.
The session_op structure gains a comp_alg field to specify a compression
algorithm.
Moved four ioctls to new ids; CIOCGSESSION, CIOCNGSESSION, CIOCCRYPT,
and CIOCNCRYPTM.
Added four backward compatible ioctls; OCIOCGSESSION, OCIOCNGSESSION,
OCIOCCRYPT, and OCIOCNCRYPTM.

Backward compatibility is maintained in ocryptodev.h and ocryptodev.c which
implement the original ioctls and set dst_len and comp_alg to 0.

Adds user-space access to compression features.

Adds software gzip support (CRYPTO_GZIP_COMP).

Adds the fast version of crc32 from zlib to libkern. This should be generally
useful and provide a place to start normalizing the various crc32 routines
in the kernel. The crc32 routine is used in this patch to support GZIP.

With input and support from tls@NetBSD.org.
 1.17 28-Feb-2008  tls branches: 1.17.4; 1.17.12; 1.17.14; 1.17.18; 1.17.22;
From Darran Hunt: you can't just specify the "wrong" underlying algorithm
with the "right" output size in the xform declaration and have the _96
HMAC variants work -- the actual algorithm machinery (hardware or software)
ignores the output-size parameter, it's just there to inform the interface
consumer.

This should fix FAST_IPSEC.
 1.16 02-Feb-2008  tls branches: 1.16.2; 1.16.6;
From Darran Hunt at Coyote Point: don't truncate HMAC to 96 bits unless
actually asked to.

Fixed in FreeBSD a while ago, discussed on tech-kern and tech-crypto.
 1.15 25-Nov-2005  thorpej branches: 1.15.46; 1.15.52;
- De-couple the software crypto implementation from the rest of the
framework. There is no need to waste the space if you are only using
algoritms provided by hardware accelerators. To get the software
implementations, add "pseudo-device swcr" to your kernel config.
- Lazily initialize the opencrypto framework when crypto drivers
(either hardware or swcr) register themselves with the framework.
 1.14 29-May-2005  christos branches: 1.14.2; 1.14.8;
Sprinkle const.
 1.13 18-Nov-2003  jonathan branches: 1.13.4;
Remove erroneous '2 *' from 'bzero(*sched, 2 * sizeof(rijndael_ctx));'
After using AES from sys/crypto, we only malloc sizeof(rijndael_ctx),
and we were bzero()ing past the end of the aes ctx, leading to panics.
 1.12 16-Nov-2003  tls Move the Skipjack algorithm from sys/opencrypto to sys/crypto/skipjack.
There are now no cryptographic algorithms in sys/opencrypto, which,
according to the comment formerly in files.opencrypto, was the original
intent.
 1.11 27-Aug-2003  thorpej Fix-up a few things missed in the rijndael_set_key() change.
 1.10 27-Aug-2003  itojun simplify rijndael.c API - always schedule encrypt/decrypt key.
reviewed by thorpej
 1.9 27-Aug-2003  thorpej Some const poisoning.
 1.8 27-Aug-2003  thorpej Use the Blowfish in crypto/blowfish, which has hooks for using tuned
assembly for the transform.
 1.7 26-Aug-2003  thorpej Move the opencrypto CAST-128 implementation to crypto/cast128, removing
the old one. Rename the functions/structures from cast_* to cast128_*.
Adapt the KAME IPsec to use the new CAST-128 code, which has a simpler
API and smaller footprint.
 1.6 26-Aug-2003  thorpej G/C extra /.
 1.5 26-Aug-2003  thorpej G/C opencrypto's local copy of rijndael. Adapt the extant rijndael
code for opencrypto's use by adding the simplified API that opencrypto
expects.
 1.4 25-Aug-2003  thorpej It's bad form to use the <opencrypto/rmd160.h> header file while
using the crypto/ripemd160/rmd160.c implementation. Remove the
opencrypto-local copies of these files entirely.
 1.3 01-Aug-2003  itojun keyed-{sha1,md5} should not be truncated to 96 bits. markus@openbsd
 1.2 28-Jul-2003  jonathan Remove vestiges of OpenBSD <sys/md5k.h> header.
 1.1 25-Jul-2003  jonathan Commit initial NetBSD port of the OpenCrypto Framework (OCF). This
code is derived from Sam Leffler's FreeBSD port of OCF, which is in
turn a port of Angelos Keromytis's OpenBSD work.
Credit to Sam and Angelos, any blame for the NetBSD port to me.
 1.13.4.6 11-Dec-2005  christos Sync with head.
 1.13.4.5 10-Nov-2005  skrll Sync with HEAD. Here we go again...
 1.13.4.4 21-Sep-2004  skrll Fix the sync with head I botched.
 1.13.4.3 18-Sep-2004  skrll Sync with HEAD.
 1.13.4.2 03-Aug-2004  skrll Sync with HEAD
 1.13.4.1 18-Nov-2003  skrll file xform.c was added on branch ktrace-lwp on 2004-08-03 10:56:25 +0000
 1.14.8.1 29-Nov-2005  yamt sync with head.
 1.14.2.3 17-Mar-2008  yamt sync with head.
 1.14.2.2 04-Feb-2008  yamt sync with head.
 1.14.2.1 21-Jun-2006  yamt sync with head.
 1.15.52.1 18-Feb-2008  mjf Sync with HEAD.
 1.15.46.1 23-Mar-2008  matt sync with HEAD
 1.16.6.1 03-Apr-2008  mjf Sync with HEAD.
 1.16.2.1 24-Mar-2008  keiichi sync with head.
 1.17.22.1 20-May-2011  matt bring matt-nb5-mips64 up to date with netbsd-5-1-RELEASE (except compat).
 1.17.18.1 13-May-2009  jym Sync with HEAD.

Commit is split, to avoid a "too many arguments" protocol error.
 1.17.14.1 03-May-2009  snj Pull up following revision(s) (requested by tls in ticket #611):
sys/lib/libkern/Makefile: patch
sys/lib/libkern/crc32.c: revision 1.1
sys/lib/libkern/crc32.h: revision 1.1
sys/lib/libkern/libkern.h: revision 1.89
sys/lib/libkern/arch/i386/Makefile.inc: revision 1.28
sys/net/zlib.h: revision 1.14 via patch
sys/opencrypto/crypto.c: revision 1.33
sys/opencrypto/cryptodev.c: revision 1.46
sys/opencrypto/cryptodev.h: revision 1.16
sys/opencrypto/cryptosoft.c: revision 1.24
sys/opencrypto/cryptosoft.h: revision 1.6
sys/opencrypto/deflate.h: revision 1.6
sys/opencrypto/cryptosoft_xform.c: revision 1.12
sys/opencrypto/deflate.c: revision 1.13
sys/opencrypto/files.opencrypto: revision 1.20
sys/opencrypto/ocryptodev.c: revision 1.1
sys/opencrypto/ocryptodev.h: revision 1.1
sys/opencrypto/xform.c: revision 1.18
sys/opencrypto/xform.h: revision 1.10
Fixes PR kern/41069 and PR kern/41070.

Extends the Opencrypto API to allow the destination buffer size to be
specified when its not the same size as the input buffer (i.e. for
operations like compress and decompress).
The crypto_op and crypt_n_op structures gain a u_int dst_len field.
The session_op structure gains a comp_alg field to specify a compression
algorithm.
Moved four ioctls to new ids; CIOCGSESSION, CIOCNGSESSION, CIOCCRYPT,
and CIOCNCRYPTM.
Added four backward compatible ioctls; OCIOCGSESSION, OCIOCNGSESSION,
OCIOCCRYPT, and OCIOCNCRYPTM.

Backward compatibility is maintained in ocryptodev.h and ocryptodev.c which
implement the original ioctls and set dst_len and comp_alg to 0.

Adds user-space access to compression features.

Adds software gzip support (CRYPTO_GZIP_COMP).

Adds the fast version of crc32 from zlib to libkern. This should be generally
useful and provide a place to start normalizing the various crc32 routines
in the kernel. The crc32 routine is used in this patch to support GZIP.

With input and support from tls@NetBSD.org.
 1.17.12.1 28-Apr-2009  skrll Sync with HEAD.
 1.17.4.1 04-May-2009  yamt sync with head.
 1.18.8.1 05-Mar-2011  bouyer Sync with HEAD
 1.18.6.1 06-Jun-2011  jruoho Sync with HEAD.
 1.18.4.3 31-May-2011  rmind sync with head
 1.18.4.2 21-Apr-2011  rmind sync with head
 1.18.4.1 05-Mar-2011  rmind sync with head
 1.28.48.1 05-Aug-2017  snj Pull up following revision(s) (requested by knakahara in ticket #178):
sys/opencrypto/crypto.c: 1.92-1.100
sys/opencrypto/cryptodev.h: 1.38-1.39
sys/opencrypto/ocryptodev.c: 1.9-1.11
sys/opencrypto/xform.c: revision 1.29
sys/opencrypto/xform.h: revision 1.20
KNF
--
Apply C99-style struct initialization to enc_xform, auth_hash and comp_algo
--
make cryptoret() context softint to balance dequeuing crypto_ret_q with enqueuing it.
--
fix panic when using ecryption devices attached earlier than ipi_sysinit().
pointed out and tested by martin@n.o, thanks.
--
fix typo
--
make crp_{,k}q percpu to scale crypto_dispatch().
update locking note later.
--
divide crp_ret_{,k}q by CPU to avoid reordering.
update locking note later.
--
update locking notes of opencrypto(9)
--
Don't disclose uninitialized 32-bit word if cryptodev_session fails.
From Ilja Van Sprundel.
--
Avert userland-controlled integer overflow.
From Ilja Van Sprundel.
--
Avoid another userland-controlled integer overflow.
From Ilja Van Sprundel.
--
refactor: remove glue macros for FreeBSD code.
--
pack crypto_drivers variables to struct and add cacheline_aligned qualifier.
--
use kmem_alloc KPI instead of malloc KPI.
--
use pool_cache(9) instead of pool(9) as they can be called concurrently.
 1.28.32.1 28-Aug-2017  skrll Sync with HEAD
 1.28.14.1 03-Dec-2017  jdolecek update from HEAD
 1.21 30-Jun-2020  riastradh Rename enc_xform_rijndael128 -> enc_xform_aes.

Update netipsec dependency.
 1.20 06-Jul-2017  ozaki-r KNF
 1.19 26-May-2011  drochner branches: 1.19.14; 1.19.32; 1.19.48;
pull in AES-GCM/GMAC support from OpenBSD
This is still somewhat experimental. Tested between 2 similar boxes
so far. There is much potential for performance improvement. For now,
I've changed the gmac code to accept any data alignment, as the "char *"
pointer suggests. As the code is practically used, 32-bit alignment
can be assumed, at the cost of data copies. I don't know whether
bytewise access or copies are worse performance-wise. For efficient
implementations using SSE2 instructions on x86, even stricter
alignment requirements might arise.
 1.18 24-May-2011  drochner copy AES-XCBC-MAC support from KAME IPSEC to FAST_IPSEC
For this to fit, an API change in cryptosoft was adopted from OpenBSD
(addition of a "Setkey" method to hashes) which was done for GCM/GMAC
support there, so it might be useful in the future anyway.
tested against KAME IPSEC
AFAICT, FAST_IPSEC now supports as much as KAME.
 1.17 24-May-2011  drochner move the "context size" struct member (which is a pure software
implementation thing) from the abstract xform descriptor to
the cryptosoft implementation part -- for sanity, and now clients
of opencrypto don't depend on headers of cipher implementations anymore
 1.16 23-May-2011  drochner add an AES-CTR xform, from OpenBSD
 1.15 23-May-2011  drochner -in the descriptor for encryption xforms, split the "blocksize" field
into "blocksize" and "IV size"
-add an "reinit" function pointer which, if set, means that the xform
does its IV handling itself and doesn't want the default CBC handling
by the framework (poor name, but left that way to avoid unecessary
differences)
This syncs with Open/FreeBSD, purpose is to allow non-CBC transforms.
Refer to ivsize instead of blocksize where appropriate.
(At this point, blocksize and ivsize are identical.)
 1.14 05-May-2011  drochner make camellia-cbc known to the opencrypto framework
 1.13 09-Mar-2011  drochner const'fy algorithm descriptors
 1.12 25-Feb-2011  drochner make the use of SHA2-HMAC by FAST_IPSEC compliant to current standards:
-RFC2104 says that the block size of the hash algorithm must be used
for key/ipad/opad calculations. While formerly all ciphers used a block
length of 64, SHA384 and SHA512 use 128 bytes. So we can't use the
HMAC_BLOCK_LEN constant anymore. Add a new field to "struct auth_hash"
for the per-cipher blocksize.
-Due to this, there can't be a single "CRYPTO_SHA2_HMAC" external name
anymore. Replace this by 3 for the 3 different keysizes.
This was done by Open/FreeBSD before.
-Also fix the number of authenticator bits used tor ESP and AH to
conform to RFC4868, and remove uses of AH_HMAC_HASHLEN which did
assume a fixed authenticator size of 12 bytes.

FAST_IPSEC will not interoperate with KAME IPSEC anymore if sha2 is used,
because the latter doesn't implement these standards. It should
interoperate with at least modern Free/OpenBSD now.
(I've only tested with NetBSD-current/FAST_IPSEC on both ends.)
 1.11 24-Feb-2011  drochner small modifications in dealing with the unknown result size of compression/
decompression:
-seperate the IPCOMP specific rule that compression must not grow the
data from general compression semantics: Introduce a special name
CRYPTO_DEFLATE_COMP_NOGROW/comp_algo_deflate_nogrow to describe
the IPCOMP semantics and use it there. (being here, fix the check
so that equal size is considered failure as well as required by
RFC2393)
Customers of CRYPTO_DEFLATE_COMP/comp_algo_deflate now always get
deflated data back, even if they are not smaller than the original.
-allow to pass a "size hint" to the DEFLATE decompression function
which is used for the initial buffer allocation. Due to the changes
done there, additional allocations and extra copies are avoided if the
initial allocation is sufficient. Set the size hint to MCLBYTES (=2k)
in IPCOMP which should be good for many use cases.
 1.10 25-Mar-2009  darran branches: 1.10.4; 1.10.6; 1.10.8;
Fixes PR kern/41069 and PR kern/41070.

Extends the Opencrypto API to allow the destination buffer size to be
specified when its not the same size as the input buffer (i.e. for
operations like compress and decompress).
The crypto_op and crypt_n_op structures gain a u_int dst_len field.
The session_op structure gains a comp_alg field to specify a compression
algorithm.
Moved four ioctls to new ids; CIOCGSESSION, CIOCNGSESSION, CIOCCRYPT,
and CIOCNCRYPTM.
Added four backward compatible ioctls; OCIOCGSESSION, OCIOCNGSESSION,
OCIOCCRYPT, and OCIOCNCRYPTM.

Backward compatibility is maintained in ocryptodev.h and ocryptodev.c which
implement the original ioctls and set dst_len and comp_alg to 0.

Adds user-space access to compression features.

Adds software gzip support (CRYPTO_GZIP_COMP).

Adds the fast version of crc32 from zlib to libkern. This should be generally
useful and provide a place to start normalizing the various crc32 routines
in the kernel. The crc32 routine is used in this patch to support GZIP.

With input and support from tls@NetBSD.org.
 1.9 02-Feb-2008  tls branches: 1.9.10; 1.9.18; 1.9.20; 1.9.24; 1.9.28;
From Darran Hunt at Coyote Point: don't truncate HMAC to 96 bits unless
actually asked to.

Fixed in FreeBSD a while ago, discussed on tech-kern and tech-crypto.
 1.8 27-Oct-2006  christos branches: 1.8.24; 1.8.30;
Merge kernel and userland rmd160 and sha2 implementation.
XXX: We still install rmd160.h and sha2.h in /usr/include/crypto, unlike
the other hash functions which get installed in /usr/include for compatibility.
 1.7 25-Nov-2005  thorpej branches: 1.7.20; 1.7.22;
- De-couple the software crypto implementation from the rest of the
framework. There is no need to waste the space if you are only using
algoritms provided by hardware accelerators. To get the software
implementations, add "pseudo-device swcr" to your kernel config.
- Lazily initialize the opencrypto framework when crypto drivers
(either hardware or swcr) register themselves with the framework.
 1.6 29-May-2005  christos branches: 1.6.2; 1.6.8;
Sprinkle const.
 1.5 27-Aug-2003  thorpej branches: 1.5.4;
Some const poisoning.
 1.4 26-Aug-2003  thorpej Remove extra /.
 1.3 25-Aug-2003  thorpej It's bad form to use the <opencrypto/rmd160.h> header file while
using the crypto/ripemd160/rmd160.c implementation. Remove the
opencrypto-local copies of these files entirely.
 1.2 28-Jul-2003  jonathan Remove vestiges of OpenBSD <sys/md5k.h> header.
 1.1 25-Jul-2003  jonathan Commit initial NetBSD port of the OpenCrypto Framework (OCF). This
code is derived from Sam Leffler's FreeBSD port of OCF, which is in
turn a port of Angelos Keromytis's OpenBSD work.
Credit to Sam and Angelos, any blame for the NetBSD port to me.
 1.5.4.6 11-Dec-2005  christos Sync with head.
 1.5.4.5 10-Nov-2005  skrll Sync with HEAD. Here we go again...
 1.5.4.4 21-Sep-2004  skrll Fix the sync with head I botched.
 1.5.4.3 18-Sep-2004  skrll Sync with HEAD.
 1.5.4.2 03-Aug-2004  skrll Sync with HEAD
 1.5.4.1 27-Aug-2003  skrll file xform.h was added on branch ktrace-lwp on 2004-08-03 10:56:25 +0000
 1.6.8.1 29-Nov-2005  yamt sync with head.
 1.6.2.3 04-Feb-2008  yamt sync with head.
 1.6.2.2 30-Dec-2006  yamt sync with head.
 1.6.2.1 21-Jun-2006  yamt sync with head.
 1.7.22.1 10-Dec-2006  yamt sync with head.
 1.7.20.1 18-Nov-2006  ad Sync with head.
 1.8.30.1 18-Feb-2008  mjf Sync with HEAD.
 1.8.24.1 23-Mar-2008  matt sync with HEAD
 1.9.28.1 20-May-2011  matt bring matt-nb5-mips64 up to date with netbsd-5-1-RELEASE (except compat).
 1.9.24.1 13-May-2009  jym Sync with HEAD.

Commit is split, to avoid a "too many arguments" protocol error.
 1.9.20.1 03-May-2009  snj Pull up following revision(s) (requested by tls in ticket #611):
sys/lib/libkern/Makefile: patch
sys/lib/libkern/crc32.c: revision 1.1
sys/lib/libkern/crc32.h: revision 1.1
sys/lib/libkern/libkern.h: revision 1.89
sys/lib/libkern/arch/i386/Makefile.inc: revision 1.28
sys/net/zlib.h: revision 1.14 via patch
sys/opencrypto/crypto.c: revision 1.33
sys/opencrypto/cryptodev.c: revision 1.46
sys/opencrypto/cryptodev.h: revision 1.16
sys/opencrypto/cryptosoft.c: revision 1.24
sys/opencrypto/cryptosoft.h: revision 1.6
sys/opencrypto/deflate.h: revision 1.6
sys/opencrypto/cryptosoft_xform.c: revision 1.12
sys/opencrypto/deflate.c: revision 1.13
sys/opencrypto/files.opencrypto: revision 1.20
sys/opencrypto/ocryptodev.c: revision 1.1
sys/opencrypto/ocryptodev.h: revision 1.1
sys/opencrypto/xform.c: revision 1.18
sys/opencrypto/xform.h: revision 1.10
Fixes PR kern/41069 and PR kern/41070.

Extends the Opencrypto API to allow the destination buffer size to be
specified when its not the same size as the input buffer (i.e. for
operations like compress and decompress).
The crypto_op and crypt_n_op structures gain a u_int dst_len field.
The session_op structure gains a comp_alg field to specify a compression
algorithm.
Moved four ioctls to new ids; CIOCGSESSION, CIOCNGSESSION, CIOCCRYPT,
and CIOCNCRYPTM.
Added four backward compatible ioctls; OCIOCGSESSION, OCIOCNGSESSION,
OCIOCCRYPT, and OCIOCNCRYPTM.

Backward compatibility is maintained in ocryptodev.h and ocryptodev.c which
implement the original ioctls and set dst_len and comp_alg to 0.

Adds user-space access to compression features.

Adds software gzip support (CRYPTO_GZIP_COMP).

Adds the fast version of crc32 from zlib to libkern. This should be generally
useful and provide a place to start normalizing the various crc32 routines
in the kernel. The crc32 routine is used in this patch to support GZIP.

With input and support from tls@NetBSD.org.
 1.9.18.1 28-Apr-2009  skrll Sync with HEAD.
 1.9.10.1 04-May-2009  yamt sync with head.
 1.10.8.1 05-Mar-2011  bouyer Sync with HEAD
 1.10.6.1 06-Jun-2011  jruoho Sync with HEAD.
 1.10.4.3 31-May-2011  rmind sync with head
 1.10.4.2 21-Apr-2011  rmind sync with head
 1.10.4.1 05-Mar-2011  rmind sync with head
 1.19.48.1 05-Aug-2017  snj Pull up following revision(s) (requested by knakahara in ticket #178):
sys/opencrypto/crypto.c: 1.92-1.100
sys/opencrypto/cryptodev.h: 1.38-1.39
sys/opencrypto/ocryptodev.c: 1.9-1.11
sys/opencrypto/xform.c: revision 1.29
sys/opencrypto/xform.h: revision 1.20
KNF
--
Apply C99-style struct initialization to enc_xform, auth_hash and comp_algo
--
make cryptoret() context softint to balance dequeuing crypto_ret_q with enqueuing it.
--
fix panic when using ecryption devices attached earlier than ipi_sysinit().
pointed out and tested by martin@n.o, thanks.
--
fix typo
--
make crp_{,k}q percpu to scale crypto_dispatch().
update locking note later.
--
divide crp_ret_{,k}q by CPU to avoid reordering.
update locking note later.
--
update locking notes of opencrypto(9)
--
Don't disclose uninitialized 32-bit word if cryptodev_session fails.
From Ilja Van Sprundel.
--
Avert userland-controlled integer overflow.
From Ilja Van Sprundel.
--
Avoid another userland-controlled integer overflow.
From Ilja Van Sprundel.
--
refactor: remove glue macros for FreeBSD code.
--
pack crypto_drivers variables to struct and add cacheline_aligned qualifier.
--
use kmem_alloc KPI instead of malloc KPI.
--
use pool_cache(9) instead of pool(9) as they can be called concurrently.
 1.19.32.1 28-Aug-2017  skrll Sync with HEAD
 1.19.14.1 03-Dec-2017  jdolecek update from HEAD

RSS XML Feed