| History log of /src/sys/opencrypto/crypto.c |
| Revision | | Date | Author | Comments |
| 1.131 |
| 26-Jun-2022 |
riastradh | opencrypto(9): Fix missing initialization in error branch.
Reported-by: syzbot+8c519140cac567be1ee1@syzkaller.appspotmail.com
|
| 1.130 |
| 22-May-2022 |
riastradh | opencrypto: Assert session id is valid in crypto_freesession.
This gives us the opportunity to detect usage mistakes like
use-after-free.
Exception: Continue to silently ignore sid=0.
|
| 1.129 |
| 22-May-2022 |
riastradh | opencrypto: crypto_dispatch never fails now. Make it return void.
Same with crypto_kdispatch.
|
| 1.128 |
| 22-May-2022 |
riastradh | opencrypto: Assert driver process routine returns 0 or ERESTART.
No other errors are allowed -- other errors must be transmitted by
crypto_done. All drivers in tree (sun8i_crypto, glxsb, via_padlock,
mvcesa, mvxpsec, hifn, qat, ubsec, cryptosoft) have been audited for
this.
|
| 1.127 |
| 22-May-2022 |
riastradh | opencrypto: Rip out EAGAIN logic when unregistering crypto drivers.
I'm pretty sure this never worked reliably based on code inspection,
and it's unlikely to have ever been tested because it only applies
when unregistering a driver -- but we have no crypto drivers for
removable devices, so it would only apply if we went out of our way
to trigger detach with drvctl.
Instead, just make the operation fail with ENODEV, and remove all the
callback logic to resubmit the request on EAGAIN. (Maybe this should
be ENXIO, but crypto_kdispatch already does ENODEV.)
|
| 1.126 |
| 22-May-2022 |
riastradh | opencrypto: Assert nonnull callback up front in crypto_dispatch.
Same with crypto_kdispatch.
Convert some dead branches downstream to assertions too.
|
| 1.125 |
| 22-May-2022 |
riastradh | opencrypto: Make crypto_freesession return void.
No callers use the return value. It is not sensible to allow this to
fail.
|
| 1.124 |
| 22-May-2022 |
riastradh | opencrypto: Make freesession callback return void.
No functional change intended: all drivers already return zero
unconditionally.
|
| 1.123 |
| 22-May-2022 |
riastradh | opencrypto: Assert crp_desc and crp_buf are nonnull.
- crypto_getreq ensures crp_desc is nonnull.
- Caller is responsible for setting crp_buf.
|
| 1.122 |
| 22-May-2022 |
riastradh | opencrypto: Assert num>0 in crypto_getreq, num=1 in crypto_kgetreq.
- For crypto_getreq this makes downstream reasoning easier: on
success, crp_desc is guaranteed to be nonnull.
- For crypto_kgetreq, this was already assumed, just silently
ignored and not checked by anything.
|
| 1.121 |
| 22-May-2022 |
riastradh | opencrypto: Nix CRYPTO_F_DONE.
Nothing uses it any more.
|
| 1.120 |
| 22-May-2022 |
riastradh | opencrypto: Make sid=0 always invalid, but OK to free.
Previously, crypto_newsession could sometimes return 0 as the
driver-specific part of the session id, and 0 as the hid, for sid=0.
But netipsec assumes that it is always safe to free sid=0 from
zero-initialized memory even if crypto_newsession has never
succeeded. So it was up to every driver in tree to gracefully handle
sid=0, if it happened to get assigned hid=0. And, as long as the
freesession callback was expected to just return an error code when
given a bogus session id, that worked out fine...because nothing ever
used the error code.
That was a terrible fragile system that should never have been
invented. Instead, let's just ensure that valid session ids are
nonzero, and make crypto_freesession with sid=0 be a no-op.
|
| 1.119 |
| 19-May-2022 |
riastradh | opencrypto: Assert !cpu_intr_p() on dispatch and invoke.
These should only ever have been potentially called from hard
interrupt context by CRYPTO_F_CBIMM callbacks (CBIMM = call back
immediately). CRYPTO_F_CBIMM is no more, so there is no more need to
allow this case of call from hard interrupt context.
|
| 1.118 |
| 19-May-2022 |
riastradh | opencrypto: Nix CRYPTO_F_USER, CRYPTO_F_CBIMM, CRYPTO_F_CBIFSYNC.
CRYPTO_F_USER is no longer needed. It was introduced in 2008 by
darran@ in crypto.c 1.30, cryptodev.c 1.45 in an attempt to avoid
double-free between the issuing thread and asynchronous callback.
But the `fix' didn't work. In 2017, knakahara@ fixed it properly in
cryptodev.c 1.87 by distinguishing `the crypto operation has
completed' (CRYPTO_F_DONE) from `the callback is done touching the
crp object' (CRYPTO_F_DQRETQ, now renamed to CRYPTODEV_F_RET).
CRYPTO_F_CBIMM formerly served to invoke the callback synchronously
from the driver's interrupt completion routine, to reduce contention
on what was once a single cryptoret thread. Now, there is a per-CPU
queue and softint for much cheaper processing, so there is less
motivation for this in the first place. So let's remove the
complicated logic. This means the callbacks never run in hard
interrupt context, which means we don't need to worry about recursion
into crypto_dispatch in hard interrupt context.
|
| 1.117 |
| 17-May-2022 |
riastradh | opencrypto: Factor setting CRYPTO_F_DONE out of branches.
This had been done in 1.30 when the locking was different. No need
any more. No functional change intended.
|
| 1.116 |
| 14-Aug-2021 |
andvar | fix typo in CRK_ALGORITHM_MIN definition to match CRK_ALGORITHM_MAX one.
while here fix few typos in comments.
|
| 1.115 |
| 09-Aug-2021 |
andvar | fix typos in asymmetry, asymmetric(al), symmetrical.
|
| 1.114 |
| 08-Apr-2020 |
pgoyette | Revert previous change to use SYSCTL_SETUP since it breaks on macppc.
For some reason, the crypto module fails to link, and this results in
opencrypto sysctl failures.
Should resolve PR kern/55154
|
| 1.113 |
| 16-Mar-2020 |
pgoyette | Use the module subsystem's ability to process SYSCTL_SETUP() entries to
automate installation of sysctl nodes.
Note that there are still a number of device and pseudo-device modules
that create entries tied to individual device units, rather than to the
module itself. These are not changed.
|
| 1.112 |
| 01-Feb-2020 |
riastradh | softint_disestablish does xc_barrier(0) for us already.
|
| 1.111 |
| 01-Feb-2020 |
riastradh | Switch opencrypto to percpu_create.
Can't sleep for allocation in percpu_foreach.
|
| 1.110 |
| 06-Oct-2019 |
uwe | branches: 1.110.2;
xc_barrier - convenience function to xc_broadcast() a nop.
Make the intent more clear and also avoid a bunch of (xcfunc_t)nullop
casts that gcc 8 -Wcast-function-type is not happy about.
|
| 1.109 |
| 01-Oct-2019 |
chs | in many device attach paths, allocate memory with KM_SLEEP instead of KM_NOSLEEP
and remove code to handle failures that can no longer happen.
|
| 1.108 |
| 11-Jul-2019 |
christos | relinguish our lock while we are autoloading.
|
| 1.107 |
| 13-Jun-2019 |
christos | Try to load swcrypto if we we did not find any software drivers.
|
| 1.106 |
| 06-Jun-2018 |
maya | branches: 1.106.2;
Remove duplicate ;
|
| 1.105 |
| 08-Jan-2018 |
knakahara | branches: 1.105.2;
Fix PR kern/52910. Reported and implemented a patch by Sevan Janiyan, thanks.
|
| 1.104 |
| 08-Jan-2018 |
knakahara | Committed debugging logs by mistake, sorry. Revert cryoto.c:r.1.103 and ip6_flow.c:r.1.37.
|
| 1.103 |
| 08-Jan-2018 |
knakahara | Fix PR kern/52910. Reported and implemented a patch by Sevan Janiyan, thanks.
|
| 1.102 |
| 09-Nov-2017 |
christos | use PR_NOWAIT.
|
| 1.101 |
| 22-Sep-2017 |
knakahara | fix opencrypto(9) part of PR kern/52515
percpu data use pointers to TAILQ instead of TAILQ itself.
|
| 1.100 |
| 31-Jul-2017 |
knakahara | use pool_cache(9) instead of pool(9) as they can be called concurrently.
|
| 1.99 |
| 31-Jul-2017 |
knakahara | use kmem_alloc KPI instead of malloc KPI.
|
| 1.98 |
| 31-Jul-2017 |
knakahara | pack crypto_drivers variables to struct and add cacheline_aligned qualifier.
|
| 1.97 |
| 31-Jul-2017 |
knakahara | refactor: remove glue macros for FreeBSD code.
|
| 1.96 |
| 26-Jul-2017 |
knakahara | divide crp_ret_{,k}q by CPU to avoid reordering.
update locking note later.
|
| 1.95 |
| 26-Jul-2017 |
knakahara | make crp_{,k}q percpu to scale crypto_dispatch().
update locking note later.
|
| 1.94 |
| 20-Jul-2017 |
knakahara | fix typo
|
| 1.93 |
| 20-Jul-2017 |
knakahara | fix panic when using ecryption devices attached earlier than ipi_sysinit().
pointed out and tested by martin@n.o, thanks.
|
| 1.92 |
| 18-Jul-2017 |
knakahara | branches: 1.92.2;
make cryptoret() context softint to balance dequeuing crypto_ret_q with enqueuing it.
|
| 1.91 |
| 26-Jun-2017 |
knakahara | simplify mutex_enter/exit(crypto_q_mtx), and fix missing exit.
|
| 1.90 |
| 15-Jun-2017 |
knakahara | Reduce crypto_ret_q_mtx lock regions.
crypto.c does not access the members of crp when the crp is in crp_q or
crp_ret_q. Furthermore, crp_q and crp_ret_q are protected by each mutex,
so the members of crp is not shared. That means crp_flags is not required
mutex in crypto.c.
|
| 1.89 |
| 14-Jun-2017 |
knakahara | support multiple encryption drivers (port from FreeBSD).
|
| 1.88 |
| 14-Jun-2017 |
knakahara | refactor crypto_newsession() like FreeBSD.
|
| 1.87 |
| 14-Jun-2017 |
knakahara | must release cap->cc_lock before calling cap->cc_newsession() because of spinlock.
|
| 1.86 |
| 08-Jun-2017 |
christos | Put back crypto_checkdriver(); use it when we need to make sure that we
get back a cryptocap that has been initialized.
|
| 1.85 |
| 06-Jun-2017 |
christos | - acquire lock
- use c99 loop indexes
- initialize featp
|
| 1.84 |
| 06-Jun-2017 |
knakahara | apply the same fix as crypto.c:r1.83 for crypto_dispatch to crypto_kdispatch.
|
| 1.83 |
| 06-Jun-2017 |
knakahara | avoid crp_q reordering as hardware interrupts.
crypto_{,k}invoke() can be called with holding crp_q_mtx now.
|
| 1.82 |
| 06-Jun-2017 |
knakahara | restructure locks(2/2): crypto_q_mtx can be adaptive now.
|
| 1.81 |
| 06-Jun-2017 |
knakahara | restructure locks(1/2): make relation between lock and data explicit.
+ crypto_drv_mtx protects
- whole crypto_drivers
+ crypto_drivers[i].cc_lock (new) protects
- crypto_drivers[i] itself
- member of crypto_drivers[i]
+ crypto_q_mtx protects
- crp_q
- crp_kq
+ crypto_ret_q_mtx protects
- crp_ret_q
- crp_ret_kq
- crypto_exit_flag
I will add locking note later.
|
| 1.80 |
| 05-Jun-2017 |
knakahara | fix reading crp_q without holding crypto_q_mtx
|
| 1.79 |
| 05-Jun-2017 |
knakahara | use crypto_checkdriver_uninit() when it may touch uninitialized crypto_drivers.
|
| 1.78 |
| 31-May-2017 |
knakahara | branches: 1.78.2;
fix: crypto_unregister didn't work.
|
| 1.77 |
| 29-May-2017 |
knakahara | strictly use crypto_checkdriver(i) instead of using crypto_drivers[i] directly.
|
| 1.76 |
| 25-May-2017 |
knakahara | add cryptkop alloc/free KPI instead of manipulating cryptkop_pool directly.
|
| 1.75 |
| 24-May-2017 |
knakahara | make the default values of q_maxlen build parameters.
|
| 1.74 |
| 24-May-2017 |
knakahara | implement crypto_ret_q limitation. original code is implemented by hsuenaga@IIJ.
|
| 1.73 |
| 24-May-2017 |
knakahara | add crypto_ret_{,k}q length sysctl entries and statistics codes.
|
| 1.72 |
| 24-May-2017 |
knakahara | initialize sysctl in the same way regardless of module or not.
|
| 1.71 |
| 17-May-2017 |
knakahara | decrease the priority of batch crp even if there are more than one batch crp.
|
| 1.70 |
| 17-May-2017 |
knakahara | refactor cryptointr(), no functional changes.
|
| 1.69 |
| 17-May-2017 |
knakahara | fix cryptointr() can process unexpected request.
If migrate crp is linked after batch crp, "submit" is already set to
the batch crp. So, cryptointr() can process the batch crp instead of
the target migrate crp.
|
| 1.68 |
| 17-May-2017 |
knakahara | refactor crypto_kdispatch() in a similar way as crypto_dispatch().
|
| 1.67 |
| 17-May-2017 |
knakahara | refactor crypto_dispatch (3/3): do "blocked" operation previously
|
| 1.66 |
| 17-May-2017 |
knakahara | refactor crypto_dispatch (2/3): divide migrate operation
|
| 1.65 |
| 17-May-2017 |
knakahara | refactor crypto_dispatch (1/3): divide batch operation
Processing batch operation at first, crypto_q_mtx's lock region is reduced
because crp does not require crypto_q_mtx.
|
| 1.64 |
| 17-May-2017 |
knakahara | opencrypto: cleanup debug messages.
|
| 1.63 |
| 10-May-2017 |
knakahara | fix: crypto_drivers[hid].cc_process() could be called even if it was null.
If a crypto driver is unregistered before calling cyrptointr(),
the crypto_drivers[hid].cc_process is null in spite of the hid
is less than crypto_drivers_num.
reffered to FreeBSD code.
|
| 1.62 |
| 10-May-2017 |
knakahara | unify implementation of crypto_unregister() and crypto_unregister_all()
|
| 1.61 |
| 10-May-2017 |
knakahara | refactor crypto_unregister()
- separate logic to crypto_unregister_locked()
- refactor cryptocap cleanup condition
|
| 1.60 |
| 10-May-2017 |
knakahara | use macro instead of immediate value
|
| 1.59 |
| 02-May-2017 |
knakahara | add some assertion. tested by ATF net/ipsec/ and crypto/.
|
| 1.58 |
| 26-Apr-2017 |
knakahara | branches: 1.58.2;
When crypto request is deferred processing, opencrypto should not return error.
When the crypto device blocks a crypto request, opnecrypto enqueues the request,
that is, the request is just deferred and no error occurs.
The pseudo error causes problems, e.g. ipsec can send wrong ICMP host unreach.
contributed by hsuenaga@IIJ, thanks.
|
| 1.57 |
| 24-Apr-2017 |
knakahara | separate crypto_drv_mtx from crypto_mtx.
crypto_mtx is used only for cryptodev.c and ocryptodev.c now.
|
| 1.56 |
| 24-Apr-2017 |
knakahara | reduce crypto_q_mtx lock regions.
|
| 1.55 |
| 24-Apr-2017 |
knakahara | refactor crypto_unblock(). No functional change.
|
| 1.54 |
| 07-Apr-2017 |
knakahara | the processing said "ghastly hacks" is unnecessary now.
|
| 1.53 |
| 06-Apr-2017 |
knakahara | fix build failure ALL
|
| 1.52 |
| 05-Apr-2017 |
knakahara | fix processes accessing /dev/crypto stall when over three processes run with a hardware encryption driver
The process has stalled at cv_wait(&crp->crp_cv) because cryptodev_cb()
is not called as cryptoret() kthread keep waiting at cv_wait(&cryptoret_cv).
Previous opencrypto implementation assumes the thread from cryptodev.c
does all processing in the same context, so skips enqueueing and sending
cryptoret_cv. However, the context can be switched, e.g. when we use
a hardware encryption driver.
And add debug messages.
|
| 1.51 |
| 29-Mar-2017 |
knakahara | fix missing mutex_exit() in crypto_destroy().
crypto_destroy() is called only in error case or unloading module.
|
| 1.50 |
| 16-Mar-2017 |
knakahara | fix: remove unmatched mutex_exit/enter. must be forgetting to remove at crypto.c:r1.41
|
| 1.49 |
| 09-Feb-2017 |
knakahara | make some locks and condvar static
|
| 1.48 |
| 07-Jul-2016 |
msaitoh | branches: 1.48.2; 1.48.4;
KNF. Remove extra spaces. No functional change.
|
| 1.47 |
| 28-Nov-2015 |
christos | fix the build
|
| 1.46 |
| 28-Nov-2015 |
pgoyette | Re-work the module init and destroy code to allow it to be unloaded and
then reloaded.
Should fix PR kern/49842
|
| 1.45 |
| 25-Feb-2014 |
pooka | branches: 1.45.6;
Ensure that the top level sysctl nodes (kern, vfs, net, ...) exist before
the sysctl link sets are processed, and remove redundancy.
Shaves >13kB off of an amd64 GENERIC, not to mention >1k duplicate
lines of code.
|
| 1.44 |
| 14-Jan-2014 |
pgoyette | Clean-up module initialization
|
| 1.43 |
| 13-Jan-2014 |
pgoyette | When using modular opencrypto, make sure the sysctl variables get
created.
|
| 1.42 |
| 01-Jan-2014 |
pgoyette | Modularize the opencrypto components and link to the build
|
| 1.41 |
| 09-Jun-2011 |
drochner | branches: 1.41.2; 1.41.12; 1.41.16;
-if an opencrypto(9) session is allocated, the driver is refcounted
and can not disappear -- no need to hold crypto_mtx to check the
driver list
(the whole check is questionable)
-crp->crp_cv (the condition variable) is used by userland cryptodev
exclusively -- move its initialization there, no need to waste
cycles of in-kernel callers
-add a comment which members of "struct cryptop" are used
by opencrypto(9) and which by crypto(4)
(this should be split, no need to waste memory for in-kernel callers)
|
| 1.40 |
| 16-May-2011 |
drochner | branches: 1.40.2;
split the "crypto_mtx" spinlock into 3: one spinlock each for
the incoming and outgoing request queues (which can be dealt with
by hardware accelerators) and an adaptive lock for "all the rest"
(mostly driver configuration, but also some unrelated stuff in
cryptodev.c which should be revisited)
The latter one seems to be uneeded at many places, but for now I've
done simple replacements only, except minor fixes (where
softint_schedule() was called without the lock held)
|
| 1.39 |
| 06-May-2011 |
drochner | As a first step towards more fine-grained locking, don't require
crypto_{new.free}session() to be called with the "crypto_mtx"
spinlock held.
This doesn't change much for now because these functions acquire
the said mutex first on entry now, but at least it keeps the nasty
locks local to the opencrypto core.
|
| 1.38 |
| 24-Feb-2011 |
drochner | make the crypto softint MPSAFE -- I see no reason not to do it, and
it didn't cause trouble for me. (It doesn't give additional parallelization
as things look now, just saves the acquisition of KERNEL_LOCK.)
|
| 1.37 |
| 26-Jan-2011 |
christos | PR/44470: Dr. Wolfgang Stukenbrock: opencrypto kernel implementation may pass
outdated argument to worker
|
| 1.36 |
| 11-Aug-2010 |
pgoyette | branches: 1.36.2; 1.36.4;
Keep condvar wmesg within 8 char limit.
|
| 1.35 |
| 02-Aug-2010 |
jakllsch | Consistently use a single CRYPTO_SESID2HID-like macro.
Improve CRYPTO_DEBUG printing a bit:
print pointers with %p
print unsigned with %u rather than %d
use CRYPTO_SESID2LID instead of just casting to uint32_t
|
| 1.34 |
| 18-Apr-2009 |
tsutsui | branches: 1.34.2; 1.34.4;
Remove extra whitespace added by a stupid tool.
XXX: more in src/sys/arch
|
| 1.33 |
| 25-Mar-2009 |
darran | Fixes PR kern/41069 and PR kern/41070.
Extends the Opencrypto API to allow the destination buffer size to be
specified when its not the same size as the input buffer (i.e. for
operations like compress and decompress).
The crypto_op and crypt_n_op structures gain a u_int dst_len field.
The session_op structure gains a comp_alg field to specify a compression
algorithm.
Moved four ioctls to new ids; CIOCGSESSION, CIOCNGSESSION, CIOCCRYPT,
and CIOCNCRYPTM.
Added four backward compatible ioctls; OCIOCGSESSION, OCIOCNGSESSION,
OCIOCCRYPT, and OCIOCNCRYPTM.
Backward compatibility is maintained in ocryptodev.h and ocryptodev.c which
implement the original ioctls and set dst_len and comp_alg to 0.
Adds user-space access to compression features.
Adds software gzip support (CRYPTO_GZIP_COMP).
Adds the fast version of crc32 from zlib to libkern. This should be generally
useful and provide a place to start normalizing the various crc32 routines
in the kernel. The crc32 routine is used in this patch to support GZIP.
With input and support from tls@NetBSD.org.
|
| 1.32 |
| 18-Mar-2009 |
cegger | bcopy -> memcpy
|
| 1.31 |
| 18-Mar-2009 |
cegger | bzero -> memset
|
| 1.30 |
| 18-Nov-2008 |
darran | branches: 1.30.4;
Fix a race condition in opencrypto where the crypto request could be
completed by the crypto device, queued on the retq, but freed by the
ioctl lwp. The problem manifests as various panics relating to the
condvar inside the request. The problem can occur whenever the crypto
device completes the request immediately and the ioctl skips the cv_wait().
The problem can be reproduced by enabling cryptosoft and running an openssl
speed test. E.g.
sysctl -w kern.cryptodevallowsoft=-1
openssl speed -engine cryptodev -evp des-ede3-cbc -multi 64
Add a macro for TAILQ_FOREACH_REVERSE_SAFE() to queue.h, since this
was missing and the opencrypto code removes requests from a list while
iterating with TAILQ_FOREACH_REVERSE().
Add missing cv_destroy() calls for the key request cleanup.
Reviewed by Thor Lancelot Simon.
|
| 1.29 |
| 03-Aug-2008 |
degroote | branches: 1.29.2; 1.29.4;
In crypto_freereq, destroy explicitly the condvar
ok by tls@
|
| 1.28 |
| 28-Apr-2008 |
martin | branches: 1.28.2; 1.28.6;
Remove clause 3 and 4 from TNF licenses
|
| 1.27 |
| 10-Apr-2008 |
tls | branches: 1.27.2; 1.27.4;
Extend crypto.4 interface:
* Asynchronous operation with result retrieval via select/poll
* Mutliple-request submit/retrieve ioctls
* Mutliple-session create-destroy ioctls
Revise/rewrite crypto.4 manual page. It should now be much easier to write
new applications to this API.
Measured performance for trivial requests: 84,000 very short modular math
operations/sec, 120,000 very short md5 hashes per sec (with a hardware
accellerator of moderate performance but very low latency, whose driver
will be contributed at a later date).
Contributed to TNF by Coyote Point Systems, Inc.
|
| 1.26 |
| 05-Feb-2008 |
ad | branches: 1.26.6;
cryptoret: avoid a sleep/wakeup race.
|
| 1.25 |
| 05-Feb-2008 |
tls | The kthread is now MPSAFE.
|
| 1.24 |
| 04-Feb-2008 |
tls | Some locking fixes (double-release mutex in softintr wakeup case, which I
hadn't tested) and an uninitialized field in cse which Darran Hunt
found. Some more debugging printfs.
Turn on MPSAFE for the kthread. We're not sure it's safe for the softint
yet. Gives a little performance kick for swcrypto with many requests on
MP systems.
|
| 1.23 |
| 04-Feb-2008 |
tls | Rework opencrypto to use a spin mutex (crypto_mtx) instead of "splcrypto"
(actually splnet) and condvars instead of tsleep/wakeup. Fix a few
miscellaneous problems and add some debugging printfs while there.
Restore set of CRYPTO_F_DONE in crypto_done() which was lost at some
point after this code came from FreeBSD -- it made it impossible to wait
properly for a condition.
Add flags analogous to the "crp" flags to the key operation's krp struct.
Add a new flag, CRYPTO_F_ONRETQ which tells us a request finished before
the kthread had a chance to dequeue it and call its callback -- this was
letting requests stick on the queues before even though done and copied
out.
Callers of crypto_newsession() or crypto_freesession() must now take the
mutex. Change netipsec to do so. Dispatch takes the mutex itself as
needed.
This was tested fairly extensively with the cryptosoft backend and lightly
with a new hardware driver. It has not been tested with FAST_IPSEC; I am
unable to ascertain whether FAST_IPSEC currently works at all in our tree.
pjd@FreeBSD.ORG, ad@NetBSD.ORG, and darran@snark.us pointed me in the
right direction several times in the course of this. Remaining bugs
are mine alone.
|
| 1.22 |
| 01-Feb-2008 |
tls | This code never worked on a released version of FreeBSD in the form it's
been in in our tree, and certainly does not work on any version of FreeBSD
now. Run through unifdef -D__NetBSD__ -U__FreeBSD__ yielding a small
reduction of size and a dramatic improvement in readability.
No, this does not yield any meaningful decrease in patchability (unlike
mechanical changes that touch live source lines) -- try it and see.
|
| 1.21 |
| 08-Oct-2007 |
ad | branches: 1.21.4;
Use the softint API.
|
| 1.20 |
| 09-Jul-2007 |
ad | branches: 1.20.6; 1.20.8; 1.20.10;
Merge some of the less invasive changes from the vmlocking branch:
- kthread, callout, devsw API changes
- select()/poll() improvements
- miscellaneous MT safety improvements
|
| 1.19 |
| 12-Mar-2007 |
ad | branches: 1.19.2;
Pass an ipl argument to pool_init/POOL_INIT to be used when initializing
the pool's lock.
|
| 1.18 |
| 12-Jan-2007 |
daniel | branches: 1.18.2; 1.18.6;
crypto_init does not call crypto_init0 only once, because the marker
created with ONCE_DECL() is local. This results in reinitializing
the driver list when crypto_get_driverid() (and leaks memory). Fix
this by making the marker static.
Fixes PR/35412.
Ack freza@.
|
| 1.17 |
| 16-Nov-2006 |
christos | branches: 1.17.2;
__unused removal on arguments; approved by core.
|
| 1.16 |
| 20-Oct-2006 |
mrg | avoid yet another GCC uninitialised warning error that only comes
up with -O3.
|
| 1.15 |
| 12-Oct-2006 |
christos | - sprinkle __unused on function decls.
- fix a couple of unused bugs
- no more -Wno-unused for i386
|
| 1.14 |
| 07-Jun-2006 |
kardel | branches: 1.14.6; 1.14.8;
merge FreeBSD timecounters from branch simonb-timecounters
- struct timeval time is gone
time.tv_sec -> time_second
- struct timeval mono_time is gone
mono_time.tv_sec -> time_uptime
- access to time via
{get,}{micro,nano,bin}time()
get* versions are fast but less precise
- support NTP nanokernel implementation (NTP API 4)
- further reading:
Timecounter Paper: http://phk.freebsd.dk/pubs/timecounter.pdf
NTP Nanokernel: http://www.eecis.udel.edu/~mills/ntp/html/kern.html
|
| 1.13 |
| 06-Mar-2006 |
christos | branches: 1.13.6;
Add the 3 missing sysctl we are supposed to export.
|
| 1.12 |
| 16-Jan-2006 |
yamt | branches: 1.12.2; 1.12.4; 1.12.6;
- tweak RUN_ONCE api to allow init_func returns an error.
- physio: handle failure of workqueue_create.
|
| 1.11 |
| 25-Nov-2005 |
thorpej | branches: 1.11.2;
- De-couple the software crypto implementation from the rest of the
framework. There is no need to waste the space if you are only using
algoritms provided by hardware accelerators. To get the software
implementations, add "pseudo-device swcr" to your kernel config.
- Lazily initialize the opencrypto framework when crypto drivers
(either hardware or swcr) register themselves with the framework.
|
| 1.10 |
| 26-Feb-2005 |
perry | branches: 1.10.4; 1.10.10;
nuke trailing whitespace
|
| 1.9 |
| 29-Apr-2004 |
jonathan | branches: 1.9.2; 1.9.6; 1.9.8;
Change control knob for /dev/crypto to only allow requests which
will be hardware-accelerated. Avoids copyin()/copyout() overhead and
spending exceessive tie inside the kernel.
Pullup after: 24 hours, or confirmation by Jason Thorpe that this is the
consensus tech-kern agreed upon last summer.
|
| 1.8 |
| 31-Dec-2003 |
jonathan | branches: 1.8.2;
Split opencrypto configuration into an attribute, usable by inkernel
clients, and a pseudo-device for userspace access.
The attribute is named `opencrypto'. The pseudo-device is renamed to
"crypto", which has a dependency on "opencrypto". The sys/conf/majors
entry and pseudo-device attach entrypoint are updated to match the
new pseudo-device name.
Fast IPsec (sys/netipsec/files.ipsec) now lists a dependency on the
"opencrypto" attribute. Drivers for crypto accelerators (ubsec,
hifn775x) also pull in opencrypto, as providers of opencrypto transforms.
|
| 1.7 |
| 19-Nov-2003 |
jonathan | Fix typo.
|
| 1.6 |
| 19-Nov-2003 |
jonathan | Clean up userlevel access to software kernel transforms, in preparation
for using /dev/crypto for OpenSSL:
1. Add comments explaining crypto_devallowsoft, explaining the
OpenBSD-style three-way logic actully implemented in crypto_newsession().
2. Pass crypto_devallowsoft as the final argument to crypto_newsession(),
instead of a constant 0 value.
3. Set the default value of crypto_devallowsoft to 1, to allow
/dev/crypto access only for hardware-supported transforms.
Items 1-3 may be revised to match the FreeBSD two-way logic, if the
consensus is that there's no point to forcing software transforms.
But as a first step, let the description match what the code actually does.
GC unused variables usercrypto, userasmcrypto, cryptodevallowsoft from
cryptodev.c, in favour of variables crypto_usercrypto, crypto_userasmcrypto,
crypto_devallowsoft, which are used as well as defined in crypto.c.
|
| 1.5 |
| 09-Nov-2003 |
scw | Fix a genuine uninitialised variable.
|
| 1.4 |
| 21-Sep-2003 |
lha | Implement nanouptime as a function to avoid 'dereferencing type-punned pointer'.
No token after #endif
|
| 1.3 |
| 31-Jul-2003 |
jonathan | Remove bogus include which snuck into previous commit (from comparison
to Quentin Garnier's suggested patch).
|
| 1.2 |
| 30-Jul-2003 |
jonathan | Move the initialization of the crypto framework from the userland
pseudo-device to init_main(), so the framework is ready for
registration requests at autoconfiguration time.
Thanks to Quentin Garnier for confirming the change was required, and
for testing a similar fix.
|
| 1.1 |
| 25-Jul-2003 |
jonathan | Commit initial NetBSD port of the OpenCrypto Framework (OCF). This
code is derived from Sam Leffler's FreeBSD port of OCF, which is in
turn a port of Angelos Keromytis's OpenBSD work.
Credit to Sam and Angelos, any blame for the NetBSD port to me.
|
| 1.8.2.1 |
| 30-Apr-2004 |
jmc | Pullup rev 1.9 (requested by jonathan in ticket #234)
Change control knob for /dev/crypto to only allow requests which
will be hardware-accelerated. Avoids copyin()/copyout() overhead and
spending exceessive tie inside the kernel.
|
| 1.9.8.1 |
| 19-Mar-2005 |
yamt | sync with head. xen and whitespace. xen part is not finished.
|
| 1.9.6.1 |
| 29-Apr-2005 |
kent | sync with -current
|
| 1.9.2.6 |
| 11-Dec-2005 |
christos | Sync with head.
|
| 1.9.2.5 |
| 04-Mar-2005 |
skrll | Sync with HEAD.
Hi Perry!
|
| 1.9.2.4 |
| 21-Sep-2004 |
skrll | Fix the sync with head I botched.
|
| 1.9.2.3 |
| 18-Sep-2004 |
skrll | Sync with HEAD.
|
| 1.9.2.2 |
| 03-Aug-2004 |
skrll | Sync with HEAD
|
| 1.9.2.1 |
| 29-Apr-2004 |
skrll | file crypto.c was added on branch ktrace-lwp on 2004-08-03 10:56:25 +0000
|
| 1.10.10.1 |
| 29-Nov-2005 |
yamt | sync with head.
|
| 1.10.4.7 |
| 11-Feb-2008 |
yamt | sync with head.
|
| 1.10.4.6 |
| 04-Feb-2008 |
yamt | sync with head.
|
| 1.10.4.5 |
| 27-Oct-2007 |
yamt | sync with head.
|
| 1.10.4.4 |
| 03-Sep-2007 |
yamt | sync with head.
|
| 1.10.4.3 |
| 26-Feb-2007 |
yamt | sync with head.
|
| 1.10.4.2 |
| 30-Dec-2006 |
yamt | sync with head.
|
| 1.10.4.1 |
| 21-Jun-2006 |
yamt | sync with head.
|
| 1.11.2.1 |
| 01-Feb-2006 |
yamt | sync with head.
|
| 1.12.6.2 |
| 26-Jun-2006 |
yamt | sync with head.
|
| 1.12.6.1 |
| 13-Mar-2006 |
yamt | sync with head.
|
| 1.12.4.2 |
| 22-Apr-2006 |
simonb | Sync with head.
|
| 1.12.4.1 |
| 04-Feb-2006 |
simonb | We have nanouptime() now.
|
| 1.12.2.1 |
| 09-Sep-2006 |
rpaulo | sync with head
|
| 1.13.6.1 |
| 19-Jun-2006 |
chap | Sync with head.
|
| 1.14.8.2 |
| 10-Dec-2006 |
yamt | sync with head.
|
| 1.14.8.1 |
| 22-Oct-2006 |
yamt | sync with head
|
| 1.14.6.2 |
| 01-Feb-2007 |
ad | Sync with head.
|
| 1.14.6.1 |
| 18-Nov-2006 |
ad | Sync with head.
|
| 1.17.2.1 |
| 20-Jan-2007 |
bouyer | Pull up following revision(s) (requested by daniel in ticket #367):
sys/opencrypto/crypto.c: revision 1.18
Make the opencrypto framework work with more than one driver again.
Fixes PR/35412.
|
| 1.18.6.5 |
| 17-Jun-2007 |
ad | - Increase the number of thread priorities from 128 to 256. How the space
is set up is to be revisited.
- Implement soft interrupts as kernel threads. A generic implementation
is provided, with hooks for fast-path MD code that can run the interrupt
threads over the top of other threads executing in the kernel.
- Split vnode::v_flag into three fields, depending on how the flag is
locked (by the interlock, by the vnode lock, by the file system).
- Miscellaneous locking fixes and improvements.
|
| 1.18.6.4 |
| 13-May-2007 |
ad | - Pass the error number and residual count to biodone(), and let it handle
setting error indicators. Prepare to eliminate B_ERROR.
- Add a flag argument to brelse() to be set into the buf's flags, instead
of doing it directly. Typically used to set B_INVAL.
- Add a "struct cpu_info *" argument to kthread_create(), to be used to
create bound threads. Change "bool mpsafe" to "int flags".
- Allow exit of LWPs in the IDL state when (l != curlwp).
- More locking fixes & conversion to the new API.
|
| 1.18.6.3 |
| 10-Apr-2007 |
ad | Nuke the deferred kthread creation stuff, as it's no longer needed.
Pointed out by thorpej@.
|
| 1.18.6.2 |
| 09-Apr-2007 |
ad | - Add two new arguments to kthread_create1: pri_t pri, bool mpsafe.
- Fork kthreads off proc0 as new LWPs, not new processes.
|
| 1.18.6.1 |
| 13-Mar-2007 |
ad | Sync with head.
|
| 1.18.2.1 |
| 24-Mar-2007 |
yamt | sync with head.
|
| 1.19.2.1 |
| 11-Jul-2007 |
mjf | Sync with head.
|
| 1.20.10.1 |
| 14-Oct-2007 |
yamt | sync with head.
|
| 1.20.8.2 |
| 23-Mar-2008 |
matt | sync with HEAD
|
| 1.20.8.1 |
| 06-Nov-2007 |
matt | sync with HEAD
|
| 1.20.6.1 |
| 26-Oct-2007 |
joerg | Sync with HEAD.
Follow the merge of pmap.c on i386 and amd64 and move
pmap_init_tmp_pgtbl into arch/x86/x86/pmap.c. Modify the ACPI wakeup
code to restore CR4 before jumping back into kernel space as the large
page option might cover that.
|
| 1.21.4.1 |
| 18-Feb-2008 |
mjf | Sync with HEAD.
|
| 1.26.6.3 |
| 17-Jan-2009 |
mjf | Sync with HEAD.
|
| 1.26.6.2 |
| 28-Sep-2008 |
mjf | Sync with HEAD.
|
| 1.26.6.1 |
| 02-Jun-2008 |
mjf | Sync with HEAD.
|
| 1.27.4.4 |
| 09-Oct-2010 |
yamt | sync with head
|
| 1.27.4.3 |
| 11-Aug-2010 |
yamt | sync with head.
|
| 1.27.4.2 |
| 04-May-2009 |
yamt | sync with head.
|
| 1.27.4.1 |
| 16-May-2008 |
yamt | sync with head.
|
| 1.27.2.1 |
| 18-May-2008 |
yamt | sync with head.
|
| 1.28.6.2 |
| 13-Dec-2008 |
haad | Update haad-dm branch to haad-dm-base2.
|
| 1.28.6.1 |
| 19-Oct-2008 |
haad | Sync with HEAD.
|
| 1.28.2.1 |
| 18-Sep-2008 |
wrstuden | Sync with wrstuden-revivesa-base-2.
|
| 1.29.4.2 |
| 03-May-2009 |
snj | Pull up following revision(s) (requested by tls in ticket #611):
sys/lib/libkern/Makefile: patch
sys/lib/libkern/crc32.c: revision 1.1
sys/lib/libkern/crc32.h: revision 1.1
sys/lib/libkern/libkern.h: revision 1.89
sys/lib/libkern/arch/i386/Makefile.inc: revision 1.28
sys/net/zlib.h: revision 1.14 via patch
sys/opencrypto/crypto.c: revision 1.33
sys/opencrypto/cryptodev.c: revision 1.46
sys/opencrypto/cryptodev.h: revision 1.16
sys/opencrypto/cryptosoft.c: revision 1.24
sys/opencrypto/cryptosoft.h: revision 1.6
sys/opencrypto/deflate.h: revision 1.6
sys/opencrypto/cryptosoft_xform.c: revision 1.12
sys/opencrypto/deflate.c: revision 1.13
sys/opencrypto/files.opencrypto: revision 1.20
sys/opencrypto/ocryptodev.c: revision 1.1
sys/opencrypto/ocryptodev.h: revision 1.1
sys/opencrypto/xform.c: revision 1.18
sys/opencrypto/xform.h: revision 1.10
Fixes PR kern/41069 and PR kern/41070.
Extends the Opencrypto API to allow the destination buffer size to be
specified when its not the same size as the input buffer (i.e. for
operations like compress and decompress).
The crypto_op and crypt_n_op structures gain a u_int dst_len field.
The session_op structure gains a comp_alg field to specify a compression
algorithm.
Moved four ioctls to new ids; CIOCGSESSION, CIOCNGSESSION, CIOCCRYPT,
and CIOCNCRYPTM.
Added four backward compatible ioctls; OCIOCGSESSION, OCIOCNGSESSION,
OCIOCCRYPT, and OCIOCNCRYPTM.
Backward compatibility is maintained in ocryptodev.h and ocryptodev.c which
implement the original ioctls and set dst_len and comp_alg to 0.
Adds user-space access to compression features.
Adds software gzip support (CRYPTO_GZIP_COMP).
Adds the fast version of crc32 from zlib to libkern. This should be generally
useful and provide a place to start normalizing the various crc32 routines
in the kernel. The crc32 routine is used in this patch to support GZIP.
With input and support from tls@NetBSD.org.
|
| 1.29.4.1 |
| 20-Nov-2008 |
snj | branches: 1.29.4.1.4;
Pull up following revision(s) (requested by darran in ticket #92):
sys/opencrypto/cryptodev.c: revision 1.45
sys/sys/queue.h: revision 1.50
sys/opencrypto/cryptodev.h: revision 1.15
sys/opencrypto/crypto.c: revision 1.30
Fix a race condition in opencrypto where the crypto request could be
completed by the crypto device, queued on the retq, but freed by the
ioctl lwp. The problem manifests as various panics relating to the
condvar inside the request. The problem can occur whenever the crypto
device completes the request immediately and the ioctl skips the
cv_wait().
The problem can be reproduced by enabling cryptosoft and running an
openssl
speed test. E.g.
sysctl -w kern.cryptodevallowsoft=-1
openssl speed -engine cryptodev -evp des-ede3-cbc -multi 64
Add a macro for TAILQ_FOREACH_REVERSE_SAFE() to queue.h, since this
was missing and the opencrypto code removes requests from a list while
iterating with TAILQ_FOREACH_REVERSE().
Add missing cv_destroy() calls for the key request cleanup.
Reviewed by Thor Lancelot Simon.
|
| 1.29.4.1.4.1 |
| 20-May-2011 |
matt | bring matt-nb5-mips64 up to date with netbsd-5-1-RELEASE (except compat).
|
| 1.29.2.2 |
| 28-Apr-2009 |
skrll | Sync with HEAD.
|
| 1.29.2.1 |
| 19-Jan-2009 |
skrll | Sync with HEAD.
|
| 1.30.4.1 |
| 13-May-2009 |
jym | Sync with HEAD.
Commit is split, to avoid a "too many arguments" protocol error.
|
| 1.34.4.3 |
| 12-Jun-2011 |
rmind | sync with head
|
| 1.34.4.2 |
| 31-May-2011 |
rmind | sync with head
|
| 1.34.4.1 |
| 05-Mar-2011 |
rmind | sync with head
|
| 1.34.2.1 |
| 17-Aug-2010 |
uebayasi | Sync with HEAD.
|
| 1.36.4.2 |
| 05-Mar-2011 |
bouyer | Sync with HEAD
|
| 1.36.4.1 |
| 08-Feb-2011 |
bouyer | Sync with HEAD
|
| 1.36.2.1 |
| 06-Jun-2011 |
jruoho | Sync with HEAD.
|
| 1.40.2.1 |
| 23-Jun-2011 |
cherry | Catchup with rmind-uvmplock merge.
|
| 1.41.16.1 |
| 18-May-2014 |
rmind | sync with head
|
| 1.41.12.2 |
| 03-Dec-2017 |
jdolecek | update from HEAD
|
| 1.41.12.1 |
| 20-Aug-2014 |
tls | Rebase to HEAD as of a few days ago.
|
| 1.41.2.1 |
| 22-May-2014 |
yamt | sync with head.
for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.
this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
|
| 1.45.6.3 |
| 28-Aug-2017 |
skrll | Sync with HEAD
|
| 1.45.6.2 |
| 09-Jul-2016 |
skrll | Sync with HEAD
|
| 1.45.6.1 |
| 27-Dec-2015 |
skrll | Sync with HEAD (as of 26th Dec)
|
| 1.48.4.1 |
| 21-Apr-2017 |
bouyer | Sync with HEAD
|
| 1.48.2.2 |
| 26-Apr-2017 |
pgoyette | Sync with HEAD
|
| 1.48.2.1 |
| 20-Mar-2017 |
pgoyette | Sync with HEAD
|
| 1.58.2.2 |
| 19-May-2017 |
pgoyette | Resolve conflicts from previous merge (all resulting from $NetBSD
keywork expansion)
|
| 1.58.2.1 |
| 11-May-2017 |
pgoyette | Sync with HEAD
|
| 1.78.2.5 |
| 27-Feb-2018 |
martin | Pull up following revision(s) (requested by mrg in ticket #593):
sys/dev/marvell/mvxpsec.c: revision 1.2
sys/arch/m68k/m68k/pmap_motorola.c: revision 1.70
sys/opencrypto/crypto.c: revision 1.102
sys/arch/sparc64/sparc64/pmap.c: revision 1.308
sys/ufs/chfs/chfs_malloc.c: revision 1.5
sys/arch/powerpc/oea/pmap.c: revision 1.95
sys/sys/pool.h: revision 1.80,1.82
sys/kern/subr_pool.c: revision 1.209-1.216,1.219-1.220
sys/arch/alpha/alpha/pmap.c: revision 1.262
sys/kern/uipc_mbuf.c: revision 1.173
sys/uvm/uvm_fault.c: revision 1.202
sys/sys/mbuf.h: revision 1.172
sys/kern/subr_extent.c: revision 1.86
sys/arch/x86/x86/pmap.c: revision 1.266 (via patch)
sys/dev/dtv/dtv_scatter.c: revision 1.4
Allow only one pending call to a pool's backing allocator at a time.
Candidate fix for problems with hanging after kva fragmentation related
to PR kern/45718.
Proposed on tech-kern:
https://mail-index.NetBSD.org/tech-kern/2017/10/23/msg022472.html
Tested by bouyer@ on i386.
This makes one small change to the semantics of pool_prime and
pool_setlowat: they may fail with EWOULDBLOCK instead of ENOMEM, if
there is a pending call to the backing allocator in another thread but
we are not actually out of memory. That is unlikely because nearly
always these are used during initialization, when the pool is not in
use.
Define the new flag too for previous commit.
pool_grow can now fail even when sleeping is ok. Catch this case in pool_get
and retry.
Assert that pool_get failure happens only with PR_NOWAIT.
This would have caught the mistake I made last week leading to null
pointer dereferences all over the place, a mistake which I evidently
poorly scheduled alongside maxv's change to the panic message on x86
for null pointer dereferences.
Since pr_lock is now used to wait for two things now (PR_GROWING and
PR_WANTED) we need to loop for the condition we wanted.
make the KASSERTMSG/panic strings consistent as '%s: [%s], __func__, wchan'
Handle the ERESTART case from pool_grow()
don't pass 0 to the pool flags
Guess pool_cache_get(pc, 0) means PR_WAITOK here.
Earlier on in the same context we use kmem_alloc(sz, KM_SLEEP).
use PR_WAITOK everywhere.
use PR_NOWAIT.
Don't use 0 for PR_NOWAIT
use PR_NOWAIT instead of 0
panic ex nihilo -- PR_NOWAITing for zerot
Add assertions that either PR_WAITOK or PR_NOWAIT are set.
- fix an assert; we can reach there if we are nowait or limitfail.
- when priming the pool and failing with ERESTART, don't decrement the number
of pages; this avoids the issue of returning an ERESTART when we get to 0,
and is more correct.
- simplify the pool_grow code, and don't wakeup things if we ENOMEM.
In pmap_enter_ma(), only try to allocate pves if we might need them,
and even if that fails, only fail the operation if we later discover
that we really do need them. This implements the requirement that
pmap_enter(PMAP_CANFAIL) must not fail when replacing an existing
mapping with the first mapping of a new page, which is an unintended
consequence of the changes from the rmind-uvmplock branch in 2011.
The problem arises when pmap_enter(PMAP_CANFAIL) is used to replace an existing
pmap mapping with a mapping of a different page (eg. to resolve a copy-on-write).
If that fails and leaves the old pmap entry in place, then UVM won't hold
the right locks when it eventually retries. This entanglement of the UVM and
pmap locking was done in rmind-uvmplock in order to improve performance,
but it also means that the UVM state and pmap state need to be kept in sync
more than they did before. It would be possible to handle this in the UVM code
instead of in the pmap code, but these pmap changes improve the handling of
low memory situations in general, and handling this in UVM would be clunky,
so this seemed like the better way to go.
This somewhat indirectly fixes PR 52706, as well as the failing assertion
about "uvm_page_locked_p(old_pg)". (but only on x86, various other platforms
will need their own changes to handle this issue.)
In uvm_fault_upper_enter(), if pmap_enter(PMAP_CANFAIL) fails, assert that
the pmap did not leave around a now-stale pmap mapping for an old page.
If such a pmap mapping still existed after we unlocked the vm_map,
the UVM code would not know later that it would need to lock the
lower layer object while calling the pmap to remove or replace that
stale pmap mapping. See PR 52706 for further details.
hopefully workaround the irregularly "fork fails in init" problem.
if a pool is growing, and the grower is PR_NOWAIT, mark this.
if another caller wants to grow the pool and is also PR_NOWAIT,
busy-wait for the original caller, which should either succeed
or hard-fail fairly quickly.
implement the busy-wait by unlocking and relocking this pools
mutex and returning ERESTART. other methods (such as having
the caller do this) were significantly more code and this hack
is fairly localised.
ok chs@ riastradh@
Don't release the lock in the PR_NOWAIT allocation. Move flags setting
after the acquiring the mutex. (from Tobias Nygren)
apply the change from arch/x86/x86/pmap.c rev. 1.266 commitid vZRjvmxG7YTHLOfA:
In pmap_enter_ma(), only try to allocate pves if we might need them,
and even if that fails, only fail the operation if we later discover
that we really do need them. If we are replacing an existing mapping,
reuse the pv structure where possible.
This implements the requirement that pmap_enter(PMAP_CANFAIL) must not fail
when replacing an existing mapping with the first mapping of a new page,
which is an unintended consequence of the changes from the rmind-uvmplock
branch in 2011.
The problem arises when pmap_enter(PMAP_CANFAIL) is used to replace an existing
pmap mapping with a mapping of a different page (eg. to resolve a copy-on-write).
If that fails and leaves the old pmap entry in place, then UVM won't hold
the right locks when it eventually retries. This entanglement of the UVM and
pmap locking was done in rmind-uvmplock in order to improve performance,
but it also means that the UVM state and pmap state need to be kept in sync
more than they did before. It would be possible to handle this in the UVM code
instead of in the pmap code, but these pmap changes improve the handling of
low memory situations in general, and handling this in UVM would be clunky,
so this seemed like the better way to go.
This somewhat indirectly fixes PR 52706 on the remaining platforms where
this problem existed.
|
| 1.78.2.4 |
| 02-Jan-2018 |
snj | Pull up following revision(s) (requested by knakahara in ticket #460):
sys/opencrypto/crypto.c: revision 1.101
fix opencrypto(9) part of PR kern/52515
percpu data use pointers to TAILQ instead of TAILQ itself.
|
| 1.78.2.3 |
| 05-Aug-2017 |
snj | Pull up following revision(s) (requested by knakahara in ticket #178):
sys/opencrypto/crypto.c: 1.92-1.100
sys/opencrypto/cryptodev.h: 1.38-1.39
sys/opencrypto/ocryptodev.c: 1.9-1.11
sys/opencrypto/xform.c: revision 1.29
sys/opencrypto/xform.h: revision 1.20
KNF
--
Apply C99-style struct initialization to enc_xform, auth_hash and comp_algo
--
make cryptoret() context softint to balance dequeuing crypto_ret_q with enqueuing it.
--
fix panic when using ecryption devices attached earlier than ipi_sysinit().
pointed out and tested by martin@n.o, thanks.
--
fix typo
--
make crp_{,k}q percpu to scale crypto_dispatch().
update locking note later.
--
divide crp_ret_{,k}q by CPU to avoid reordering.
update locking note later.
--
update locking notes of opencrypto(9)
--
Don't disclose uninitialized 32-bit word if cryptodev_session fails.
From Ilja Van Sprundel.
--
Avert userland-controlled integer overflow.
From Ilja Van Sprundel.
--
Avoid another userland-controlled integer overflow.
From Ilja Van Sprundel.
--
refactor: remove glue macros for FreeBSD code.
--
pack crypto_drivers variables to struct and add cacheline_aligned qualifier.
--
use kmem_alloc KPI instead of malloc KPI.
--
use pool_cache(9) instead of pool(9) as they can be called concurrently.
|
| 1.78.2.2 |
| 05-Jul-2017 |
snj | Pull up following revision(s) (requested by knakahara in ticket #97):
sys/opencrypto/crypto.c: 1.87-1.91
sys/opencrypto/cryptodev.c: 1.93-1.95
sys/opencrypto/cryptodev.h: 1.37
sys/opencrypto/cryptosoft.c: 1.52
sys/rump/dev/lib/libopencrypto/opencrypto_component.c: 1.5
sanitize count used for kmem_alloc size.
Hmm, who uses CIOCNGSESSION, CIOCNFSESSION, CIOCNCRYPTM or CIOCNFKEYM?
--
sanitize in CIOCNCRYPTM and initialize comp_alg in CIOCNGSESSION
--
must release cap->cc_lock before calling cap->cc_newsession() because of spinlock.
--
refactor crypto_newsession() like FreeBSD.
--
support multiple encryption drivers (port from FreeBSD).
--
Divide crp_devflags from crp_flags to write exclusively.
CRYPTO_F_DQRETQ(new name is CRYPTODEV_F_RET) is used by cryptodev.c only.
It should be divided to other member.
--
Reduce crypto_ret_q_mtx lock regions.
crypto.c does not access the members of crp when the crp is in crp_q or
crp_ret_q. Furthermore, crp_q and crp_ret_q are protected by each mutex,
so the members of crp is not shared. That means crp_flags is not required
mutex in crypto.c.
--
fix cryptosoft.c:r1.51 mistake. swcrypto_attach() must not be called from module_init_class().
swcrypto_attach() will call softint_establish(), it must be called after cpus
attached. module_init_class() is too early to call softint_establish().
--
simplify mutex_enter/exit(crypto_q_mtx), and fix missing exit.
--
reduce rump waring message. pointed out by ozaki-r@n.o, thanks.
|
| 1.78.2.1 |
| 22-Jun-2017 |
snj | Pull up following revision(s) (requested by christos/knakahara in ticket #37):
sys/opencrypto/crypto.c: 1.79-1.86
sys/opencrypto/cryptodev.h: 1.35, 1.36
use crypto_checkdriver_uninit() when it may touch uninitialized crypto_drivers.
--
fix reading crp_q without holding crypto_q_mtx
--
restructure locks(1/2): make relation between lock and data explicit.
+ crypto_drv_mtx protects
- whole crypto_drivers
+ crypto_drivers[i].cc_lock (new) protects
- crypto_drivers[i] itself
- member of crypto_drivers[i]
+ crypto_q_mtx protects
- crp_q
- crp_kq
+ crypto_ret_q_mtx protects
- crp_ret_q
- crp_ret_kq
- crypto_exit_flag
I will add locking note later.
--
restructure locks(2/2): crypto_q_mtx can be adaptive now.
--
add locking notes.
--
avoid crp_q reordering as hardware interrupts.
crypto_{,k}invoke() can be called with holding crp_q_mtx now.
--
apply the same fix as crypto.c:r1.83 for crypto_dispatch to crypto_kdispatch.
--
- acquire lock
- use c99 loop indexes
- initialize featp
--
Put back crypto_checkdriver(); use it when we need to make sure that we
get back a cryptocap that has been initialized.
|
| 1.92.2.2 |
| 18-Jul-2017 |
knakahara | 3212654
|
| 1.92.2.1 |
| 18-Jul-2017 |
knakahara | file crypto.c was added on branch perseant-stdc-iso10646 on 2017-07-18 06:01:37 +0000
|
| 1.105.2.1 |
| 25-Jun-2018 |
pgoyette | Sync with HEAD
|
| 1.106.2.1 |
| 13-Apr-2020 |
martin | Mostly merge changes from HEAD upto 20200411
|
| 1.110.2.1 |
| 29-Feb-2020 |
ad | Sync with head.
|
