Home | History | Annotate | Download | only in kern
History log of /src/sys/kern/kern_kthread.c
RevisionDateAuthorComments
 1.49  23-Sep-2023  ad kernel_lock isn't needed to synchronise kthread_exit() and kthread_join().
 1.48  17-Jul-2023  riastradh kthread(9): Fix nested kthread_join.

No reason for one kthread_join to interfere with another, or to cause
non-cyclic dependencies to get stuck.

Uses struct lwp::l_private for this purpose, which for user threads
stores the tls pointer. I don't think anything in kthread(9) uses
l_private -- generally kernel threads will use lwp specificdata. But
maybe this should use a new member instead, or a union member with an
existing pointer for the purpose.
 1.47  13-Sep-2022  riastradh kthread(9): Attribute uarea to caller for kmsan.
 1.46  01-Aug-2020  riastradh New functions kthread_fpu_enter/exit.

The MI definitions don't do anything but maintain a flag, but MD code
can define kthread_fpu_enter/exit_md to actually enable/disable the
FPU. (These are almost pcu_load/discard on systems that use pcu(9),
except they apply to all PCUs.)

Discussed on tech-kern:
https://mail-index.netbsd.org/tech-kern/2020/06/20/msg026524.html

The proposed kthread flag KTHREAD_FPU is not included because I
couldn't find any particular need for it that would not be covered by
just calling kthread_fpu_enter/exit in the kthread function.
 1.45  08-Jan-2020  ad Hopefully fix some problems seen with MP support on non-x86, in particular
where curcpu() is defined as curlwp->l_cpu:

- mi_switch(): undo the ~2007ish optimisation to unlock curlwp before
calling cpu_switchto(). It's not safe to let other actors mess with the
LWP (in particular l->l_cpu) while it's still context switching. This
removes l->l_ctxswtch.

- Move the LP_RUNNING flag into l->l_flag and rename to LW_RUNNING since
it's now covered by the LWP's lock.

- Ditch lwp_exit_switchaway() and just call mi_switch() instead. Everything
is in cache anyway so it wasn't buying much by trying to avoid saving old
state. This means cpu_switchto() will never be called with prevlwp ==
NULL.

- Remove some KERNEL_LOCK handling which hasn't been needed for years.
 1.44  23-Nov-2019  ad branches: 1.44.2;
Minor scheduler cleanup:

- Adapt to cpu_need_resched() changes. Avoid lost & duplicate IPIs and ASTs.
sched_resched_cpu() and sched_resched_lwp() contain the logic for this.
- Changes for LSIDL to make the locking scheme match the intended design.
- Reduce lock contention and false sharing further.
- Numerous small bugfixes, including some corrections for SCHED_FIFO/RT.
- Use setrunnable() in more places, and merge cut & pasted code.
 1.43  09-Jan-2018  pgoyette branches: 1.43.4;
KASSERT() that kthread_join()'s target is expecting to be joined.

As discussed on IRC.
 1.42  21-Apr-2017  christos - Propagate the signal mask from the ucontext_t to the newly created thread
as specified by _lwp_create(2)
- Reset the signal stack for threads created with _lwp_create(2)
 1.41  21-Apr-2015  pooka branches: 1.41.2; 1.41.4;
Remove the unused and all-but-unusable kthread_destroy(). Also makes
it more obvious that the opposite of kthread_create() is kthread_exit().
 1.40  21-Apr-2015  pooka Don't check if constant-sized KM_SLEEP allocations succeeded.
 1.39  01-Sep-2012  matt branches: 1.39.2; 1.39.16;
Add a __HAVE_CPU_UAREA_IDLELWP hook so that the MD code can allocate
special UAREAs for idle lwp's.
 1.38  01-Nov-2011  jym branches: 1.38.2; 1.38.10;
Avoid crashing when we do not explictly pass down a CPU when creating
a kthread(9) in the LSIDL (KTHREAD_IDLE) state.
 1.37  29-Oct-2011  jym Typo fix.
 1.36  07-Aug-2011  rmind Rename slightly misleading KTHREAD_JOINABLE to KTHREAD_MUSTJOIN.
 1.35  17-Jul-2011  joerg Retire varargs.h support. Move machine/stdarg.h logic into MI
sys/stdarg.h and expect compiler to provide proper builtins, defaulting
to the GCC interface. lint still has a special fallback.
Reduce abuse of _BSD_VA_LIST_ by defining __va_list by default and
derive va_list as required by standards.
 1.34  24-May-2011  joerg Use a real panic as safe guard
 1.33  19-May-2011  rmind Re-implement kthread_join(9), so that it actually works (hi haad@).
 1.32  26-Apr-2011  ahoka Hold the current lwp's mutex, instead of the lwp we want to wait for
in kthread_join(). This fixes panics with DIAGNOSTIC and possibly some
rare race conditons.
 1.31  17-Feb-2011  matt Add support for cpu-specific uarea allocation routines. Allows different
allocation for user and system lwps. MIPS will use this to map uareas of
system lwp used direct-mapped addresses (to reduce the overhead of
switching to kernel threads). ibm4xx could use to map uareas via direct
mapped addresses and avoid the problem of having the kernel stack not in
the TLB.
 1.30  13-Jun-2010  yamt branches: 1.30.2; 1.30.4;
increment p_nrlwps in lwp_create rather than letting callers do so
as it's always decremented by lwp_exit. this fixes error recovery of
eg. aio_procinit.
 1.29  12-May-2010  haad Fix bogus KASSERT, typo and KNFize my previous patch.
 1.28  11-May-2010  haad Add support for kthread_join in our kernel thread implementation. This is used
by zfs but I think that it can be generaly usefull. Thread need to be created
with KTHREAD_JOINABLE flag and can be joined only once.
When joinable thread was created it will not e automatically reaped from system
and kthread_join must be called on it to reap it.

Ok by ad@.
 1.27  21-Oct-2009  rmind branches: 1.27.2; 1.27.4;
Remove uarea swap-out functionality:

- Addresses the issue described in PR/38828.
- Some simplification in threading and sleepq subsystems.
- Eliminates pmap_collect() and, as a side note, allows pmap optimisations.
- Eliminates XS_CTL_DATA_ONSTACK in scsipi code.
- Avoids few scans on LWP list and thus potentially long holds of proc_lock.
- Cuts ~1.5k lines of code. Reduces amd64 kernel size by ~4k.
- Removes __SWAP_BROKEN cases.

Tested on x86, mips, acorn32 (thanks <mpumford>) and partly tested on
acorn26 (thanks to <bjh21>).

Discussed on <tech-kern>, reviewed by <ad>.
 1.26  30-Jan-2009  agc Fix a tyop in the previous commit
 1.25  29-Jan-2009  ad - Allow creating timeshard kthreads. To be used to fix the RAIDframe
parity rewrite issue.
- Create kthreads in the SCHED_RR class by default, not SCHED_FIFO.
 1.24  28-Apr-2008  martin branches: 1.24.8; 1.24.10;
Remove clause 3 and 4 from TNF licenses
 1.23  24-Apr-2008  ad branches: 1.23.2;
Merge proc::p_mutex and proc::p_smutex into a single adaptive mutex, since
we no longer need to guard against access from hardware interrupt handlers.

Additionally, if cloning a process with CLONE_SIGHAND, arrange to have the
child process share the parent's lock so that signal state may be kept in
sync. Partially addresses PR kern/37437.
 1.22  12-Apr-2008  ad branches: 1.22.2;
Move the LW_BOUND flag into the thread-private flag word. It can be tested
by other threads/CPUs but that is only done when the LWP is known to be in a
quiescent state (for example, on a run queue).
 1.21  14-Feb-2008  ad branches: 1.21.6;
Make schedstate_percpu::spc_lwplock an exernally allocated item. Remove
the hacks in sparc/cpu.c to reinitialize it. This should be in its own
cache line but that's another change.
 1.20  02-Jan-2008  ad Merge vmlocking2 to head.
 1.19  06-Nov-2007  ad branches: 1.19.2; 1.19.6;
Merge scheduler changes from the vmlocking branch. All discussed on
tech-kern:

- Invert priority space so that zero is the lowest priority. Rearrange
number and type of priority levels into bands. Add new bands like
'kernel real time'.
- Ignore the priority level passed to tsleep. Compute priority for
sleep dynamically.
- For SCHED_4BSD, make priority adjustment per-LWP, not per-process.
 1.18  10-Jul-2007  ad branches: 1.18.6; 1.18.8; 1.18.12; 1.18.14;
kthread_exit: l->l_name could be NULL.
 1.17  09-Jul-2007  ad Merge some of the less invasive changes from the vmlocking branch:

- kthread, callout, devsw API changes
- select()/poll() improvements
- miscellaneous MT safety improvements
 1.16  09-Feb-2007  ad branches: 1.16.6; 1.16.8;
Merge newlock2 to head.
 1.15  18-Jan-2003  thorpej branches: 1.15.18; 1.15.48;
Merge the nathanw_sa branch.
 1.14  17-Nov-2002  chs change uvm_uarea_alloc() to indicate whether the returned uarea is already
backed by physical pages (ie. because it reused a previously-freed one),
so that we can skip a bunch of useless work in that case.
this fixes the underlying problem behind PR 18543, and also speeds up fork()
quite a bit (eg. 7% on my pc, 1% on my ultra2) when we get a cache hit.
 1.13  01-Jun-2002  lukem SIMPLEQ rototill:
- implement SIMPLEQ_REMOVE(head, elm, type, field). whilst it's O(n),
this mirrors the functionality of SLIST_REMOVE() (the other
singly-linked list type) and FreeBSD's STAILQ_REMOVE()
- remove the unnecessary elm arg from SIMPLEQ_REMOVE_HEAD().
this mirrors the functionality of SLIST_REMOVE_HEAD() (the other
singly-linked list type) and FreeBSD's STAILQ_REMOVE_HEAD()
- remove notes about SIMPLEQ not supporting arbitrary element removal
- use SIMPLEQ_FOREACH() instead of home-grown for loops
- use SIMPLEQ_EMPTY() appropriately
- use SIMPLEQ_*() instead of accessing sqh_first,sqh_last,sqe_next directly
- reorder manual page; be consistent about how the types are listed
- other minor cleanups
 1.12  12-Nov-2001  lukem branches: 1.12.8;
add RCSIDs
 1.11  14-Jul-2000  thorpej branches: 1.11.2; 1.11.4; 1.11.8;
ANSI'ify.
 1.10  28-May-2000  thorpej Rather than starting init and creating kthreads by forking and then
doing a cpu_set_kpc(), just pass the entry point and argument all
the way down the fork path starting with fork1(). In order to
avoid special-casing the normal fork in every cpu_fork(), MI code
passes down child_return() and the child process pointer explicitly.

This fixes a race condition on multiprocessor systems; a CPU could
grab the newly created processes (which has been placed on a run queue)
before cpu_set_kpc() would be performed.
 1.9  08-May-2000  thorpej branches: 1.9.2;
__predict_false() fork1() failing.
 1.8  06-Jul-1999  thorpej branches: 1.8.2;
Make the kthread API a bit more friendly to loadable kernel modules.
 1.7  13-May-1999  thorpej Allow the caller to specify a stack for the child process. If NULL,
the child inherits the stack pointer from the parent (traditional
behavior). Like the signal stack, the stack area is secified as
a low address and a size; machine-dependent code accounts for stack
direction.

This is required for clone(2).
 1.6  13-May-1999  thorpej Allow an alternate exit signal (i.e. not SIGCHLD) to be delivered to the
parent, specified at fork time. Specify a new flag to wait4(2), WALTSIG,
to wait for processes which use an alternate exit signal.

This is required for clone(2).
 1.5  30-Apr-1999  thorpej Share everything with proc0.
 1.4  12-Apr-1999  gwr minor nits -- replace vsprintf with vsnprinf
 1.3  22-Dec-1998  kleink branches: 1.3.4;
#ifdef __STDC__ -> #if __STDC__
 1.2  14-Nov-1998  thorpej Implement a way to queue kernel threads for creation after init,
pagedaemon, reaper, etc. Caller provides a callback function and
argument which will be called to create the threads.
 1.1  11-Nov-1998  thorpej Move fork_kthread() to a new file, kern_kthread.c, and rename it to
kthread_create(). Implement kthread_exit() (causes a thrad to exit).
Set P_NOCLDWAIT on kernel threads, which will cause any of their children
to be reparented to init(8) (which is already prepared to wait out orphaned
processes).
 1.3.4.2  02-Aug-1999  thorpej Update from trunk.
 1.3.4.1  21-Jun-1999  thorpej Sync w/ -current.
 1.8.2.1  20-Nov-2000  bouyer Update thorpej_scsipi to -current as of a month ago
 1.9.2.1  22-Jun-2000  minoura Sync w/ netbsd-1-5-base.
 1.11.8.1  12-Nov-2001  thorpej Sync the thorpej-mips-cache branch with -current.
 1.11.4.2  23-Jun-2002  jdolecek catch up with -current on kqueue branch
 1.11.4.1  10-Jan-2002  thorpej Sync kqueue branch with -current.
 1.11.2.7  11-Dec-2002  thorpej Sync with HEAD.
 1.11.2.6  17-Jul-2002  nathanw Whitespace.
 1.11.2.5  12-Jul-2002  nathanw No longer need to pull in lwp.h; proc.h pulls it in for us.
 1.11.2.4  24-Jun-2002  nathanw Curproc->curlwp renaming.

Change uses of "curproc->l_proc" back to "curproc", which is more like the
original use. Bare uses of "curproc" are now "curlwp".

"curproc" is now #defined in proc.h as ((curlwp) ? (curlwp)->l_proc) : NULL)
so that it is always safe to reference curproc (*de*referencing curproc
is another story, but that's always been true).
 1.11.2.3  20-Jun-2002  nathanw Catch up to -current.
 1.11.2.2  14-Nov-2001  nathanw Catch up to -current.
 1.11.2.1  05-Mar-2001  nathanw Initial commit of scheduler activations and lightweight process support.
 1.12.8.1  20-Jun-2002  gehenna catch up with -current.
 1.15.48.1  20-Oct-2006  ad - Pass FORK_SYSTEM to fork() and have it set up kthreads correctly.
- Acquire p->p_smutex where appropriate.
 1.15.18.5  27-Feb-2008  yamt sync with head.
 1.15.18.4  21-Jan-2008  yamt sync with head
 1.15.18.3  15-Nov-2007  yamt sync with head.
 1.15.18.2  03-Sep-2007  yamt sync with head.
 1.15.18.1  26-Feb-2007  yamt sync with head.
 1.16.8.1  11-Jul-2007  mjf Sync with head.
 1.16.6.14  05-Nov-2007  ad Expand the LWP priority space again to include 32 levels for kthreads,
so that they always run before user processes.
 1.16.6.13  01-Nov-2007  ad - Fix interactivity problems under high load. Beacuse soft interrupts
are being stacked on top of regular LWPs, more often than not aston()
was being called on a soft interrupt thread instead of a user thread,
meaning that preemption was not happening on EOI.

- Don't use bool in a couple of data structures. Sub-word writes are not
always atomic and may clobber other fields in the containing word.

- For SCHED_4BSD, make p_estcpu per thread (l_estcpu). Rework how the
dynamic priority level is calculated - it's much better behaved now.

- Kill the l_usrpri/l_priority split now that priorities are no longer
directly assigned by tsleep(). There are three fields describing LWP
priority:

l_priority: Dynamic priority calculated by the scheduler.
This does not change for kernel/realtime threads,
and always stays within the correct band. Eg for
timeshared LWPs it never moves out of the user
priority range. This is basically what l_usrpri
was before.

l_inheritedprio: Lent to the LWP due to priority inheritance
(turnstiles).

l_kpriority: A boolean value set true the first time an LWP
sleeps within the kernel. This indicates that the LWP
should get a priority boost as compensation for blocking.
lwp_eprio() now does the equivalent of sched_kpri() if
the flag is set. The flag is cleared in userret().

- Keep track of scheduling class (OTHER, FIFO, RR) in struct lwp, and use
this to make decisions in a few places where we previously tested for a
kernel thread.

- Partially fix itimers and usr/sys/intr time accounting in the presence
of software interrupts.

- Use kthread_create() to create idle LWPs. Move priority definitions
from the various modules into sys/param.h.

- newlwp -> lwp_create
 1.16.6.12  23-Oct-2007  yamt kthread_create: don't hold locks when manipulating local variables.
 1.16.6.11  18-Oct-2007  ad Free uareas back to the uarea cache on the CPU where they were last used.
 1.16.6.10  18-Oct-2007  ad Update for soft interrupt changes. See kern_softint.c 1.1.2.17 for details.
 1.16.6.9  23-Aug-2007  ad - Remove most calls to get kernel_lock during fault handling.
- By default start new LWPs without kernel_lock. Those returning to
userspace don't need it.
 1.16.6.8  15-Jul-2007  ad Sync with head.
 1.16.6.7  01-Jul-2007  ad Make kthread_create() work with schedulers that use per-CPU locks.
Pointed out by rmind@.
 1.16.6.6  17-Jun-2007  ad - Increase the number of thread priorities from 128 to 256. How the space
is set up is to be revisited.
- Implement soft interrupts as kernel threads. A generic implementation
is provided, with hooks for fast-path MD code that can run the interrupt
threads over the top of other threads executing in the kernel.
- Split vnode::v_flag into three fields, depending on how the flag is
locked (by the interlock, by the vnode lock, by the file system).
- Miscellaneous locking fixes and improvements.
 1.16.6.5  08-Jun-2007  ad Sync with head.
 1.16.6.4  13-May-2007  ad - Pass the error number and residual count to biodone(), and let it handle
setting error indicators. Prepare to eliminate B_ERROR.
- Add a flag argument to brelse() to be set into the buf's flags, instead
of doing it directly. Typically used to set B_INVAL.
- Add a "struct cpu_info *" argument to kthread_create(), to be used to
create bound threads. Change "bool mpsafe" to "int flags".
- Allow exit of LWPs in the IDL state when (l != curlwp).
- More locking fixes & conversion to the new API.
 1.16.6.3  10-Apr-2007  ad - Free the uarea if newlwp fails.
- Cosmetic changes.
 1.16.6.2  10-Apr-2007  ad Nuke the deferred kthread creation stuff, as it's no longer needed.
Pointed out by thorpej@.
 1.16.6.1  09-Apr-2007  ad - Add two new arguments to kthread_create1: pri_t pri, bool mpsafe.
- Fork kthreads off proc0 as new LWPs, not new processes.
 1.18.14.2  18-Feb-2008  mjf Sync with HEAD.
 1.18.14.1  19-Nov-2007  mjf Sync with HEAD.
 1.18.12.1  13-Nov-2007  bouyer Sync with HEAD
 1.18.8.3  23-Mar-2008  matt sync with HEAD
 1.18.8.2  09-Jan-2008  matt sync with HEAD
 1.18.8.1  06-Nov-2007  matt sync with HEAD
 1.18.6.1  06-Nov-2007  joerg Sync with HEAD.
 1.19.6.1  02-Jan-2008  bouyer Sync with HEAD
 1.19.2.1  04-Dec-2007  ad Pull the vmlocking changes into a new branch.
 1.21.6.1  02-Jun-2008  mjf Sync with HEAD.
 1.22.2.1  18-May-2008  yamt sync with head.
 1.23.2.4  11-Aug-2010  yamt sync with head.
 1.23.2.3  11-Mar-2010  yamt sync with head
 1.23.2.2  04-May-2009  yamt sync with head.
 1.23.2.1  16-May-2008  yamt sync with head.
 1.24.10.3  09-Aug-2011  riz Pull up following revision(s) (requested by matt in ticket #1652):
sys/sys/kthread.h: revision 1.7.10.1.8.1
sys/kern/kern_kthread.c: revision 1.24.10.2.8.1
Pullup <a rel="nofollow" href="http://mail-index.netbsd.org/source-changes/2010/05/11/msg009810.html">http://mail-index.netbsd.org/source-changes/2010/05/11/msg009810.html</a>
to this branch because the new flash framework will need it.
 1.24.10.2  02-Feb-2009  snj branches: 1.24.10.2.4; 1.24.10.2.8;
Pull up following revision(s) (requested by ad in ticket #397):
sys/kern/kern_kthread.c: revision 1.26
Fix a tyop in the previous commit
 1.24.10.1  02-Feb-2009  snj Pull up following revision(s) (requested by ad in ticket #397):
share/man/man9/kthread.9: revision 1.18
sys/kern/kern_kthread.c: revision 1.25
sys/sys/kthread.h: revision 1.8
- Allow creating timeshard kthreads. To be used to fix the RAIDframe
parity rewrite issue.
- Create kthreads in the SCHED_RR class by default, not SCHED_FIFO.
 1.24.10.2.8.2  14-Oct-2011  matt Improved kthread join support
 1.24.10.2.8.1  20-Feb-2011  matt Pullup http://mail-index.netbsd.org/source-changes/2010/05/11/msg009810.html
to this branch because the new flash framework will need it.
 1.24.10.2.4.1  27-Dec-2011  matt Backport joinable kthread support (for flash infrastructure).
 1.24.8.1  03-Mar-2009  skrll Sync with HEAD.
 1.27.4.4  31-May-2011  rmind sync with head
 1.27.4.3  05-Mar-2011  rmind sync with head
 1.27.4.2  03-Jul-2010  rmind sync with head
 1.27.4.1  30-May-2010  rmind sync with head
 1.27.2.1  17-Aug-2010  uebayasi Sync with HEAD.
 1.30.4.1  05-Mar-2011  bouyer Sync with HEAD
 1.30.2.1  06-Jun-2011  jruoho Sync with HEAD.
 1.38.10.1  28-Nov-2012  matt Pull from HEAD:
Add a __HAVE_CPU_UAREA_IDLELWP hook so that the MD code can allocate
special UAREAs for idle lwp's.
 1.38.2.1  30-Oct-2012  yamt sync with head
 1.39.16.2  28-Aug-2017  skrll Sync with HEAD
 1.39.16.1  06-Jun-2015  skrll Sync with HEAD
 1.39.2.1  03-Dec-2017  jdolecek update from HEAD
 1.41.4.1  21-Apr-2017  bouyer Sync with HEAD
 1.41.2.1  26-Apr-2017  pgoyette Sync with HEAD
 1.43.4.1  08-Apr-2020  martin Merge changes from current as of 20200406
 1.44.2.1  17-Jan-2020  ad Sync with head.

RSS XML Feed