Home | History | Annotate | Download | only in uvm
History log of /src/sys/uvm/uvm_amap.h
RevisionDateAuthorComments
 1.41  20-Mar-2020  ad Go back to freeing struct vm_anon one by one. There may have been an
advantage circa ~2008 but there isn't now.
 1.40  23-Feb-2020  ad UVM locking changes, proposed on tech-kern:

- Change the lock on uvm_object, vm_amap and vm_anon to be a RW lock.
- Break v_interlock and vmobjlock apart. v_interlock remains a mutex.
- Do partial PV list locking in the x86 pmap. Others to follow later.
 1.39  02-Jan-2020  ad branches: 1.39.2;
Back out the amap allocation changes from earlier today - have seen a panic
with them. Retain the lock changes.
 1.38  01-Jan-2020  ad - Start trying to reduce the high cache miss rate observed around vm_amap.
On _LP64, pad struct vm_amap to 128 bytes and use the additional space to
hold the arrays for tiny amaps which are common. Carefully size the array
allocations to avoid false sharing, and for smaller amaps try to share
allocated cache lines.

- Eliminate most contention due to amap_list: maintain the list in the pool
cache constructor / destructor like we do for struct file. Cache the
mutexes we allocate here.

- Don't do PR_WAITOK mutex allocations when NOWAIT has been specified.
 1.37  12-Jun-2011  rmind branches: 1.37.2; 1.37.54;
Welcome to 5.99.53! Merge rmind-uvmplock branch:

- Reorganize locking in UVM and provide extra serialisation for pmap(9).
New lock order: [vmpage-owner-lock] -> pmap-lock.

- Simplify locking in some pmap(9) modules by removing P->V locking.

- Use lock object on vmobjlock (and thus vnode_t::v_interlock) to share
the locks amongst UVM objects where necessary (tmpfs, layerfs, unionfs).

- Rewrite and optimise x86 TLB shootdown code, make it simpler and cleaner.
Add TLBSTATS option for x86 to collect statistics about TLB shootdowns.

- Unify /dev/mem et al in MI code and provide required locking (removes
kernel-lock on some ports). Also, avoid cache-aliasing issues.

Thanks to Andrew Doran and Joerg Sonnenberger, as their initial patches
formed the core changes of this branch.
 1.36  23-Apr-2011  rmind branches: 1.36.2;
Replace "malloc" in comments, remove unnecessary header inclusions.
 1.35  02-Feb-2011  chuck udpate license clauses on my code to match the new-style BSD licenses.
based on diff that rmind@ sent me.

no functional change with this commit.
 1.34  26-Oct-2008  bjs branches: 1.34.16; 1.34.22; 1.34.24;
"sparce" -> "sparse" + commas after "large", prior to "sparse"
 1.33  21-Jul-2007  ad branches: 1.33.26; 1.33.30; 1.33.36; 1.33.38;
Merge unobtrusive locking changes from the vmlocking branch.
 1.32  22-Feb-2007  matt branches: 1.32.4; 1.32.12;
Fix lossage from boolean_t -> bool and updated x86 bus_dma.
 1.31  21-Feb-2007  thorpej Replace the Mach-derived boolean_t type with the C99 bool type. A
future commit will replace use of TRUE and FALSE with true and false.
 1.30  25-Jun-2006  yamt branches: 1.30.10;
make amap use kmem_alloc, rather than malloc.
(ie. make it use kernel_map, rather than kmem_map.)
kmem_map is more restricted than kernel_map,
and there's no point for amap to use it.
 1.29  15-Feb-2006  yamt branches: 1.29.2; 1.29.10;
- amap_copy: take a "flags" argument instead of booleans.
- add AMAP_COPY_NOMERGE flag, and use it for uvm_map_extract.
PR/32806 from Julio M. Merino Vidal.
 1.28  11-Feb-2006  yamt remove the following options. no objections on tech-kern@.

UVM_PAGER_INLINE
UVM_AMAP_INLINE
UVM_PAGE_INLINE
UVM_MAP_INLINE
 1.27  24-Dec-2005  perry branches: 1.27.2; 1.27.4; 1.27.6;
Remove leading __ from __(const|inline|signed|volatile) -- it is obsolete.
 1.26  11-Dec-2005  christos merge ktrace-lwp.
 1.25  11-May-2005  yamt branches: 1.25.2;
allocate anons on-demand, rather than reserving static amount of
them on boot/swapon.
 1.24  25-Apr-2004  simonb Initialise (most) pools from a link set instead of explicit calls
to pool_init. Untouched pools are ones that either in arch-specific
code, or aren't initialiased during initial system startup.

Convert struct session, ucred and lockf to pools.
 1.23  24-Mar-2004  junyoung Nuke __P().
 1.22  01-Feb-2003  thorpej branches: 1.22.2;
Add extensible malloc types, adapted from FreeBSD. This turns
malloc types into a structure, a pointer to which is passed around,
instead of an int constant. Allow the limit to be adjusted when the
malloc type is defined, or with a function call, as suggested by
Jonathan Stone.
 1.21  20-Dec-2002  atatat Properly set page references counts at the start of the newly
allocated ppref data to zero in the case of an amap that has empty
space at the front.

Don't set anything in the ppref array if "len" is zero.

Many thanks to Sami Kantoluoto for providing gdb access to a machine
that would reliably crash with problems related to the above, and to
Stephan Thesing for corroborating that the patch properly addressed
the problem.

Note that the ar_pageoff (and related variables) types must be changed
soon. The use of "int" here is not theoretically sufficient.
 1.20  30-Nov-2002  bouyer Change uvm_km_kmemalloc() to accept flag UVM_KMF_NOWAIT and pass it to
uvm_map(). Change uvm_map() to honnor UVM_KMF_NOWAIT. For this, change
amap_extend() to take a flags parameter instead of just boolean for
direction, and introduce AMAP_EXTEND_FORWARDS and AMAP_EXTEND_NOWAIT flags
(AMAP_EXTEND_BACKWARDS is still defined as 0x0, to keep the code easier to
read).
Add a flag parameter to uvm_mapent_alloc().
This solves a problem a pool_get(PR_NOWAIT) could trigger a pool_get(PR_WAITOK)
in uvm_mapent_alloc().
Thanks to Chuck Silvers, enami tsugutomo, Andrew Brown and Jason R Thorpe
for feedback.
 1.19  14-Nov-2002  atatat Implement backwards extension of amaps. There are three cases to deal
with:

Case #1 -- adjust offset: The slot offset in the aref can be
decremented to cover the required size addition.

Case #2 -- move pages and adjust offset: The slot offset is not large
enough, but the amap contains enough inactive space *after* the mapped
pages to make up the difference, so active slots are slid to the "end"
of the amap, and the slot offset is, again, adjusted to cover the
required size addition. This optimizes for hitting case #1 again on
the next small extension.

Case #3 -- reallocate, move pages, and adjust offset: There is not
enough inactive space in the amap, so the arrays are reallocated, and
the active pages are copied again to the "end" of the amap, and the
slot offset is adjusted to cover the required size. This also
optimizes for hitting case #1 on the next backwards extension.

This provides the missing piece in the "forward extension of
vm_map_entries" logic, so the merge failure counters have been
removed.

Not many applications will make any use of this at this time (except
for jvms and perhaps gcc3), but a "top-down" memory allocator will use
it extensively.
 1.18  15-Sep-2002  chs add a new km flag UVM_KMF_CANFAIL, which causes uvm_km_kmemalloc() to
return failure if swap is full and there are no free physical pages.
have malloc() use this flag if M_CANFAIL is passed to it.
use M_CANFAIL to allow amap_extend() to fail when memory is scarce.
this should prevent most of the remaining hangs in low-memory situations.
 1.17  02-Jun-2001  chs branches: 1.17.2; 1.17.10; 1.17.16;
replace vm_map{,_entry}_t with struct vm_map{,_entry} *.
 1.16  26-May-2001  chs replace {simple_,}lock{_data,}_t with struct {simple,}lock {,*}.
 1.15  25-May-2001  chs remove trailing whitespace.
 1.14  18-Feb-2001  chs branches: 1.14.2;
clean up DIAGNOSTIC checks, use KASSERT().
 1.13  25-Nov-2000  chs lots of cleanup:
use queue.h macros and KASSERT().
address amap offsets in pages instead of bytes.
make amap_ref() and amap_unref() take an amap, offset and length
instead of a vm_map_entry_t.
improve whitespace and comments.
 1.12  07-Jul-1999  thorpej branches: 1.12.2;
Don't bother returning the "slot" number from amap_add():
* Nothing currently uses this return value.
* It's arguably an abstraction violation.

Fix amap_unadd()'s API to be consistent w/ amap_add()'s: rather than
take a vm_amap * and a slot number, take a vm_aref * and an offset.

It's now actually possible to use amap_unadd() to remove an anon from
an amap.
 1.11  21-Jun-1999  thorpej Protect prototypes, certain macros, and inlines from userland.
 1.10  28-Jan-1999  chuck branches: 1.10.4;
comment cleanup, shift around the inline stuff a bit,
rename VM_AMAP_PPREF (to UVM_AMAP_PPREF).
 1.9  24-Jan-1999  chuck cleanup/reorg:
- break anon related functions out of uvm_amap.c and put them in their own
file (uvm_anon.c). includes break up uvm_anon_init into an amap and an
an anon init function
- ensure that only functions within the amap module access amap structure
fields (add macros to amap api as needed)
 1.8  18-Oct-1998  chs branches: 1.8.2;
shift by PAGE_SHIFT instead of multiplying or dividing by PAGE_SIZE.
 1.7  13-Aug-1998  eeh Merge paddr_t changes into the main branch.
 1.6  09-Mar-1998  mrg branches: 1.6.2;
KNF.
 1.5  10-Feb-1998  mrg - add defopt's for UVM, UVMHIST and PMAP_NEW.
- remove unnecessary UVMHIST_DECL's.
 1.4  10-Feb-1998  perry add/cleanup multiple inclusion protection.
 1.3  07-Feb-1998  mrg restore rcsids
 1.2  06-Feb-1998  thorpej RCS ID police.
 1.1  05-Feb-1998  mrg branches: 1.1.1;
Initial revision
 1.1.1.1  05-Feb-1998  mrg initial import of the new virtual memory system, UVM, into -current.

UVM was written by chuck cranor <chuck@maria.wustl.edu>, with some
minor portions derived from the old Mach code. i provided some help
getting swap and paging working, and other bug fixes/ideas. chuck
silvers <chuq@chuq.com> also provided some other fixes.

this is the UVM kernel code portion.


this will be KNF'd shortly. :-)
 1.6.2.1  30-Jul-1998  eeh Split vm_offset_t and vm_size_t into paddr_t, psize_t, vaddr_t, and vsize_t.
 1.8.2.1  09-Nov-1998  chs initial snapshot. lots left to do.
 1.10.4.3  02-Aug-1999  thorpej Update from trunk.
 1.10.4.2  01-Jul-1999  thorpej Sync w/ -current.
 1.10.4.1  07-Jun-1999  chs merge everything from chs-ubc branch.
 1.12.2.2  12-Mar-2001  bouyer Sync with HEAD.
 1.12.2.1  08-Dec-2000  bouyer Sync with HEAD.
 1.14.2.4  20-Dec-2002  thorpej Sync with HEAD.
 1.14.2.3  11-Dec-2002  thorpej Sync with HEAD.
 1.14.2.2  17-Sep-2002  nathanw Catch up to -current.
 1.14.2.1  21-Jun-2001  nathanw Catch up to -current.
 1.17.16.1  02-Jun-2003  tron Pull up revision 1.18 (requested by skrll):
add a new km flag UVM_KMF_CANFAIL, which causes uvm_km_kmemalloc() to
return failure if swap is full and there are no free physical pages.
have malloc() use this flag if M_CANFAIL is passed to it.
use M_CANFAIL to allow amap_extend() to fail when memory is scarce.
this should prevent most of the remaining hangs in low-memory situations.
 1.17.10.1  12-Mar-2002  thorpej Make the amap lock an adaptive mutex.
 1.17.2.1  10-Oct-2002  jdolecek sync kqueue with -current; this includes merge of gehenna-devsw branch,
merge of i386 MP branch, and part of autoconf rototil work
 1.22.2.4  10-Nov-2005  skrll Sync with HEAD. Here we go again...
 1.22.2.3  21-Sep-2004  skrll Fix the sync with head I botched.
 1.22.2.2  18-Sep-2004  skrll Sync with HEAD.
 1.22.2.1  03-Aug-2004  skrll Sync with HEAD
 1.25.2.4  03-Sep-2007  yamt sync with head.
 1.25.2.3  26-Feb-2007  yamt sync with head.
 1.25.2.2  30-Dec-2006  yamt sync with head.
 1.25.2.1  21-Jun-2006  yamt sync with head.
 1.27.6.1  22-Apr-2006  simonb Sync with head.
 1.27.4.1  09-Sep-2006  rpaulo sync with head
 1.27.2.1  18-Feb-2006  yamt sync with head.
 1.29.10.1  13-Jul-2006  gdamore Merge from HEAD.
 1.29.2.1  26-Jun-2006  yamt sync with head.
 1.30.10.1  27-Feb-2007  yamt - sync with head.
- move sched_changepri back to kern_synch.c as it doesn't know PPQ anymore.
 1.32.12.1  15-Aug-2007  skrll Sync with HEAD.
 1.32.4.1  13-Mar-2007  ad Pull in the initial set of changes for the vmlocking branch.
 1.33.38.2  21-Jul-2007  ad Merge unobtrusive locking changes from the vmlocking branch.
 1.33.38.1  21-Jul-2007  ad file uvm_amap.h was added on branch matt-mips64 on 2007-07-21 19:21:54 +0000
 1.33.36.1  13-Dec-2008  haad Update haad-dm branch to haad-dm-base2.
 1.33.30.1  04-May-2009  yamt sync with head.
 1.33.26.1  17-Jan-2009  mjf Sync with HEAD.
 1.34.24.1  08-Feb-2011  bouyer Sync with HEAD
 1.34.22.1  06-Jun-2011  jruoho Sync with HEAD.
 1.34.16.3  31-May-2011  rmind sync with head
 1.34.16.2  05-Mar-2011  rmind sync with head
 1.34.16.1  17-Mar-2010  rmind Reorganise UVM locking to protect P->V state and serialise pmap(9)
operations on the same page(s) by always locking their owner. Hence
lock order: "vmpage"-lock -> pmap-lock.

Patch, proposed on tech-kern@, from Andrew Doran.
 1.36.2.1  23-Jun-2011  cherry Catchup with rmind-uvmplock merge.
 1.37.54.1  08-Apr-2020  martin Merge changes from current as of 20200406
 1.37.2.1  26-Dec-2011  yamt - use O->A loan to serve read(2). based on a patch from Chuck Silvers
- associated O->A loan fixes.
 1.39.2.1  29-Feb-2020  ad Sync with head.

RSS XML Feed