Home | History | Annotate | Download | only in uvm
History log of /src/sys/uvm/uvm_anon.c
RevisionDateAuthorComments
 1.80  25-Oct-2020  chs Handle PG_PAGEOUT in uvm_anon_release() too.
 1.79  09-Jul-2020  skrll Consistently use UVMHIST(__func__)

Convert UVMHIST_{CALLED,LOG} into UVMHIST_CALLARGS
 1.78  08-Jul-2020  skrll Trailing whitespace
 1.77  22-Mar-2020  ad Process concurrent page faults on individual uvm_objects / vm_amaps in
parallel, where the relevant pages are already in-core. Proposed on
tech-kern.

Temporarily disabled on MP architectures with __HAVE_UNLOCKED_PMAP until
adjustments are made to their pmaps.
 1.76  20-Mar-2020  ad Go back to freeing struct vm_anon one by one. There may have been an
advantage circa ~2008 but there isn't now.
 1.75  14-Mar-2020  ad Make page waits (WANTED vs BUSY) interlocked by pg->interlock. Gets RW
locks out of the equation for sleep/wakeup, and allows observing+waiting
for busy pages when holding only a read lock. Proposed on tech-kern.
 1.74  24-Feb-2020  rin 0x%#x --> %#x for non-external codes.
Also, stop mixing up 0x%x and %#x in single files as far as possible.
 1.73  23-Feb-2020  ad Use rw_lock_op().
 1.72  23-Feb-2020  ad UVM locking changes, proposed on tech-kern:

- Change the lock on uvm_object, vm_amap and vm_anon to be a RW lock.
- Break v_interlock and vmobjlock apart. v_interlock remains a mutex.
- Do partial PV list locking in the x86 pmap. Others to follow later.
 1.71  15-Jan-2020  ad Merge from yamt-pagecache (after much testing):

- Reduce unnecessary page scan in putpages esp. when an object has a ton of
pages cached but only a few of them are dirty.

- Reduce the number of pmap operations by tracking page dirtiness more
precisely in uvm layer.
 1.70  31-Dec-2019  ad branches: 1.70.2;
- Add and use wrapper functions that take and acquire page interlocks, and pairs
of page interlocks. Require that the page interlock be held over calls to
uvm_pageactivate(), uvm_pagewire() and similar.

- Solve the concurrency problem with page replacement state. Rather than
updating the global state synchronously, set an intended state on
individual pages (active, inactive, enqueued, dequeued) while holding the
page interlock. After the interlock is released put the pages on a 128
entry per-CPU queue for their state changes to be made real in batch.
This results in in a ~400 fold decrease in contention on my test system.
Proposed on tech-kern but modified to use the page interlock rather than
atomics to synchronise as it's much easier to maintain that way, and
cheaper.
 1.69  13-Dec-2019  ad Break the global uvm_pageqlock into a per-page identity lock and a private
lock for use of the pagedaemon policy code. Discussed on tech-kern.

PR kern/54209: NetBSD 8 large memory performance extremely low
PR kern/54210: NetBSD-8 processes presumably not exiting
PR kern/54727: writing a large file causes unreasonable system behaviour
 1.68  02-Dec-2019  chs fix the build for when UVMHIST is enabled.
 1.67  01-Dec-2019  uwe Add missing #include <sys/atomic.h>
 1.66  01-Dec-2019  ad Free pages in batch instead of taking uvm_pageqlock for each one.
 1.65  01-Dec-2019  ad - Adjust uvmexp.swpgonly with atomics, and make uvm_swap_data_lock static.
- A bit more __cacheline_aligned on mutexes.
 1.64  28-Oct-2017  pgoyette branches: 1.64.4; 1.64.8;
Update the kernhist(9) kernel history code to address issues identified
in PR kern/52639, as well as some general cleaning-up...

(As proposed on tech-kern@ with additional changes and enhancements.)

Details of changes:

* All history arguments are now stored as uintmax_t values[1], both in
the kernel and in the structures used for exporting the history data
to userland via sysctl(9). This avoids problems on some architectures
where passing a 64-bit (or larger) value to printf(3) can cause it to
process the value as multiple arguments. (This can be particularly
problematic when printf()'s format string is not a literal, since in
that case the compiler cannot know how large each argument should be.)

* Update the data structures used for exporting kernel history data to
include a version number as well as the length of history arguments.

* All [2] existing users of kernhist(9) have had their format strings
updated. Each format specifier now includes an explicit length
modifier 'j' to refer to numeric values of the size of uintmax_t.

* All [2] existing users of kernhist(9) have had their format strings
updated to replace uses of "%p" with "%#jx", and the pointer
arguments are now cast to (uintptr_t) before being subsequently cast
to (uintmax_t). This is needed to avoid compiler warnings about
casting "pointer to integer of a different size."

* All [2] existing users of kernhist(9) have had instances of "%s" or
"%c" format strings replaced with numeric formats; several instances
of mis-match between format string and argument list have been fixed.

* vmstat(1) has been modified to handle the new size of arguments in the
history data as exported by sysctl(9).

* vmstat(1) now provides a warning message if the history requested with
the -u option does not exist (previously, this condition was silently
ignored, with only a single blank line being printed).

* vmstat(1) now checks the version and argument length included in the
data exported via sysctl(9) and exits if they do not match the values
with which vmstat was built.

* The kernhist(9) man-page has been updated to note the additional
requirements imposed on the format strings, along with several other
minor changes and enhancements.

[1] It would have been possible to use an explicit length (for example,
uint64_t) for the history arguments. But that would require another
"rototill" of all the users in the future when we add support for an
architecture that supports a larger size. Also, the printf(3) format
specifiers for explicitly-sized values, such as "%"PRIu64, are much
more verbose (and less aesthetically appealing, IMHO) than simply
using "%ju".

[2] I've tried very hard to find "all [the] existing users of kernhist(9)"
but it is possible that I've missed some of them. I would be glad to
update any stragglers that anyone identifies.
 1.63  25-Oct-2013  martin branches: 1.63.22;
Mark a diagnostic-only variable
 1.62  18-Aug-2011  yamt branches: 1.62.2; 1.62.12; 1.62.16;
uvm_anon_release:
- don't forget to call uvm_anon_dispose.
- simplify code a little.
 1.61  18-Aug-2011  yamt uvm_anon_freelst:
- clear an_link/an_ref when deferring anon disposal. otherwise others can
see bogus an_ref.
- fix the code to remove anon from the list.
 1.60  14-Aug-2011  rmind uvm_anon_freelst: do not free PG_RELEASED pages (change uvm_anon_dispose()
to indicate them with a return value).
 1.59  06-Aug-2011  rmind - Rework uvm_anfree() into uvm_anon_freelst(), which always drops the lock.
- Free anons in uvm_anon_freelst() without lock held.
- Mechanic sync to unused loaning code.
 1.58  05-Jul-2011  yamt reduce the number of atomic ops in common cases. it's exceptional for
anons to remain longer than amap.
 1.57  24-Jun-2011  rmind Fix uvmplock regression - a lock against oneself case in amap_swap_off().
Happens since amap is NULL in uvmfault_anonget(), so uvmfault_unlockall()
keeps anon locked, when it should unlock it.
 1.56  24-Jun-2011  yamt uvm_anon_release: fix a locking error after the rmind-uvmplock merge
 1.55  17-Jun-2011  rmind Improve comments on uvm_anon.c, tidy up slightly.
No functional changes.
 1.54  12-Jun-2011  rmind Welcome to 5.99.53! Merge rmind-uvmplock branch:

- Reorganize locking in UVM and provide extra serialisation for pmap(9).
New lock order: [vmpage-owner-lock] -> pmap-lock.

- Simplify locking in some pmap(9) modules by removing P->V locking.

- Use lock object on vmobjlock (and thus vnode_t::v_interlock) to share
the locks amongst UVM objects where necessary (tmpfs, layerfs, unionfs).

- Rewrite and optimise x86 TLB shootdown code, make it simpler and cleaner.
Add TLBSTATS option for x86 to collect statistics about TLB shootdowns.

- Unify /dev/mem et al in MI code and provide required locking (removes
kernel-lock on some ports). Also, avoid cache-aliasing issues.

Thanks to Andrew Doran and Joerg Sonnenberger, as their initial patches
formed the core changes of this branch.
 1.53  23-Apr-2011  rmind branches: 1.53.2;
Replace "malloc" in comments, remove unnecessary header inclusions.
 1.52  02-Feb-2011  chuck udpate license clauses on my code to match the new-style BSD licenses.
based on diff that rmind@ sent me.

no functional change with this commit.
 1.51  18-Jan-2008  yamt branches: 1.51.28; 1.51.32; 1.51.38; 1.51.40;
push pmap_clear_reference calls into pdpolicy code, where reference bits
actually matter.
 1.50  02-Jan-2008  ad Merge vmlocking2 to head.
 1.49  20-Dec-2007  ad Specify PR_LARGECACHE for anon_cache (which is insanely busy).
 1.48  13-Nov-2007  yamt branches: 1.48.2; 1.48.6;
g/c unused uvm_anon_pool.
 1.47  07-Nov-2007  ad Merge from vmlocking:

- pool_cache changes.
- Debugger/procfs locking fixes.
- Other minor changes.
 1.46  11-Oct-2007  ad branches: 1.46.2; 1.46.4;
Remove LOCK_ASSERT(!simple_lock_held(&foo));
 1.45  21-Jul-2007  ad branches: 1.45.4; 1.45.6; 1.45.8; 1.45.10;
Merge unobtrusive locking changes from the vmlocking branch.
 1.44  12-Mar-2007  ad branches: 1.44.8;
Pass an ipl argument to pool_init/POOL_INIT to be used when initializing
the pool's lock.
 1.43  22-Feb-2007  thorpej branches: 1.43.4;
TRUE -> true, FALSE -> false
 1.42  21-Feb-2007  thorpej Replace the Mach-derived boolean_t type with the C99 bool type. A
future commit will replace use of TRUE and FALSE with true and false.
 1.41  01-Nov-2006  yamt branches: 1.41.4;
remove some __unused from function parameters.
 1.40  12-Oct-2006  christos - sprinkle __unused on function decls.
- fix a couple of unused bugs
- no more -Wno-unused for i386
 1.39  15-Sep-2006  yamt branches: 1.39.2;
merge yamt-pdpolicy branch.
- separate page replacement policy from the rest of kernel
- implement an alternative replacement policy
 1.38  11-Dec-2005  christos branches: 1.38.8; 1.38.20;
merge ktrace-lwp.
 1.37  13-Sep-2005  yamt wrap swap related code by #ifdef VMSWAP. always #define VMSWAP for now.
 1.36  31-Jul-2005  yamt revert "defflag VMSWAP" changes for now.
there seems to be far more people who don't want to edit
their kernel config files than i thought.
 1.35  30-Jul-2005  yamt defflag VMSWAP.
 1.34  27-Jun-2005  thorpej branches: 1.34.2;
Use ANSI function decls.
 1.33  11-May-2005  yamt allocate anons on-demand, rather than reserving static amount of
them on boot/swapon.
 1.32  01-Apr-2005  yamt merge yamt-km branch.
- don't use managed mappings/backing objects for wired memory allocations.
save some resources like pv_entry. also fix (most of) PR/27030.
- simplify kernel memory management API.
- simplify pmap bootstrap of some ports.
- some related cleanups.
 1.31  01-Sep-2004  yamt branches: 1.31.4; 1.31.6;
uvm_pagefree: when orphaning an A->K loaned page,
- decrement uvmexp.anonpages as it's no longer an anon page.
- null out anon->u.an_page as the anon no longer own the page.
uvm_anfree: add related assertions.
 1.30  01-Sep-2004  yamt uvm_anfree: remove a comment which is no longer true.
 1.29  05-May-2004  yamt fix a amap_wirerange deadlock problem by re-introducing
PG_RELEASED for anon pages. PR/23171 from Christian Limpach.
for details, see discussion filed in the PR database.

uvm_anon_release: a new function to free anon-owned PG_RELEASED page.
uvm_anfree: we can't wait for the page here because the caller might hold
amap lock. instead, just mark the page as PG_RELEASED.
who unbusy the page should check the PG_RELEASED.
uvm_aio_aiodone: uvm_anon_release() instead of uvm_page_unbusy()
if appropriate.
uvmfault_anonget: check PG_RELEASED.
 1.28  24-Mar-2004  junyoung branches: 1.28.2;
Nuke __P().
 1.27  06-Jan-2004  chs fix lock initialization in uvm_anon_add(). from PR 23831.
 1.26  28-Aug-2003  pk When retiring a swap device with marked bad blocks on it we should update
the `# swap page in use' and `# swap page only' counters. However, at the
time of swap device removal we can no longer figure out how many of the
bad swap pages are actually also `swap only' pages.

So, on swap I/O errors arrange things to not include the bad swap pages in
the `swpgonly' counter as follows: uvm_swap_markbad() decrements `swpgonly'
by the number of bad pages, and the various VM object deallocation routines
do not decrement `swpgonly' for swap slots marked as SWSLOT_BAD.
 1.25  11-Aug-2003  pk uao_pagein_page() & anon_pagein():
* return failure if the page cannot be retrieved.
* wakeup any waiters when releasing a page after successful page in.
 1.24  11-Aug-2003  pk Only deactivate pages if their wired count is zero.
 1.23  11-Aug-2003  pk Make sure to call uvm_swap_free() and uvm_swap_markbad() with valid (i.e.
positive) slot numbers.
 1.22  21-Sep-2002  chs branches: 1.22.6;
add missing anon lock around call to uvm_anon_lockloanpg().
 1.21  10-Nov-2001  lukem branches: 1.21.4;
add RCSIDs, and in some cases, slightly cleanup #include order
 1.20  06-Nov-2001  chs several changes prompted by loaning problems:
- fix the loaned case in uvm_pagefree().
- redo uvmexp.swpgonly accounting to work with page loaning.
add an assertion before each place we adjust uvmexp.swpgonly.
- fix uvm_km_pgremove() to always free any swap space associated with
the range being removed.
- get rid of UVM_LOAN_WIRED flag. instead, we just make sure that
pages loaned to the kernel are never on the page queues.
this allows us to assert that pages are not loaned and wired
at the same time.
- add yet more assertions.
 1.19  21-Oct-2001  chs branches: 1.19.2;
add some missing spinlocks.
 1.18  15-Sep-2001  chs a whole bunch of changes to improve performance and robustness under load:

- remove special treatment of pager_map mappings in pmaps. this is
required now, since I've removed the globals that expose the address range.
pager_map now uses pmap_kenter_pa() instead of pmap_enter(), so there's
no longer any need to special-case it.
- eliminate struct uvm_vnode by moving its fields into struct vnode.
- rewrite the pageout path. the pager is now responsible for handling the
high-level requests instead of only getting control after a bunch of work
has already been done on its behalf. this will allow us to UBCify LFS,
which needs tighter control over its pages than other filesystems do.
writing a page to disk no longer requires making it read-only, which
allows us to write wired pages without causing all kinds of havoc.
- use a new PG_PAGEOUT flag to indicate that a page should be freed
on behalf of the pagedaemon when it's unlocked. this flag is very similar
to PG_RELEASED, but unlike PG_RELEASED, PG_PAGEOUT can be cleared if the
pageout fails due to eg. an indirect-block buffer being locked.
this allows us to remove the "version" field from struct vm_page,
and together with shrinking "loan_count" from 32 bits to 16,
struct vm_page is now 4 bytes smaller.
- no longer use PG_RELEASED for swap-backed pages. if the page is busy
because it's being paged out, we can't release the swap slot to be
reallocated until that write is complete, but unlike with vnodes we
don't keep a count of in-progress writes so there's no good way to
know when the write is done. instead, when we need to free a busy
swap-backed page, just sleep until we can get it busy ourselves.
- implement a fast-path for extending writes which allows us to avoid
zeroing new pages. this substantially reduces cpu usage.
- encapsulate the data used by the genfs code in a struct genfs_node,
which must be the first element of the filesystem-specific vnode data
for filesystems which use genfs_{get,put}pages().
- eliminate many of the UVM pagerops, since they aren't needed anymore
now that the pager "put" operation is a higher-level operation.
- enhance the genfs code to allow NFS to use the genfs_{get,put}pages
instead of a modified copy.
- clean up struct vnode by removing all the fields that used to be used by
the vfs_cluster.c code (which we don't use anymore with UBC).
- remove kmem_object and mb_object since they were useless.
instead of allocating pages to these objects, we now just allocate
pages with no object. such pages are mapped in the kernel until they
are freed, so we can use the mapping to find the page to free it.
this allows us to remove splvm() protection in several places.

The sum of all these changes improves write throughput on my
decstation 5000/200 to within 1% of the rate of NetBSD 1.5
and reduces the elapsed time for "make release" of a NetBSD 1.5
source tree on my 128MB pc to 10% less than a 1.5 kernel took.
 1.17  25-May-2001  chs branches: 1.17.2; 1.17.4;
remove trailing whitespace.
 1.16  10-Mar-2001  chs eliminate the VM_PAGER_* error codes in favor of the traditional E* codes.
the mapping is:

VM_PAGER_OK 0
VM_PAGER_BAD <unused>
VM_PAGER_FAIL <unused>
VM_PAGER_PEND 0 (see below)
VM_PAGER_ERROR EIO
VM_PAGER_AGAIN EAGAIN
VM_PAGER_UNLOCK EBUSY
VM_PAGER_REFAULT ERESTART

for async i/o requests, it used to be possible for the request to
be convert to sync, and the pager would return VM_PAGER_OK or VM_PAGER_PEND
to indicate whether the caller should perform post-i/o cleanup.
this is no longer allowed; pagers must now return 0 to indicate that
the async i/o was successfully started, and the caller never needs to
worry about doing the post-i/o cleanup.
 1.15  18-Feb-2001  chs branches: 1.15.2;
clean up DIAGNOSTIC checks, use KASSERT().
 1.14  28-Jan-2001  thorpej Page scanner improvements, behavior is actually a bit more like
Mach VM's now. Specific changes:
- Pages now need not have all of their mappings removed before being
put on the inactive list. They only need to have the "referenced"
attribute cleared. This makes putting pages onto the inactive list
much more efficient. In order to eliminate redundant clearings of
"refrenced", callers of uvm_pagedeactivate() must now do this
themselves.
- When checking the "modified" attribute for a page (for clearing
PG_CLEAN), make sure to only do it if PG_CLEAN is currently set on
the page (saves a potentially expensive pmap operation).
- When scanning the inactive list, if a page is referenced, reactivate
it (this part was actually added in uvm_pdaemon.c,v 1.27). This
now works properly now that pages on the inactive list are allowed to
have mappings.
- When scanning the inactive list and considering a page for freeing,
remove all mappings, and then check the "modified" attribute if the
page is marked PG_CLEAN.
- When scanning the active list, if the page was referenced since its
last sweep by the scanner, don't deactivate it. (This part was
actually added in uvm_pdaemon.c,v 1.28.)

These changes greatly improve interactive performance during
moderate to high memory and I/O load.
 1.13  23-Jan-2001  thorpej Change uvm_analloc() to return a locked anon, update all callers,
and fix an anon locking protocol error in uvm_loanzero().
 1.12  23-Jan-2001  thorpej Sprinkle some assertions:
amap_free(): Assert that the amap is locked.
amap_share_protect(): Assert that the amap is locked.
amap_wipeout(): Assert that the amap is locked.
uvm_anfree(): Assert that the anon has a reference count of 0 and is
not locked.
uvm_anon_lockloanpg(): Assert that the anon is locked.
anon_pagein(): Assert that the anon is locked.
uvmfault_anonget(): Assert that the anon is locked.
uvm_pagealloc_strat(): Assert that the uobj or the anon is locked

And fix the problems these have uncovered:
amap_cow_now(): Lock the new anon after allocating it, and unref and
unlock it (rather than lock!) before freeing it in case
of an error condition. This should fix a problem reported
by Dan Carosone using cdrecord on an i386 MP kernel.
uvm_fault(): Case1B -- Lock the new anon afer allocating it, and unlock
it later when we unlock the old anon.
Case2 -- Lock the new anon after allocating it, and unlock
it later by passing it to uvmfault_unlockall() (we set anon
to NULL if we're not doing a promote fault).
 1.11  27-Dec-2000  chs when we fail to allocate anons to represent new swap space,
just return an error rather than panicing.
 1.10  25-Nov-2000  chs lots of cleanup:
use queue.h macros and KASSERT().
address amap offsets in pages instead of bytes.
make amap_ref() and amap_unref() take an amap, offset and length
instead of a vm_map_entry_t.
improve whitespace and comments.
 1.9  06-Aug-2000  thorpej Do something sane with a DIAGNOSTIC condition in an non-DIAGNOSTIC
kernel.
 1.8  05-Aug-2000  thorpej Correct a comment about locking wrt. uvmfault_anonget().
 1.7  27-Jun-2000  mrg remove include of <vm/vm.h>
 1.6  26-Jun-2000  mrg remove/move more mach vm header files:

<vm/pglist.h> -> <uvm/uvm_pglist.h>
<vm/vm_inherit.h> -> <uvm/uvm_inherit.h>
<vm/vm_kern.h> -> into <uvm/uvm_extern.h>
<vm/vm_object.h> -> nothing
<vm/vm_pager.h> -> into <uvm/uvm_pager.h>

also includes a bunch of <vm/vm_page.h> include removals (due to redudancy
with <vm/vm.h>), and a scattering of other similar headers.
 1.5  11-Jan-2000  chs branches: 1.5.4;
add support for ``swapctl -d'' (removing swap space).
improve handling of i/o errors in swap space.

reviewed by: Chuck Cranor
 1.4  12-Sep-1999  chs branches: 1.4.2;
eliminate the PMAP_NEW option by making it required for all ports.
ports which previously had no support for PMAP_NEW now implement
the pmap_k* interfaces as wrappers around the non-k versions.
 1.3  14-Aug-1999  ross In uvm_anon_init() and uvm_anon_add(), initialize the ref count lock.
 1.2  26-Mar-1999  chs add uvmexp.swpgonly and use it to detect out-of-swap conditions.
 1.1  24-Jan-1999  chuck cleanup/reorg:
- break anon related functions out of uvm_amap.c and put them in their own
file (uvm_anon.c). includes break up uvm_anon_init into an amap and an
an anon init function
- ensure that only functions within the amap module access amap structure
fields (add macros to amap api as needed)
 1.4.2.5  12-Mar-2001  bouyer Sync with HEAD.
 1.4.2.4  11-Feb-2001  bouyer Sync with HEAD.
 1.4.2.3  05-Jan-2001  bouyer Sync with HEAD
 1.4.2.2  08-Dec-2000  bouyer Sync with HEAD.
 1.4.2.1  20-Nov-2000  bouyer Update thorpej_scsipi to -current as of a month ago
 1.5.4.3  14-Feb-2002  he Pull up revision 1.11 (requested by chs):
Make memory allocation failures during ``swapctl -a'' return an error
instead of causing a panic.
 1.5.4.2  06-Aug-2000  thorpej Pull up rev. 1.9:
Do something sane with a DIAGNOSTIC condition in an non-DIAGNOSTIC
kernel.
 1.5.4.1  06-Aug-2000  thorpej Pull up rev. 1.8:
Correct a comment about locking wrt. uvmfault_anonget().
 1.15.2.6  18-Oct-2002  nathanw Catch up to -current.
 1.15.2.5  14-Nov-2001  nathanw Catch up to -current.
 1.15.2.4  22-Oct-2001  nathanw Catch up to -current.
 1.15.2.3  21-Sep-2001  nathanw Catch up to -current.
 1.15.2.2  21-Jun-2001  nathanw Catch up to -current.
 1.15.2.1  09-Apr-2001  nathanw Catch up with -current.
 1.17.4.1  01-Oct-2001  fvdl Catch up with -current.
 1.17.2.2  10-Oct-2002  jdolecek sync kqueue with -current; this includes merge of gehenna-devsw branch,
merge of i386 MP branch, and part of autoconf rototil work
 1.17.2.1  10-Jan-2002  thorpej Sync kqueue branch with -current.
 1.19.2.1  12-Nov-2001  thorpej Sync the thorpej-mips-cache branch with -current.
 1.21.4.2  12-Mar-2002  thorpej Make afreelock an adaptive mutex, and rename it to afree_mutex.
 1.21.4.1  11-Mar-2002  thorpej Convert swap_syscall_lock and uvm.swap_data_lock to adaptive mutexes,
and rename them apporpriately.
 1.22.6.6  10-Nov-2005  skrll Sync with HEAD. Here we go again...
 1.22.6.5  01-Apr-2005  skrll Sync with HEAD.
 1.22.6.4  21-Sep-2004  skrll Fix the sync with head I botched.
 1.22.6.3  18-Sep-2004  skrll Sync with HEAD.
 1.22.6.2  03-Sep-2004  skrll Sync with HEAD
 1.22.6.1  03-Aug-2004  skrll Sync with HEAD
 1.28.2.2  11-Sep-2004  he Pull up revisions 1.30-1.31 (requested by yamt in ticket #830):
Correct page accounting for anon pages: decrement
uvmexp.anonpages when orphaning an A->K loaned page, and
null out anon.u.an_page as the anon no longer owns the page
in that case. Add a few related assertions. Also correct
a comment.
 1.28.2.1  10-May-2004  tron Pull up revision 1.29 (requested by yamt in ticket #271):
fix a amap_wirerange deadlock problem by re-introducing
PG_RELEASED for anon pages. PR/23171 from Christian Limpach.
for details, see discussion filed in the PR database.
uvm_anon_release: a new function to free anon-owned PG_RELEASED page.
uvm_anfree: we can't wait for the page here because the caller might hold
amap lock. instead, just mark the page as PG_RELEASED.
who unbusy the page should check the PG_RELEASED.
uvm_aio_aiodone: uvm_anon_release() instead of uvm_page_unbusy()
if appropriate.
uvmfault_anonget: check PG_RELEASED.
 1.31.6.1  25-Jan-2005  yamt - don't use uvm_object or managed mappings for wired allocations.
(eg. malloc(9))
- simplify uvm_km_* apis.
 1.31.4.1  29-Apr-2005  kent sync with -current
 1.34.2.7  21-Jan-2008  yamt sync with head
 1.34.2.6  15-Nov-2007  yamt sync with head.
 1.34.2.5  27-Oct-2007  yamt sync with head.
 1.34.2.4  03-Sep-2007  yamt sync with head.
 1.34.2.3  26-Feb-2007  yamt sync with head.
 1.34.2.2  30-Dec-2006  yamt sync with head.
 1.34.2.1  21-Jun-2006  yamt sync with head.
 1.38.20.1  18-Nov-2006  ad Sync with head.
 1.38.8.2  15-Sep-2006  yamt make UVM_KICK_PDAEMON() a real function and stop including
uvm_pdpolicy.h from uvm.h. this also fixes build of pmap(1).
 1.38.8.1  05-Mar-2006  yamt separate page replacement policy from the rest of kernel.
 1.39.2.2  10-Dec-2006  yamt sync with head.
 1.39.2.1  22-Oct-2006  yamt sync with head
 1.41.4.2  24-Mar-2007  yamt sync with head.
 1.41.4.1  27-Feb-2007  yamt - sync with head.
- move sched_changepri back to kern_synch.c as it doesn't know PPQ anymore.
 1.43.4.5  01-Nov-2007  ad Yielding to avoid livelock doesn't work well, so just sleep for 1 tick.
This too is inadequate and a better solution must be found. Discussed
with yamt@.
 1.43.4.4  01-Sep-2007  ad Update for pool_cache API changes.
 1.43.4.3  03-Jul-2007  yamt if wrong-order trylocking failed, avoid livelock by yielding cpu
before retrying. ok'ed by Andrew Doran.
 1.43.4.2  13-Mar-2007  ad Pull in the initial set of changes for the vmlocking branch.
 1.43.4.1  13-Mar-2007  ad Sync with head.
 1.44.8.1  15-Aug-2007  skrll Sync with HEAD.
 1.45.10.2  21-Jul-2007  ad Merge unobtrusive locking changes from the vmlocking branch.
 1.45.10.1  21-Jul-2007  ad file uvm_anon.c was added on branch matt-mips64 on 2007-07-21 19:21:54 +0000
 1.45.8.1  14-Oct-2007  yamt sync with head.
 1.45.6.4  23-Mar-2008  matt sync with HEAD
 1.45.6.3  09-Jan-2008  matt sync with HEAD
 1.45.6.2  08-Nov-2007  matt sync with -HEAD
 1.45.6.1  06-Nov-2007  matt sync with HEAD
 1.45.4.3  14-Nov-2007  joerg Sync with HEAD.
 1.45.4.2  11-Nov-2007  joerg Sync with HEAD.
 1.45.4.1  26-Oct-2007  joerg Sync with HEAD.

Follow the merge of pmap.c on i386 and amd64 and move
pmap_init_tmp_pgtbl into arch/x86/x86/pmap.c. Modify the ACPI wakeup
code to restore CR4 before jumping back into kernel space as the large
page option might cover that.
 1.46.4.3  18-Feb-2008  mjf Sync with HEAD.
 1.46.4.2  27-Dec-2007  mjf Sync with HEAD.
 1.46.4.1  19-Nov-2007  mjf Sync with HEAD.
 1.46.2.1  13-Nov-2007  bouyer Sync with HEAD
 1.48.6.2  19-Jan-2008  bouyer Sync with HEAD
 1.48.6.1  02-Jan-2008  bouyer Sync with HEAD
 1.48.2.2  26-Dec-2007  ad Sync with head.
 1.48.2.1  04-Dec-2007  ad Pull the vmlocking changes into a new branch.
 1.51.40.1  08-Feb-2011  bouyer Sync with HEAD
 1.51.38.1  06-Jun-2011  jruoho Sync with HEAD.
 1.51.32.4  31-May-2011  rmind sync with head
 1.51.32.3  05-Mar-2011  rmind sync with head
 1.51.32.2  17-Mar-2010  rmind Reorganise UVM locking to protect P->V state and serialise pmap(9)
operations on the same page(s) by always locking their owner. Hence
lock order: "vmpage"-lock -> pmap-lock.

Patch, proposed on tech-kern@, from Andrew Doran.
 1.51.32.1  16-Mar-2010  rmind Change struct uvm_object::vmobjlock to be dynamically allocated with
mutex_obj_alloc(). It allows us to share the locks among UVM objects.
 1.51.28.3  07-May-2012  matt Move call to uvm_anon_dropswap to with #ifdef VMSWAP
 1.51.28.2  16-Feb-2012  matt Track the victims selected by the pagedaemon and what happens to then.
Keep a hint for what page group has the most free pages for a given color.
 1.51.28.1  14-Feb-2012  matt Add more KASSERTs (more! more! more!).
When returning page to the free pool, make sure to dequeue the pages before
hand or free page queue corruption will happen.
 1.53.2.1  23-Jun-2011  cherry Catchup with rmind-uvmplock merge.
 1.62.16.1  18-May-2014  rmind sync with head
 1.62.12.2  03-Dec-2017  jdolecek update from HEAD
 1.62.12.1  20-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.62.2.5  22-May-2014  yamt g/c a write-only variable
 1.62.2.4  22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.62.2.3  23-Jan-2012  yamt fix swapoff locking
 1.62.2.2  26-Dec-2011  yamt - use O->A loan to serve read(2). based on a patch from Chuck Silvers
- associated O->A loan fixes.
 1.62.2.1  02-Nov-2011  yamt page cache related changes

- maintain object pages in radix tree rather than rb tree.
- reduce unnecessary page scan in putpages. esp. when an object has a ton of
pages cached but only a few of them are dirty.
- reduce the number of pmap operations by tracking page dirtiness more
precisely in uvm layer.
- fix nfs commit range tracking.
- fix nfs write clustering. XXX hack
 1.63.22.1  02-Nov-2017  snj Pull up following revision(s) (requested by pgoyette in ticket #335):
share/man/man9/kernhist.9: 1.5-1.8
sys/arch/acorn26/acorn26/pmap.c: 1.39
sys/arch/arm/arm32/fault.c: 1.105 via patch
sys/arch/arm/arm32/pmap.c: 1.350, 1.359
sys/arch/arm/broadcom/bcm2835_bsc.c: 1.7
sys/arch/arm/omap/if_cpsw.c: 1.20
sys/arch/arm/omap/tiotg.c: 1.7
sys/arch/evbarm/conf/RPI2_INSTALL: 1.3
sys/dev/ic/sl811hs.c: 1.98
sys/dev/usb/ehci.c: 1.256
sys/dev/usb/if_axe.c: 1.83
sys/dev/usb/motg.c: 1.18
sys/dev/usb/ohci.c: 1.274
sys/dev/usb/ucom.c: 1.119
sys/dev/usb/uhci.c: 1.277
sys/dev/usb/uhub.c: 1.137
sys/dev/usb/umass.c: 1.160-1.162
sys/dev/usb/umass_quirks.c: 1.100
sys/dev/usb/umass_scsipi.c: 1.55
sys/dev/usb/usb.c: 1.168
sys/dev/usb/usb_mem.c: 1.70
sys/dev/usb/usb_subr.c: 1.221
sys/dev/usb/usbdi.c: 1.175
sys/dev/usb/usbdi_util.c: 1.67-1.70
sys/dev/usb/usbroothub.c: 1.3
sys/dev/usb/xhci.c: 1.75
sys/external/bsd/drm2/dist/drm/i915/i915_gem.c: 1.34
sys/kern/kern_history.c: 1.15
sys/kern/kern_xxx.c: 1.74
sys/kern/vfs_bio.c: 1.275-1.276
sys/miscfs/genfs/genfs_io.c: 1.71
sys/sys/kernhist.h: 1.21
sys/ufs/ffs/ffs_balloc.c: 1.63
sys/ufs/lfs/lfs_vfsops.c: 1.361
sys/ufs/lfs/ulfs_inode.c: 1.21
sys/ufs/lfs/ulfs_vnops.c: 1.52
sys/ufs/ufs/ufs_inode.c: 1.102
sys/ufs/ufs/ufs_vnops.c: 1.239
sys/uvm/pmap/pmap.c: 1.37-1.39
sys/uvm/pmap/pmap_tlb.c: 1.22
sys/uvm/uvm_amap.c: 1.108
sys/uvm/uvm_anon.c: 1.64
sys/uvm/uvm_aobj.c: 1.126
sys/uvm/uvm_bio.c: 1.91
sys/uvm/uvm_device.c: 1.66
sys/uvm/uvm_fault.c: 1.201
sys/uvm/uvm_km.c: 1.144
sys/uvm/uvm_loan.c: 1.85
sys/uvm/uvm_map.c: 1.353
sys/uvm/uvm_page.c: 1.194
sys/uvm/uvm_pager.c: 1.111
sys/uvm/uvm_pdaemon.c: 1.109
sys/uvm/uvm_swap.c: 1.175
sys/uvm/uvm_vnode.c: 1.103
usr.bin/vmstat/vmstat.c: 1.219
Reorder to test for null before null deref in debug code
--
Reorder to test for null before null deref in debug code
--
KNF
--
No need for '\n' in UVMHIST_LOG
--
normalise a BIOHIST log message
--
Update the kernhist(9) kernel history code to address issues identified
in PR kern/52639, as well as some general cleaning-up...
(As proposed on tech-kern@ with additional changes and enhancements.)
Details of changes:
* All history arguments are now stored as uintmax_t values[1], both in
the kernel and in the structures used for exporting the history data
to userland via sysctl(9). This avoids problems on some architectures
where passing a 64-bit (or larger) value to printf(3) can cause it to
process the value as multiple arguments. (This can be particularly
problematic when printf()'s format string is not a literal, since in
that case the compiler cannot know how large each argument should be.)
* Update the data structures used for exporting kernel history data to
include a version number as well as the length of history arguments.
* All [2] existing users of kernhist(9) have had their format strings
updated. Each format specifier now includes an explicit length
modifier 'j' to refer to numeric values of the size of uintmax_t.
* All [2] existing users of kernhist(9) have had their format strings
updated to replace uses of "%p" with "%#jx", and the pointer
arguments are now cast to (uintptr_t) before being subsequently cast
to (uintmax_t). This is needed to avoid compiler warnings about
casting "pointer to integer of a different size."
* All [2] existing users of kernhist(9) have had instances of "%s" or
"%c" format strings replaced with numeric formats; several instances
of mis-match between format string and argument list have been fixed.
* vmstat(1) has been modified to handle the new size of arguments in the
history data as exported by sysctl(9).
* vmstat(1) now provides a warning message if the history requested with
the -u option does not exist (previously, this condition was silently
ignored, with only a single blank line being printed).
* vmstat(1) now checks the version and argument length included in the
data exported via sysctl(9) and exits if they do not match the values
with which vmstat was built.
* The kernhist(9) man-page has been updated to note the additional
requirements imposed on the format strings, along with several other
minor changes and enhancements.
[1] It would have been possible to use an explicit length (for example,
uint64_t) for the history arguments. But that would require another
"rototill" of all the users in the future when we add support for an
architecture that supports a larger size. Also, the printf(3)
format
specifiers for explicitly-sized values, such as "%"PRIu64, are much
more verbose (and less aesthetically appealing, IMHO) than simply
using "%ju".
[2] I've tried very hard to find "all [the] existing users of
kernhist(9)"
but it is possible that I've missed some of them. I would be glad
to
update any stragglers that anyone identifies.
--
For some reason this single kernel seems to have outgrown its declared
size as a result of the kernhist(9) changes. Bump the size.
XXX The amount of increase may be excessive - anyone with more detailed
XXX knowledge please feel free to further adjust the value
appropriately.
--
Misssed one cast of pointer --> uintptr_t in previous kernhist(9) commit
--
And yet another one. :(
--
Use correct mark-up for NetBSD version.
--
More improvements in grammar and readability.
--
Remove a stray '"' (obvious typo) and add a couple of casts that are
probably needed.
--
And replace an instance of "%p" conversion with "%#jx"
--
Whitespace fix. Give Bl tag table a width. Fix Xr.
 1.64.8.1  06-Jul-2021  martin Pull up following revision(s) - all via patch -
(requested by riastradh in ticket #1317):

sys/uvm/uvm_page.c: revision 1.248
sys/uvm/uvm_anon.c: revision 1.80
sys/rump/librump/rumpvfs/vm_vfs.c: revision 1.40
sys/rump/librump/rumpvfs/vm_vfs.c: revision 1.41
sys/rump/librump/rumpkern/vm.c: revision 1.191
sys/uvm/uvm_pager.c: revision 1.130
external/cddl/osnet/dist/uts/common/fs/zfs/zfs_vnops.c: revision 1.71
tests/rump/rumpkern/t_vm.c: revision 1.5
tests/rump/rumpkern/t_vm.c: revision 1.6
sys/rump/librump/rumpvfs/vm_vfs.c: revision 1.39

Move the handling of PG_PAGEOUT from uvm_aio_aiodone_pages() to
uvm_page_unbusy() so that all callers of uvm_page_unbusy() don't need to
handle this flag separately. Split out the pages part of uvm_aio_aiodone()
into uvm_aio_aiodone_pages() in rump just like in the real kernel.

In ZFS functions that can fail to copy data between the ARC and VM pages,
use uvm_aio_aiodone_pages() rather than uvm_page_unbusy() so that we can
handle these "I/O" errors. Fixes PR 55702.

fix an incorrect assertion in the previous commit.

Handle PG_PAGEOUT in uvm_anon_release() too.

Commit the ZFS file that I forgot in this previous commit:

Move the handling of PG_PAGEOUT from uvm_aio_aiodone_pages() to
uvm_page_unbusy() so that all callers of uvm_page_unbusy() don't need to
handle this flag separately. Split out the pages part of uvm_aio_aiodone()
into uvm_aio_aiodone_pages() in rump just like in the real kernel.

In ZFS functions that can fail to copy data between the ARC and VM pages,
use uvm_aio_aiodone_pages() rather than uvm_page_unbusy() so that we can
handle these "I/O" errors. Fixes PR 55702.
update the rump copy of uvm_page_unbusy() to match the real version,
in particular handle PG_PAGEOUT. fixes a few atf tests.
the busypage test is buggy, expect it to fail.

make rump's uvm_aio_aiodone_pages() look more like the kernel version.
fixes some more rumpy assertions.

for the busypage test, replace atf_tc_expect_fail() with atf_tc_skip()
because atf apparently has no way to expect a test program to crash.
fixes PR 55945.
 1.64.4.1  08-Apr-2020  martin Merge changes from current as of 20200406
 1.70.2.2  29-Feb-2020  ad Sync with head.
 1.70.2.1  17-Jan-2020  ad Sync with head.

RSS XML Feed