Home | History | Annotate | Download | only in uvm
History log of /src/sys/uvm/uvm_aobj.c
RevisionDateAuthorComments
 1.157  24-Feb-2023  riastradh uvm: Eliminate __HAVE_ATOMIC_AS_MEMBAR conditionals.

Discussed on tech-kern:
https://mail-index.netbsd.org/tech-kern/2023/02/23/msg028729.html
 1.156  31-May-2022  andvar fix various typos in comments, documentation and messages.
 1.155  09-Apr-2022  riastradh sys: Use membar_release/acquire around reference drop.

This just goes through my recent reference count membar audit and
changes membar_exit to membar_release and membar_enter to
membar_acquire -- this should make everything cheaper on most CPUs
without hurting correctness, because membar_acquire is generally
cheaper than membar_enter.
 1.154  12-Mar-2022  riastradh sys: Membar audit around reference count releases.

If two threads are using an object that is freed when the reference
count goes to zero, we need to ensure that all memory operations
related to the object happen before freeing the object.

Using an atomic_dec_uint_nv(&refcnt) == 0 ensures that only one
thread takes responsibility for freeing, but it's not enough to
ensure that the other thread's memory operations happen before the
freeing.

Consider:

Thread A Thread B
obj->foo = 42; obj->baz = 73;
mumble(&obj->bar); grumble(&obj->quux);
/* membar_exit(); */ /* membar_exit(); */
atomic_dec -- not last atomic_dec -- last
/* membar_enter(); */
KASSERT(invariant(obj->foo,
obj->bar));
free_stuff(obj);

The memory barriers ensure that

obj->foo = 42;
mumble(&obj->bar);

in thread A happens before

KASSERT(invariant(obj->foo, obj->bar));
free_stuff(obj);

in thread B. Without them, this ordering is not guaranteed.

So in general it is necessary to do

membar_exit();
if (atomic_dec_uint_nv(&obj->refcnt) != 0)
return;
membar_enter();

to release a reference, for the `last one out hit the lights' style
of reference counting. (This is in contrast to the style where one
thread blocks new references and then waits under a lock for existing
ones to drain with a condvar -- no membar needed thanks to mutex(9).)

I searched for atomic_dec to find all these. Obviously we ought to
have a better abstraction for this because there's so much copypasta.
This is a stop-gap measure to fix actual bugs until we have that. It
would be nice if an abstraction could gracefully handle the different
styles of reference counting in use -- some years ago I drafted an
API for this, but making it cover everything got a little out of hand
(particularly with struct vnode::v_usecount) and I ended up setting
it aside to work on psref/localcount instead for better scalability.

I got bored of adding #ifdef __HAVE_ATOMIC_AS_MEMBAR everywhere, so I
only put it on things that look performance-critical on 5sec review.
We should really adopt membar_enter_preatomic/membar_exit_postatomic
or something (except they are applicable only to atomic r/m/w, not to
atomic_load/store_*, making the naming annoying) and get rid of all
the ifdefs.
 1.153  13-Mar-2021  skrll Consistently use %#jx instead of 0x%jx or just %jx in UVMHIST_LOG formats
 1.152  04-Nov-2020  chs In uvmpd_tryownerlock(), if the initial try-lock of the owner lock fails
then rather than do more try-locks and eventually sleep for a tick,
take a hold on the current owner's lock, drop the page interlock,
and acquire the lock that we took the hold on in a blocking fashion.
After we get the lock, check if the lock that we acquired is still
the lock for the owner of the page that we're interested in.
If the owner hasn't changed then can proceed with this page,
otherwise we will skip this page and move on to a different page.
This dramatically reduces the amount of time that the pagedaemon
sleeps trying to get locks, since even 1 tick is an eternity to sleep
in this context and it was easy to trigger that case in practice,
and with this new method the pagedaemon only very rarely actually blocks
to acquire the lock that it wants since the object locks are adaptive,
and when the pagedaemon does block then the amount of time it spends
sleeping will be generally be much less than 1 tick.
 1.151  19-Aug-2020  chs branches: 1.151.2;
in uao_get(), if we unlock the uobj to read a page from swap,
we must clear the cached page array because it is now stale.
also add a missing call to uvm_page_array_fini() if the I/O fails.
fixes PR 55493.
 1.150  19-Aug-2020  simonb Remove trailing \n from UVMHIST_LOG() format strings.
 1.149  09-Jul-2020  skrll Consistently use UVMHIST(__func__)

Convert UVMHIST_{CALLED,LOG} into UVMHIST_CALLARGS
 1.148  08-Jul-2020  skrll Trailing whitespace
 1.147  25-May-2020  ad uao_get(): in the PGO_SYNCIO case use uvm_page_array and simplify control
flow a little bit.
 1.146  25-May-2020  ad - Alter the convention for uvm_page_array slightly, so the basic search
parameters can't change part way through a search: move the "uobj" and
"flags" arguments over to uvm_page_array_init() and store those with the
array.

- With that, detect when it's not possible to find any more pages in the
tree with the given search parameters, and avoid repeated tree lookups if
the caller loops over uvm_page_array_fill_and_peek().
 1.145  25-May-2020  ad PR kern/55300: ubciomove triggers page not dirty assertion

If overwriting an existing page, mark it dirty since there may be no
managed mapping to track the modification.
 1.144  22-May-2020  ad uao_get(): handle PGO_OVERWRITE.
 1.143  20-May-2020  hannken Suppress GCC warnings and fix a UVMHIST_LOG() statement.

Kernels ALL/amd64 and ALL/i386 and port sparc64 build again.
 1.142  19-May-2020  ad PR kern/32166: pgo_get protocol is ambiguous
Also problems with tmpfs+nfs noted by hannken@.

Don't pass PGO_ALLPAGES to pgo_get, and ignore PGO_DONTCARE in the
!PGO_LOCKED case. In uao_get() have uvm_pagealloc() take care of page
zeroing and release busy pages on error.
 1.141  17-May-2020  ad Start trying to reduce cache misses on vm_page during fault processing.

- Make PGO_LOCKED getpages imply PGO_NOBUSY and remove the latter. Mark
pages busy only when there's actually I/O to do.

- When doing COW on a uvm_object, don't mess with neighbouring pages. In
all likelyhood they're already entered.

- Don't mess with neighbouring VAs that have existing mappings as replacing
those mappings with same can be quite costly.

- Don't enqueue pages for neighbour faults unless not enqueued already, and
don't activate centre pages unless uvmpdpol says its useful.

Also:

- Make PGO_LOCKED getpages on UAOs work more like vnodes: do gang lookup in
the radix tree, and don't allocate new pages.

- Fix many assertion failures around faults/loans with tmpfs.
 1.140  15-May-2020  ad PR kern/55268: tmpfs is slow

uao_get(): in the PGO_LOCKED case, we're okay to allocate a new page as long
as the caller holds a write lock. PGO_NOBUSY doesn't put a stop to that.
 1.139  22-Mar-2020  ad Process concurrent page faults on individual uvm_objects / vm_amaps in
parallel, where the relevant pages are already in-core. Proposed on
tech-kern.

Temporarily disabled on MP architectures with __HAVE_UNLOCKED_PMAP until
adjustments are made to their pmaps.
 1.138  17-Mar-2020  ad Tweak the March 14th change to make page waits interlocked by pg->interlock.
Remove unneeded changes and only deal with the PQ_WANTED flag, to exclude
possible bugs.
 1.137  14-Mar-2020  ad Make page waits (WANTED vs BUSY) interlocked by pg->interlock. Gets RW
locks out of the equation for sleep/wakeup, and allows observing+waiting
for busy pages when holding only a read lock. Proposed on tech-kern.
 1.136  24-Feb-2020  rin 0x%#x --> %#x for non-external codes.
Also, stop mixing up 0x%x and %#x in single files as far as possible.
 1.135  23-Feb-2020  ad UVM locking changes, proposed on tech-kern:

- Change the lock on uvm_object, vm_amap and vm_anon to be a RW lock.
- Break v_interlock and vmobjlock apart. v_interlock remains a mutex.
- Do partial PV list locking in the x86 pmap. Others to follow later.
 1.134  15-Jan-2020  ad Merge from yamt-pagecache (after much testing):

- Reduce unnecessary page scan in putpages esp. when an object has a ton of
pages cached but only a few of them are dirty.

- Reduce the number of pmap operations by tracking page dirtiness more
precisely in uvm layer.
 1.133  31-Dec-2019  ad branches: 1.133.2;
- Add and use wrapper functions that take and acquire page interlocks, and pairs
of page interlocks. Require that the page interlock be held over calls to
uvm_pageactivate(), uvm_pagewire() and similar.

- Solve the concurrency problem with page replacement state. Rather than
updating the global state synchronously, set an intended state on
individual pages (active, inactive, enqueued, dequeued) while holding the
page interlock. After the interlock is released put the pages on a 128
entry per-CPU queue for their state changes to be made real in batch.
This results in in a ~400 fold decrease in contention on my test system.
Proposed on tech-kern but modified to use the page interlock rather than
atomics to synchronise as it's much easier to maintain that way, and
cheaper.
 1.132  15-Dec-2019  ad Merge from yamt-pagecache:

- do gang lookup of pages using radixtree.
- remove now unused uvm_object::uo_memq and vm_page::listq.queue.
 1.131  13-Dec-2019  ad Break the global uvm_pageqlock into a per-page identity lock and a private
lock for use of the pagedaemon policy code. Discussed on tech-kern.

PR kern/54209: NetBSD 8 large memory performance extremely low
PR kern/54210: NetBSD-8 processes presumably not exiting
PR kern/54727: writing a large file causes unreasonable system behaviour
 1.130  01-Dec-2019  ad Avoid calling pmap_page_protect() while under uvm_pageqlock.
 1.129  01-Dec-2019  ad - Adjust uvmexp.swpgonly with atomics, and make uvm_swap_data_lock static.
- A bit more __cacheline_aligned on mutexes.
 1.128  28-Jul-2019  msaitoh Avoid undefined behavior in uao_pagein_page(). Found by kUBSan. OK'd by
riastradh. I think this is a real bug on amd64 at least.
 1.127  28-May-2018  chs branches: 1.127.2;
allow tmpfs files to be larger than 4GB.
 1.126  28-Oct-2017  pgoyette branches: 1.126.2;
Update the kernhist(9) kernel history code to address issues identified
in PR kern/52639, as well as some general cleaning-up...

(As proposed on tech-kern@ with additional changes and enhancements.)

Details of changes:

* All history arguments are now stored as uintmax_t values[1], both in
the kernel and in the structures used for exporting the history data
to userland via sysctl(9). This avoids problems on some architectures
where passing a 64-bit (or larger) value to printf(3) can cause it to
process the value as multiple arguments. (This can be particularly
problematic when printf()'s format string is not a literal, since in
that case the compiler cannot know how large each argument should be.)

* Update the data structures used for exporting kernel history data to
include a version number as well as the length of history arguments.

* All [2] existing users of kernhist(9) have had their format strings
updated. Each format specifier now includes an explicit length
modifier 'j' to refer to numeric values of the size of uintmax_t.

* All [2] existing users of kernhist(9) have had their format strings
updated to replace uses of "%p" with "%#jx", and the pointer
arguments are now cast to (uintptr_t) before being subsequently cast
to (uintmax_t). This is needed to avoid compiler warnings about
casting "pointer to integer of a different size."

* All [2] existing users of kernhist(9) have had instances of "%s" or
"%c" format strings replaced with numeric formats; several instances
of mis-match between format string and argument list have been fixed.

* vmstat(1) has been modified to handle the new size of arguments in the
history data as exported by sysctl(9).

* vmstat(1) now provides a warning message if the history requested with
the -u option does not exist (previously, this condition was silently
ignored, with only a single blank line being printed).

* vmstat(1) now checks the version and argument length included in the
data exported via sysctl(9) and exits if they do not match the values
with which vmstat was built.

* The kernhist(9) man-page has been updated to note the additional
requirements imposed on the format strings, along with several other
minor changes and enhancements.

[1] It would have been possible to use an explicit length (for example,
uint64_t) for the history arguments. But that would require another
"rototill" of all the users in the future when we add support for an
architecture that supports a larger size. Also, the printf(3) format
specifiers for explicitly-sized values, such as "%"PRIu64, are much
more verbose (and less aesthetically appealing, IMHO) than simply
using "%ju".

[2] I've tried very hard to find "all [the] existing users of kernhist(9)"
but it is possible that I've missed some of them. I would be glad to
update any stragglers that anyone identifies.
 1.125  30-May-2017  chs branches: 1.125.2;
add assertions that would have caught the recent audio mmap bugs.
 1.124  28-Jul-2016  martin PR kern/51371: fix misleading indentation
 1.123  24-Aug-2015  pooka branches: 1.123.2;
to garnish, dust with _KERNEL_OPT
 1.122  25-May-2014  riastradh branches: 1.122.4;
Allow VM_NFREELIST in uao_set_pgfl, meaning any freelist is OK.
 1.121  22-May-2014  riastradh Add uao_set_pgfl to limit a uvm_aobj's pages to a specified freelist.

Brought up on tech-kern:

https://mail-index.netbsd.org/tech-kern/2014/05/20/msg017095.html
 1.120  25-Oct-2013  martin branches: 1.120.2;
Mark a diagnostic-only variable
 1.119  15-Sep-2012  matt branches: 1.119.2;
#include <sys/atomic.h>
 1.118  14-Sep-2012  rmind - Manage anonymous UVM object reference count with atomic ops.
- Fix an old bug of possible lock against oneself (uao_detach_locked() is
called from uao_swap_off() with uao_list_lock acquired). Also removes
the try-lock dance in uao_swap_off(), since the lock order changes.
 1.117  14-Sep-2012  rmind - Describe uvm_aobj and the lock order.
- Remove unnecessary uao_dropswap_range1() wrapper.
- KNF. Sprinkle some __cacheline_aligned.
 1.116  06-Sep-2011  matt branches: 1.116.2; 1.116.8; 1.116.12;
Allocate color appropriate pages.
 1.115  12-Jun-2011  rmind Welcome to 5.99.53! Merge rmind-uvmplock branch:

- Reorganize locking in UVM and provide extra serialisation for pmap(9).
New lock order: [vmpage-owner-lock] -> pmap-lock.

- Simplify locking in some pmap(9) modules by removing P->V locking.

- Use lock object on vmobjlock (and thus vnode_t::v_interlock) to share
the locks amongst UVM objects where necessary (tmpfs, layerfs, unionfs).

- Rewrite and optimise x86 TLB shootdown code, make it simpler and cleaner.
Add TLBSTATS option for x86 to collect statistics about TLB shootdowns.

- Unify /dev/mem et al in MI code and provide required locking (removes
kernel-lock on some ports). Also, avoid cache-aliasing issues.

Thanks to Andrew Doran and Joerg Sonnenberger, as their initial patches
formed the core changes of this branch.
 1.114  23-Apr-2011  rmind branches: 1.114.2;
Replace "malloc" in comments, remove unnecessary header inclusions.
 1.113  11-Feb-2011  rmind Replace uvm_aobj_cache with kmem(9).
 1.112  02-Feb-2011  chuck udpate license clauses on chuck^2 code to match the new-style BSD licenses.
based on diff that rmind@ sent me (and confirmed with chs@ via email).

no functional change with this commit.
 1.111  25-Jan-2011  enami Remove nop code; the code is moved to uao_dropswap_range1() when it is
introduced in rev. 1.75.
 1.110  29-Jul-2010  hannken branches: 1.110.2; 1.110.4;
Add vm page flag PG_MARKER and use it to tag dummy marker pages
in genfs_do_putpages() and uao_put().
Use 'v_uobj.uo_npages' to check for an empty memq.
Put some assertions where these marker pages may not appear.

Ok: YAMAMOTO Takashi <yamt@netbsd.org>
 1.109  28-May-2010  rmind uvm_fault_{upper,lower}_done: move drop-swap outside the page-queues lock.
Assert for object lock being held (or ref count 0) in uao_set_swslot().
 1.108  21-Oct-2009  rmind branches: 1.108.2; 1.108.4;
Remove uarea swap-out functionality:

- Addresses the issue described in PR/38828.
- Some simplification in threading and sleepq subsystems.
- Eliminates pmap_collect() and, as a side note, allows pmap optimisations.
- Eliminates XS_CTL_DATA_ONSTACK in scsipi code.
- Avoids few scans on LWP list and thus potentially long holds of proc_lock.
- Cuts ~1.5k lines of code. Reduces amd64 kernel size by ~4k.
- Removes __SWAP_BROKEN cases.

Tested on x86, mips, acorn32 (thanks <mpumford>) and partly tested on
acorn26 (thanks to <bjh21>).

Discussed on <tech-kern>, reviewed by <ad>.
 1.107  13-Sep-2009  pooka Wipe out the last vestiges of POOL_INIT with one swift stroke. In
most cases, use a proper constructor. For proplib, give a local
equivalent of POOL_INIT for the kernel object implementation. This
way the code structure can be preserved, and a local link set is
not hazardous anyway (unless proplib is split to several modules,
but that'll be the day).

tested by booting a kernel in qemu and compile-testing i386/ALL
 1.106  18-Feb-2009  yamt make some functions static.
 1.105  16-Jan-2009  yamt branches: 1.105.2;
- g/c stale function prototypes.
- rename UVM_PAGE_HASH_PENALTY to UVM_PAGE_TREE_PENALTY.
 1.104  18-Oct-2008  rmind branches: 1.104.2; 1.104.10;
- Initialize pool subsystem and kmem(9) earlier, when UVM is up enough.
- Remove uao_hashinit() workaround used for anon-objects.
- Replace malloc with kmem.

OK by <yamt>.
 1.103  25-Jun-2008  ad branches: 1.103.2;
Use pool_cache.
 1.102  04-Jun-2008  ad branches: 1.102.2;
vm_page: put TAILQ_ENTRY into a union with LIST_ENTRY, so we can use both.
 1.101  03-Jun-2008  ad uao_reference, uao_detach: we don't do reference counting on kernel objects,
so don't lock them needlessly.
 1.100  05-May-2008  ad branches: 1.100.2;
- Convert hashinit() to use kmem_alloc(). The hash tables can be large
and it's better to not have them in kmem_map.
- Convert a couple of minor items along the way to kmem_alloc().
- Fix some memory leaks.
 1.99  27-Feb-2008  ad branches: 1.99.2; 1.99.4;
Minor corrections to comments.
 1.98  27-Feb-2008  yamt uao_put: fix a race with pageout.
 1.97  18-Jan-2008  yamt branches: 1.97.2; 1.97.6;
push pmap_clear_reference calls into pdpolicy code, where reference bits
actually matter.
 1.96  02-Jan-2008  ad Merge vmlocking2 to head.
 1.95  01-Dec-2007  yamt branches: 1.95.2; 1.95.6;
constify pagerops.
 1.94  01-Dec-2007  yamt use designated initiaizers for uvm_pagerops.
 1.93  05-Aug-2007  pooka branches: 1.93.2; 1.93.8; 1.93.10;
In uao_get(), drop object lock only after dropswap to avoid KASSERT panic.

Should fix tmpfs problem reported by riz on current-users. yamt ok.
 1.92  24-Jul-2007  ad branches: 1.92.4;
In order to pacify assertions, make uao_list_lock + uvm_swap_data_lock
spinlocks for the time being.
 1.91  21-Jul-2007  ad Temporarily work around an assertion from mutex_enter.
 1.90  21-Jul-2007  ad Merge unobtrusive locking changes from the vmlocking branch.
 1.89  09-Jul-2007  ad branches: 1.89.2;
Merge some of the less invasive changes from the vmlocking branch:

- kthread, callout, devsw API changes
- select()/poll() improvements
- miscellaneous MT safety improvements
 1.88  12-Mar-2007  ad branches: 1.88.2;
Pass an ipl argument to pool_init/POOL_INIT to be used when initializing
the pool's lock.
 1.87  22-Feb-2007  thorpej branches: 1.87.4;
TRUE -> true, FALSE -> false
 1.86  22-Feb-2007  matt Fix lossage from boolean_t -> bool and updated x86 bus_dma.
 1.85  21-Feb-2007  thorpej Replace the Mach-derived boolean_t type with the C99 bool type. A
future commit will replace use of TRUE and FALSE with true and false.
 1.84  24-Jan-2007  hubertf branches: 1.84.2;
Remove duplicate #includes, patch contributed in private mail
by Slava Semushin <slava.semushin@gmail.com>.

To verify that no nasty side effects of duplicate includes (or their
removal) have an effect here, I've compiled an i386/ALL kernel with
and without the patch, and the only difference in the resulting .o
files was in shifted line numbers in some assert() calls.
The comparison of the .o files was based on the output of "objdump -D".

Thanks to martin@ for the input on testing.
 1.83  15-Dec-2006  yamt put ->K loaned pages on the page queue, so that page loaning doesn't
disturb pagedaemon/pdpolicy.
 1.82  01-Nov-2006  yamt branches: 1.82.2; 1.82.4;
remove some __unused from function parameters.
 1.81  12-Oct-2006  christos - sprinkle __unused on function decls.
- fix a couple of unused bugs
- no more -Wno-unused for i386
 1.80  15-Sep-2006  yamt branches: 1.80.2;
merge yamt-pdpolicy branch.
- separate page replacement policy from the rest of kernel
- implement an alternative replacement policy
 1.79  01-Sep-2006  cherry branches: 1.79.2;
bumps kernel aobj to 64 bit. \
See: http://mail-index.netbsd.org/tech-kern/2006/03/07/0007.html
 1.78  24-Dec-2005  yamt branches: 1.78.4; 1.78.8;
uao_get: don't mark pages dirty unless it's a write fault.
 1.77  05-Dec-2005  yamt uao_pagein_page: pass PGO_SYNCIO to uao_get.
uao_get doesn't always assume PGO_SYNCIO after yamt-readahead merge.

reported and a dump provided by Masanori Kanaoka.
 1.76  29-Nov-2005  yamt merge yamt-readahead branch.
 1.75  08-Nov-2005  yamt branches: 1.75.2;
add a function to drop all swap slots in a given range. for tmpfs.
XXX maybe it's better to implement true truncation.
 1.74  17-Sep-2005  yamt make VMSWAP optional again.
 1.73  14-Sep-2005  yamt uao_put: don't skip loaned or wired pages.
 1.72  13-Sep-2005  yamt wrap swap related code by #ifdef VMSWAP. always #define VMSWAP for now.
 1.71  13-Sep-2005  yamt uao_put: recognize endoff == 0 as "to the end of the object",
as VOP_PUTPAGES (thus vnode pager) does. for tmpfs.
 1.70  31-Jul-2005  yamt revert "defflag VMSWAP" changes for now.
there seems to be far more people who don't want to edit
their kernel config files than i thought.
 1.69  30-Jul-2005  yamt defflag VMSWAP.
 1.68  27-Jun-2005  thorpej branches: 1.68.2;
Sprinkle some static.
 1.67  27-Jun-2005  thorpej Use ANSI function decls.
 1.66  06-Jun-2005  yamt introduce a macro to initialize uvm_object and use it.
 1.65  29-May-2005  christos avoid shadow variables.
remove unneeded casts.
 1.64  25-Apr-2004  simonb Initialise (most) pools from a link set instead of explicit calls
to pool_init. Untouched pools are ones that either in arch-specific
code, or aren't initialiased during initial system startup.

Convert struct session, ucred and lockf to pools.
 1.63  05-Apr-2004  simonb Fix a tyop.
 1.62  24-Mar-2004  junyoung Nuke __P().
 1.61  18-Sep-2003  drochner Fix a reversed logic in swap deallocation which could lead to
uvm_swap_free() being called with a zero slot; this might have been
the reason for crashes with sysvshm and heavy swapping.
(PR kern/22752 by Tom Spindler)
Confirmed by Chuck Silvers.
 1.60  28-Aug-2003  pk When retiring a swap device with marked bad blocks on it we should update
the `# swap page in use' and `# swap page only' counters. However, at the
time of swap device removal we can no longer figure out how many of the
bad swap pages are actually also `swap only' pages.

So, on swap I/O errors arrange things to not include the bad swap pages in
the `swpgonly' counter as follows: uvm_swap_markbad() decrements `swpgonly'
by the number of bad pages, and the various VM object deallocation routines
do not decrement `swpgonly' for swap slots marked as SWSLOT_BAD.
 1.59  11-Aug-2003  pk uao_pagein_page() & anon_pagein():
* return failure if the page cannot be retrieved.
* wakeup any waiters when releasing a page after successful page in.
 1.58  11-Aug-2003  pk Only deactivate pages if their wired count is zero.
 1.57  11-Aug-2003  pk Make sure to call uvm_swap_free() and uvm_swap_markbad() with valid (i.e.
positive) slot numbers.
 1.56  12-Apr-2003  yamt branches: 1.56.2;
unbusy a page after put it on the queue.
fix a panic with UVM_PAGE_TRKOWN when doing swapoff.
 1.55  09-Feb-2003  pk uao_put: release uvm object's lock only after we're done with its page list.
 1.54  01-Feb-2003  thorpej Add extensible malloc types, adapted from FreeBSD. This turns
malloc types into a structure, a pointer to which is passed around,
instead of an int constant. Allow the limit to be adjusted when the
malloc type is defined, or with a function call, as suggested by
Jonathan Stone.
 1.53  18-Jan-2003  thorpej Merge the nathanw_sa branch.
 1.52  24-Nov-2002  scw Quell uninitialised variable warnings.
 1.51  09-May-2002  enami In uao_put(), if we wait for the busy page owned by someone else,
we can't simply reuse the pointor to the page. Instead, we need to
acquire it again. So, rearrange the loop like genfs_putpages() does.
Reviewed by chuq.
 1.50  08-Mar-2002  thorpej branches: 1.50.2;
Pool deals fairly well with physical memory shortage, but it doesn't
deal with shortages of the VM maps where the backing pages are mapped
(usually kmem_map). Try to deal with this:

* Group all information about the backend allocator for a pool in a
separate structure. The pool references this structure, rather than
the individual fields.
* Change the pool_init() API accordingly, and adjust all callers.
* Link all pools using the same backend allocator on a list.
* The backend allocator is responsible for waiting for physical memory
to become available, but will still fail if it cannot callocate KVA
space for the pages. If this happens, carefully drain all pools using
the same backend allocator, so that some KVA space can be freed.
* Change pool_reclaim() to indicate if it actually succeeded in freeing
some pages, and use that information to make draining easier and more
efficient.
* Get rid of PR_URGENT. There was only one use of it, and it could be
dealt with by the caller.

From art@openbsd.org.
 1.49  10-Nov-2001  lukem add RCSIDs, and in some cases, slightly cleanup #include order
 1.48  07-Nov-2001  chs only acquire the lock for swpgonly if we actually need to adjust it.
 1.47  06-Nov-2001  chs several changes prompted by loaning problems:
- fix the loaned case in uvm_pagefree().
- redo uvmexp.swpgonly accounting to work with page loaning.
add an assertion before each place we adjust uvmexp.swpgonly.
- fix uvm_km_pgremove() to always free any swap space associated with
the range being removed.
- get rid of UVM_LOAN_WIRED flag. instead, we just make sure that
pages loaned to the kernel are never on the page queues.
this allows us to assert that pages are not loaned and wired
at the same time.
- add yet more assertions.
 1.46  15-Sep-2001  chs branches: 1.46.2;
a whole bunch of changes to improve performance and robustness under load:

- remove special treatment of pager_map mappings in pmaps. this is
required now, since I've removed the globals that expose the address range.
pager_map now uses pmap_kenter_pa() instead of pmap_enter(), so there's
no longer any need to special-case it.
- eliminate struct uvm_vnode by moving its fields into struct vnode.
- rewrite the pageout path. the pager is now responsible for handling the
high-level requests instead of only getting control after a bunch of work
has already been done on its behalf. this will allow us to UBCify LFS,
which needs tighter control over its pages than other filesystems do.
writing a page to disk no longer requires making it read-only, which
allows us to write wired pages without causing all kinds of havoc.
- use a new PG_PAGEOUT flag to indicate that a page should be freed
on behalf of the pagedaemon when it's unlocked. this flag is very similar
to PG_RELEASED, but unlike PG_RELEASED, PG_PAGEOUT can be cleared if the
pageout fails due to eg. an indirect-block buffer being locked.
this allows us to remove the "version" field from struct vm_page,
and together with shrinking "loan_count" from 32 bits to 16,
struct vm_page is now 4 bytes smaller.
- no longer use PG_RELEASED for swap-backed pages. if the page is busy
because it's being paged out, we can't release the swap slot to be
reallocated until that write is complete, but unlike with vnodes we
don't keep a count of in-progress writes so there's no good way to
know when the write is done. instead, when we need to free a busy
swap-backed page, just sleep until we can get it busy ourselves.
- implement a fast-path for extending writes which allows us to avoid
zeroing new pages. this substantially reduces cpu usage.
- encapsulate the data used by the genfs code in a struct genfs_node,
which must be the first element of the filesystem-specific vnode data
for filesystems which use genfs_{get,put}pages().
- eliminate many of the UVM pagerops, since they aren't needed anymore
now that the pager "put" operation is a higher-level operation.
- enhance the genfs code to allow NFS to use the genfs_{get,put}pages
instead of a modified copy.
- clean up struct vnode by removing all the fields that used to be used by
the vfs_cluster.c code (which we don't use anymore with UBC).
- remove kmem_object and mb_object since they were useless.
instead of allocating pages to these objects, we now just allocate
pages with no object. such pages are mapped in the kernel until they
are freed, so we can use the mapping to find the page to free it.
this allows us to remove splvm() protection in several places.

The sum of all these changes improves write throughput on my
decstation 5000/200 to within 1% of the rate of NetBSD 1.5
and reduces the elapsed time for "make release" of a NetBSD 1.5
source tree on my 128MB pc to 10% less than a 1.5 kernel took.
 1.45  23-Jun-2001  chs branches: 1.45.2; 1.45.4;
don't for memory in uao_set_swlot() since we're holding spinlocks,
instead return -1. adjust callers to handle this new error return.
fixes PR 13194.
 1.44  22-Jun-2001  chs don't use the list pointers after we take an object off its list.
 1.43  26-May-2001  chs replace vm_page_t with struct vm_page *.
 1.42  26-May-2001  chs replace {simple_,}lock{_data,}_t with struct {simple,}lock {,*}.
 1.41  25-May-2001  chs remove trailing whitespace.
 1.40  10-Mar-2001  chs eliminate the VM_PAGER_* error codes in favor of the traditional E* codes.
the mapping is:

VM_PAGER_OK 0
VM_PAGER_BAD <unused>
VM_PAGER_FAIL <unused>
VM_PAGER_PEND 0 (see below)
VM_PAGER_ERROR EIO
VM_PAGER_AGAIN EAGAIN
VM_PAGER_UNLOCK EBUSY
VM_PAGER_REFAULT ERESTART

for async i/o requests, it used to be possible for the request to
be convert to sync, and the pager would return VM_PAGER_OK or VM_PAGER_PEND
to indicate whether the caller should perform post-i/o cleanup.
this is no longer allowed; pagers must now return 0 to indicate that
the async i/o was successfully started, and the caller never needs to
worry about doing the post-i/o cleanup.
 1.39  18-Feb-2001  chs branches: 1.39.2;
clean up DIAGNOSTIC checks, use KASSERT().
 1.38  28-Jan-2001  thorpej Page scanner improvements, behavior is actually a bit more like
Mach VM's now. Specific changes:
- Pages now need not have all of their mappings removed before being
put on the inactive list. They only need to have the "referenced"
attribute cleared. This makes putting pages onto the inactive list
much more efficient. In order to eliminate redundant clearings of
"refrenced", callers of uvm_pagedeactivate() must now do this
themselves.
- When checking the "modified" attribute for a page (for clearing
PG_CLEAN), make sure to only do it if PG_CLEAN is currently set on
the page (saves a potentially expensive pmap operation).
- When scanning the inactive list, if a page is referenced, reactivate
it (this part was actually added in uvm_pdaemon.c,v 1.27). This
now works properly now that pages on the inactive list are allowed to
have mappings.
- When scanning the inactive list and considering a page for freeing,
remove all mappings, and then check the "modified" attribute if the
page is marked PG_CLEAN.
- When scanning the active list, if the page was referenced since its
last sweep by the scanner, don't deactivate it. (This part was
actually added in uvm_pdaemon.c,v 1.28.)

These changes greatly improve interactive performance during
moderate to high memory and I/O load.
 1.37  25-Nov-2000  chs lots of cleanup:
use queue.h macros and KASSERT().
address amap offsets in pages instead of bytes.
make amap_ref() and amap_unref() take an amap, offset and length
instead of a vm_map_entry_t.
improve whitespace and comments.
 1.36  24-Nov-2000  chs g/c unused pager ops "asyncget" and "aiodone".
 1.35  08-Nov-2000  ad Update for hashinit() change.
 1.34  02-Aug-2000  thorpej MALLOC()/FREE() are not to be used for variable-sized allocations.
 1.33  27-Jun-2000  mrg remove include of <vm/vm.h>
 1.32  26-Jun-2000  mrg remove/move more mach vm header files:

<vm/pglist.h> -> <uvm/uvm_pglist.h>
<vm/vm_inherit.h> -> <uvm/uvm_inherit.h>
<vm/vm_kern.h> -> into <uvm/uvm_extern.h>
<vm/vm_object.h> -> nothing
<vm/vm_pager.h> -> into <uvm/uvm_pager.h>

also includes a bunch of <vm/vm_page.h> include removals (due to redudancy
with <vm/vm.h>), and a scattering of other similar headers.
 1.31  19-May-2000  thorpej NULL != 0
 1.30  10-Apr-2000  thorpej Use UVM_PGA_ZERO in a few (easy) places.
 1.29  03-Apr-2000  chs remove the "shareprot" pagerop. it's not needed anymore since
share maps are long gone.
 1.28  26-Mar-2000  kleink Merge parts of chs-ubc2 into the trunk:
Add a new type voff_t (defined as a synonym for off_t) to describe offsets
into uvm objects, and update the appropriate interfaces to use it, the
most visible effect being the ability to mmap() file offsets beyond
the range of a vaddr_t.

Originally by Chuck Silvers; blame me for problems caused by merging this
into non-UBC.
 1.27  11-Jan-2000  chs add support for ``swapctl -d'' (removing swap space).
improve handling of i/o errors in swap space.

reviewed by: Chuck Cranor
 1.26  12-Sep-1999  chs branches: 1.26.2;
eliminate the PMAP_NEW option by making it required for all ports.
ports which previously had no support for PMAP_NEW now implement
the pmap_k* interfaces as wrappers around the non-k versions.
 1.25  21-Aug-1999  thorpej When handling the MADV_FREE case, if the amap or aobj has more than
one reference, go through the deactivate path; the page may actually
be in use by another process.

Fixes kern/8239.
 1.24  22-Jul-1999  thorpej Garbage collect thread_sleep()/thread_wakeup() left over from the old
Mach VM code. Also nuke iprintf(), which was no longer used anywhere.

Add proclist locking where appropriate.
 1.23  22-Jul-1999  thorpej 0 -> FALSE in a few places.
 1.22  17-Jul-1999  thorpej Implement uao_flush(). This is pretty much identical to the "amap flush"
code in uvm_map_clean().
 1.21  07-Jul-1999  thorpej Update a comment in uao_flush().
 1.20  25-May-1999  thorpej Macro'ize the test for "object is a kernel object".
 1.19  11-Apr-1999  chs add a `flags' argument to uvm_pagealloc_strat().
define a flag UVM_PGA_USERESERVE to allow non-kernel object
allocations to use pages from the reserve.
use the new flag for allocations in pmap modules.
 1.18  26-Mar-1999  chs branches: 1.18.2;
add uvmexp.swpgonly and use it to detect out-of-swap conditions.
 1.17  25-Mar-1999  mrg remove now >1 year old pre-release message.
 1.16  24-Mar-1999  cgd after discussion with chuck, nuke pgo_attach from uvm_pagerops
 1.15  18-Oct-1998  chs branches: 1.15.2;
shift by PAGE_SHIFT instead of multiplying or dividing by PAGE_SIZE.
 1.14  18-Sep-1998  thorpej Add a comment documenting the last change.
 1.13  18-Sep-1998  thorpej Don't use the nointr pool page allocator for the uao_swhash_elt pool. We
need to ensure that these come from a non-pageable kernel map, otherwise
we can run into a deadlock condition (as noticed by Chuck Silvers).
 1.12  31-Aug-1998  thorpej Use the pool allocator w/ the "nointr" pool page allocator for uvm_aobj
and uao_swhash_elt structures. Also, fix a bug in uao_set_swlot() where
if setting the swslot to 0 (freeing swap resources), and no swslot was
currently allocated, a new entry would be allocated anyhow (revealed during
pool'ification).
 1.11  13-Aug-1998  drochner minor consistency nit: the page index into an anon object is always
assigned to from integer types, and it is compared to integers. So
let it be an integer instead of vsize_t.
 1.10  13-Aug-1998  eeh Merge paddr_t changes into the main branch.
 1.9  09-Aug-1998  perry bzero->memset, bcopy->memcpy, bcmp->memcmp
 1.8  01-Mar-1998  fvdl branches: 1.8.2;
Merge with Lite2 + local changes
 1.7  12-Feb-1998  chs add copyright.
 1.6  10-Feb-1998  mrg - add defopt's for UVM, UVMHIST and PMAP_NEW.
- remove unnecessary UVMHIST_DECL's.
 1.5  09-Feb-1998  mrg KNF.
 1.4  07-Feb-1998  mrg restore rcsids
 1.3  07-Feb-1998  chs enable hashtables for swapslot storage - deadlock is fixed.
fix initialization of swhash entries.
use malloc(M_NOWAIT) for creating kernel object.
avoid dereferencing a vm_page once the page has been freed.
 1.2  06-Feb-1998  thorpej RCS ID police.
 1.1  05-Feb-1998  mrg branches: 1.1.1;
Initial revision
 1.1.1.1  05-Feb-1998  mrg initial import of the new virtual memory system, UVM, into -current.

UVM was written by chuck cranor <chuck@maria.wustl.edu>, with some
minor portions derived from the old Mach code. i provided some help
getting swap and paging working, and other bug fixes/ideas. chuck
silvers <chuq@chuq.com> also provided some other fixes.

this is the UVM kernel code portion.


this will be KNF'd shortly. :-)
 1.8.2.1  30-Jul-1998  eeh Split vm_offset_t and vm_size_t into paddr_t, psize_t, vaddr_t, and vsize_t.
 1.15.2.2  25-Feb-1999  chs in uao_pagein_page() move pmap ops before clearing of page busy bit.
thread_wakeup() -> wakeup().
 1.15.2.1  09-Nov-1998  chs initial snapshot. lots left to do.
 1.18.2.1  16-Apr-1999  chs branches: 1.18.2.1.2;
pull up 1.18 -> 1.19:
add a `flags' argument to uvm_pagealloc_strat().
define a flag UVM_PGA_USERESERVE to allow non-kernel object
allocations to use pages from the reserve.
use the new flag for allocations in pmap modules.
 1.18.2.1.2.5  09-Aug-1999  chs create a new type "voff_t" for uvm_object offsets
and define it to be "off_t". also, remove pgo_asyncget().
 1.18.2.1.2.4  02-Aug-1999  thorpej Update from trunk.
 1.18.2.1.2.3  21-Jun-1999  thorpej Fix a merge error.
 1.18.2.1.2.2  21-Jun-1999  thorpej Sync w/ -current.
 1.18.2.1.2.1  07-Jun-1999  chs merge everything from chs-ubc branch.
 1.26.2.5  12-Mar-2001  bouyer Sync with HEAD.
 1.26.2.4  11-Feb-2001  bouyer Sync with HEAD.
 1.26.2.3  08-Dec-2000  bouyer Sync with HEAD.
 1.26.2.2  22-Nov-2000  bouyer Sync with HEAD.
 1.26.2.1  20-Nov-2000  bouyer Update thorpej_scsipi to -current as of a month ago
 1.39.2.9  11-Dec-2002  thorpej Sync with HEAD.
 1.39.2.8  24-Jun-2002  nathanw Curproc->curlwp renaming.

Change uses of "curproc->l_proc" back to "curproc", which is more like the
original use. Bare uses of "curproc" are now "curlwp".

"curproc" is now #defined in proc.h as ((curlwp) ? (curlwp)->l_proc) : NULL)
so that it is always safe to reference curproc (*de*referencing curproc
is another story, but that's always been true).
 1.39.2.7  20-Jun-2002  nathanw Catch up to -current.
 1.39.2.6  01-Apr-2002  nathanw Catch up to -current.
(CVS: It's not just a program. It's an adventure!)
 1.39.2.5  14-Nov-2001  nathanw Catch up to -current.
 1.39.2.4  21-Sep-2001  nathanw Catch up to -current.
 1.39.2.3  24-Aug-2001  nathanw Catch up with -current.
 1.39.2.2  21-Jun-2001  nathanw Catch up to -current.
 1.39.2.1  09-Apr-2001  nathanw Catch up with -current.
 1.45.4.1  01-Oct-2001  fvdl Catch up with -current.
 1.45.2.3  23-Jun-2002  jdolecek catch up with -current on kqueue branch
 1.45.2.2  16-Mar-2002  jdolecek Catch up with -current.
 1.45.2.1  10-Jan-2002  thorpej Sync kqueue branch with -current.
 1.46.2.1  12-Nov-2001  thorpej Sync the thorpej-mips-cache branch with -current.
 1.50.2.1  11-Mar-2002  thorpej Convert swap_syscall_lock and uvm.swap_data_lock to adaptive mutexes,
and rename them apporpriately.
 1.56.2.5  11-Dec-2005  christos Sync with head.
 1.56.2.4  10-Nov-2005  skrll Sync with HEAD. Here we go again...
 1.56.2.3  21-Sep-2004  skrll Fix the sync with head I botched.
 1.56.2.2  18-Sep-2004  skrll Sync with HEAD.
 1.56.2.1  03-Aug-2004  skrll Sync with HEAD
 1.68.2.7  17-Mar-2008  yamt sync with head.
 1.68.2.6  21-Jan-2008  yamt sync with head
 1.68.2.5  07-Dec-2007  yamt sync with head
 1.68.2.4  03-Sep-2007  yamt sync with head.
 1.68.2.3  26-Feb-2007  yamt sync with head.
 1.68.2.2  30-Dec-2006  yamt sync with head.
 1.68.2.1  21-Jun-2006  yamt sync with head.
 1.75.2.1  26-Nov-2005  yamt add minimum support of async get. ie. ignore them.
 1.78.8.2  03-Sep-2006  yamt sync with head.
 1.78.8.1  05-Mar-2006  yamt separate page replacement policy from the rest of kernel.
 1.78.4.1  09-Sep-2006  rpaulo sync with head
 1.79.2.3  01-Feb-2007  ad Sync with head.
 1.79.2.2  12-Jan-2007  ad Sync with head.
 1.79.2.1  18-Nov-2006  ad Sync with head.
 1.80.2.3  18-Dec-2006  yamt sync with head.
 1.80.2.2  10-Dec-2006  yamt sync with head.
 1.80.2.1  22-Oct-2006  yamt sync with head
 1.82.4.1  03-Sep-2007  wrstuden Sync w/ NetBSD-4-RC_1
 1.82.2.1  24-Aug-2007  liamjfoy Pull up following revision(s) (requested by pooka in ticket #825):
sys/uvm/uvm_aobj.c: revision 1.93
In uao_get(), drop object lock only after dropswap to avoid KASSERT panic.
Should fix tmpfs problem reported by riz on current-users. yamt ok.
 1.84.2.2  24-Mar-2007  yamt sync with head.
 1.84.2.1  27-Feb-2007  yamt - sync with head.
- move sched_changepri back to kern_synch.c as it doesn't know PPQ anymore.
 1.87.4.7  01-Nov-2007  ad Yielding to avoid livelock doesn't work well, so just sleep for 1 tick.
This too is inadequate and a better solution must be found. Discussed
with yamt@.
 1.87.4.6  21-Aug-2007  yamt destroy vmobjlock.
 1.87.4.5  20-Aug-2007  ad Sync with HEAD.
 1.87.4.4  03-Jul-2007  yamt if wrong-order trylocking failed, avoid livelock by yielding cpu
before retrying. ok'ed by Andrew Doran.
 1.87.4.3  05-Apr-2007  ad - Put a per-LWP lock around swapin / swapout.
- Replace use of lockmgr().
- Minor locking fixes and assertions.
- uvm_map.h no longer pulls in proc.h, etc.
- Use kpause where appropriate.
 1.87.4.2  13-Mar-2007  ad Pull in the initial set of changes for the vmlocking branch.
 1.87.4.1  13-Mar-2007  ad Sync with head.
 1.88.2.1  11-Jul-2007  mjf Sync with head.
 1.89.2.1  15-Aug-2007  skrll Sync with HEAD.
 1.92.4.2  03-Dec-2007  joerg Sync with HEAD.
 1.92.4.1  09-Aug-2007  jmcneill Sync with HEAD.
 1.93.10.2  05-Aug-2007  pooka In uao_get(), drop object lock only after dropswap to avoid KASSERT panic.

Should fix tmpfs problem reported by riz on current-users. yamt ok.
 1.93.10.1  05-Aug-2007  pooka file uvm_aobj.c was added on branch matt-mips64 on 2007-08-05 10:19:24 +0000
 1.93.8.2  18-Feb-2008  mjf Sync with HEAD.
 1.93.8.1  08-Dec-2007  mjf Sync with HEAD.
 1.93.2.2  23-Mar-2008  matt sync with HEAD
 1.93.2.1  09-Jan-2008  matt sync with HEAD
 1.95.6.2  19-Jan-2008  bouyer Sync with HEAD
 1.95.6.1  02-Jan-2008  bouyer Sync with HEAD
 1.95.2.1  04-Dec-2007  ad Pull the vmlocking changes into a new branch.
 1.97.6.5  17-Jan-2009  mjf Sync with HEAD.
 1.97.6.4  29-Jun-2008  mjf Sync with HEAD.
 1.97.6.3  05-Jun-2008  mjf Sync with HEAD.

Also fix build.
 1.97.6.2  02-Jun-2008  mjf Sync with HEAD.
 1.97.6.1  03-Apr-2008  mjf Sync with HEAD.
 1.97.2.1  24-Mar-2008  keiichi sync with head.
 1.99.4.5  11-Aug-2010  yamt sync with head.
 1.99.4.4  11-Mar-2010  yamt sync with head
 1.99.4.3  16-Sep-2009  yamt sync with head
 1.99.4.2  04-May-2009  yamt sync with head.
 1.99.4.1  16-May-2008  yamt sync with head.
 1.99.2.3  17-Jun-2008  yamt sync with head.
 1.99.2.2  04-Jun-2008  yamt sync with head
 1.99.2.1  18-May-2008  yamt sync with head.
 1.100.2.2  18-Sep-2008  wrstuden Sync with wrstuden-revivesa-base-2.
 1.100.2.1  23-Jun-2008  wrstuden Sync w/ -current. 34 merge conflicts to follow.
 1.102.2.1  27-Jun-2008  simonb Sync with head.
 1.103.2.1  19-Oct-2008  haad Sync with HEAD.
 1.104.10.3  29-Feb-2012  matt Improve UVM_PAGE_TRKOWN.
Add more asserts to uvm_page.
 1.104.10.2  03-Jun-2011  matt Restore $NetBSD$
 1.104.10.1  25-May-2011  matt Make uvm_map recognize UVM_FLAG_COLORMATCH which tells uvm_map that the
'align' argument specifies the starting color of the KVA range to be returned.

When calling uvm_km_alloc with UVM_KMF_VAONLY, also specify the starting
color of the kva range returned (UMV_KMF_COLORMATCH) and pass those to
uvm_map.

In uvm_pglistalloc, make sure the pages being returned have sequentially
advancing colors (so they can be mapped in a contiguous address range).
Add a few missing UVM_FLAG_COLORMATCH flags to uvm_pagealloc calls.

Make the socket and pipe loan color-safe.

Make the mips pmap enforce strict page color (color(VA) == color(PA)).
 1.104.2.2  03-Mar-2009  skrll Sync with HEAD.
 1.104.2.1  19-Jan-2009  skrll Sync with HEAD.
 1.105.2.1  13-May-2009  jym Sync with HEAD.

Commit is split, to avoid a "too many arguments" protocol error.
 1.108.4.5  31-May-2011  rmind sync with head
 1.108.4.4  19-May-2011  rmind Implement sharing of vnode_t::v_interlock amongst vnodes:
- Lock is shared amongst UVM objects using uvm_obj_setlock() or getnewvnode().
- Adjust vnode cache to handle unsharing, add VI_LOCKSHARE flag for that.
- Use sharing in tmpfs and layerfs for underlying object.
- Simplify locking in ubc_fault().
- Sprinkle some asserts.

Discussed with ad@.
 1.108.4.3  05-Mar-2011  rmind sync with head
 1.108.4.2  30-May-2010  rmind sync with head
 1.108.4.1  16-Mar-2010  rmind Change struct uvm_object::vmobjlock to be dynamically allocated with
mutex_obj_alloc(). It allows us to share the locks among UVM objects.
 1.108.2.1  17-Aug-2010  uebayasi Sync with HEAD.
 1.110.4.2  17-Feb-2011  bouyer Sync with HEAD
 1.110.4.1  08-Feb-2011  bouyer Sync with HEAD
 1.110.2.1  06-Jun-2011  jruoho Sync with HEAD.
 1.114.2.1  23-Jun-2011  cherry Catchup with rmind-uvmplock merge.
 1.116.12.3  03-Dec-2017  jdolecek update from HEAD
 1.116.12.2  20-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.116.12.1  20-Nov-2012  tls Resync to 2012-11-19 00:00:00 UTC
 1.116.8.1  22-Nov-2012  riz Pull up following revision(s) (requested by rmind in ticket #694):
sys/uvm/uvm_aobj.h: revision 1.22
sys/uvm/uvm_aobj.c: revision 1.117
sys/uvm/uvm_aobj.c: revision 1.118
sys/uvm/uvm_aobj.c: revision 1.119
sys/uvm/uvm_object.h: revision 1.33
- Describe uvm_aobj and the lock order.
- Remove unnecessary uao_dropswap_range1() wrapper.
- KNF. Sprinkle some __cacheline_aligned.
- Manage anonymous UVM object reference count with atomic ops.
- Fix an old bug of possible lock against oneself (uao_detach_locked() is
called from uao_swap_off() with uao_list_lock acquired). Also removes
the try-lock dance in uao_swap_off(), since the lock order changes.
 1.116.2.9  22-May-2014  yamt fix a merge botch
 1.116.2.8  22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.116.2.7  30-Oct-2012  yamt sync with head
 1.116.2.6  26-Nov-2011  yamt - uvm_page_array_fill: add some more parameters
- uvn_findpages: use gang-lookup
- genfs_putpages: re-enable backward clustering
- mechanical changes after the recent radixtree.h api changes
 1.116.2.5  18-Nov-2011  yamt - use mutex obj for pageable object
- add a function to wait for a mutex obj being available
- replace some "livelock" kpauses with it
 1.116.2.4  13-Nov-2011  yamt cache UVM_OBJ_IS_VNODE in pqflags
 1.116.2.3  06-Nov-2011  yamt remove pg->listq and uobj->memq
 1.116.2.2  06-Nov-2011  yamt adapt aobj
 1.116.2.1  02-Nov-2011  yamt page cache related changes

- maintain object pages in radix tree rather than rb tree.
- reduce unnecessary page scan in putpages. esp. when an object has a ton of
pages cached but only a few of them are dirty.
- reduce the number of pmap operations by tracking page dirtiness more
precisely in uvm layer.
- fix nfs commit range tracking.
- fix nfs write clustering. XXX hack
 1.119.2.1  18-May-2014  rmind sync with head
 1.120.2.1  10-Aug-2014  tls Rebase.
 1.122.4.3  28-Aug-2017  skrll Sync with HEAD
 1.122.4.2  05-Oct-2016  skrll Sync with HEAD
 1.122.4.1  22-Sep-2015  skrll Sync with HEAD
 1.123.2.1  06-Aug-2016  pgoyette Sync with HEAD
 1.125.2.2  21-Aug-2019  martin Pull up following revision(s) (requested by msaitoh in ticket #1342):

sys/uvm/uvm_aobj.c: revision 1.128

Avoid undefined behavior in uao_pagein_page(). Found by kUBSan. OK'd by
riastradh. I think this is a real bug on amd64 at least.
 1.125.2.1  02-Nov-2017  snj Pull up following revision(s) (requested by pgoyette in ticket #335):
share/man/man9/kernhist.9: 1.5-1.8
sys/arch/acorn26/acorn26/pmap.c: 1.39
sys/arch/arm/arm32/fault.c: 1.105 via patch
sys/arch/arm/arm32/pmap.c: 1.350, 1.359
sys/arch/arm/broadcom/bcm2835_bsc.c: 1.7
sys/arch/arm/omap/if_cpsw.c: 1.20
sys/arch/arm/omap/tiotg.c: 1.7
sys/arch/evbarm/conf/RPI2_INSTALL: 1.3
sys/dev/ic/sl811hs.c: 1.98
sys/dev/usb/ehci.c: 1.256
sys/dev/usb/if_axe.c: 1.83
sys/dev/usb/motg.c: 1.18
sys/dev/usb/ohci.c: 1.274
sys/dev/usb/ucom.c: 1.119
sys/dev/usb/uhci.c: 1.277
sys/dev/usb/uhub.c: 1.137
sys/dev/usb/umass.c: 1.160-1.162
sys/dev/usb/umass_quirks.c: 1.100
sys/dev/usb/umass_scsipi.c: 1.55
sys/dev/usb/usb.c: 1.168
sys/dev/usb/usb_mem.c: 1.70
sys/dev/usb/usb_subr.c: 1.221
sys/dev/usb/usbdi.c: 1.175
sys/dev/usb/usbdi_util.c: 1.67-1.70
sys/dev/usb/usbroothub.c: 1.3
sys/dev/usb/xhci.c: 1.75
sys/external/bsd/drm2/dist/drm/i915/i915_gem.c: 1.34
sys/kern/kern_history.c: 1.15
sys/kern/kern_xxx.c: 1.74
sys/kern/vfs_bio.c: 1.275-1.276
sys/miscfs/genfs/genfs_io.c: 1.71
sys/sys/kernhist.h: 1.21
sys/ufs/ffs/ffs_balloc.c: 1.63
sys/ufs/lfs/lfs_vfsops.c: 1.361
sys/ufs/lfs/ulfs_inode.c: 1.21
sys/ufs/lfs/ulfs_vnops.c: 1.52
sys/ufs/ufs/ufs_inode.c: 1.102
sys/ufs/ufs/ufs_vnops.c: 1.239
sys/uvm/pmap/pmap.c: 1.37-1.39
sys/uvm/pmap/pmap_tlb.c: 1.22
sys/uvm/uvm_amap.c: 1.108
sys/uvm/uvm_anon.c: 1.64
sys/uvm/uvm_aobj.c: 1.126
sys/uvm/uvm_bio.c: 1.91
sys/uvm/uvm_device.c: 1.66
sys/uvm/uvm_fault.c: 1.201
sys/uvm/uvm_km.c: 1.144
sys/uvm/uvm_loan.c: 1.85
sys/uvm/uvm_map.c: 1.353
sys/uvm/uvm_page.c: 1.194
sys/uvm/uvm_pager.c: 1.111
sys/uvm/uvm_pdaemon.c: 1.109
sys/uvm/uvm_swap.c: 1.175
sys/uvm/uvm_vnode.c: 1.103
usr.bin/vmstat/vmstat.c: 1.219
Reorder to test for null before null deref in debug code
--
Reorder to test for null before null deref in debug code
--
KNF
--
No need for '\n' in UVMHIST_LOG
--
normalise a BIOHIST log message
--
Update the kernhist(9) kernel history code to address issues identified
in PR kern/52639, as well as some general cleaning-up...
(As proposed on tech-kern@ with additional changes and enhancements.)
Details of changes:
* All history arguments are now stored as uintmax_t values[1], both in
the kernel and in the structures used for exporting the history data
to userland via sysctl(9). This avoids problems on some architectures
where passing a 64-bit (or larger) value to printf(3) can cause it to
process the value as multiple arguments. (This can be particularly
problematic when printf()'s format string is not a literal, since in
that case the compiler cannot know how large each argument should be.)
* Update the data structures used for exporting kernel history data to
include a version number as well as the length of history arguments.
* All [2] existing users of kernhist(9) have had their format strings
updated. Each format specifier now includes an explicit length
modifier 'j' to refer to numeric values of the size of uintmax_t.
* All [2] existing users of kernhist(9) have had their format strings
updated to replace uses of "%p" with "%#jx", and the pointer
arguments are now cast to (uintptr_t) before being subsequently cast
to (uintmax_t). This is needed to avoid compiler warnings about
casting "pointer to integer of a different size."
* All [2] existing users of kernhist(9) have had instances of "%s" or
"%c" format strings replaced with numeric formats; several instances
of mis-match between format string and argument list have been fixed.
* vmstat(1) has been modified to handle the new size of arguments in the
history data as exported by sysctl(9).
* vmstat(1) now provides a warning message if the history requested with
the -u option does not exist (previously, this condition was silently
ignored, with only a single blank line being printed).
* vmstat(1) now checks the version and argument length included in the
data exported via sysctl(9) and exits if they do not match the values
with which vmstat was built.
* The kernhist(9) man-page has been updated to note the additional
requirements imposed on the format strings, along with several other
minor changes and enhancements.
[1] It would have been possible to use an explicit length (for example,
uint64_t) for the history arguments. But that would require another
"rototill" of all the users in the future when we add support for an
architecture that supports a larger size. Also, the printf(3)
format
specifiers for explicitly-sized values, such as "%"PRIu64, are much
more verbose (and less aesthetically appealing, IMHO) than simply
using "%ju".
[2] I've tried very hard to find "all [the] existing users of
kernhist(9)"
but it is possible that I've missed some of them. I would be glad
to
update any stragglers that anyone identifies.
--
For some reason this single kernel seems to have outgrown its declared
size as a result of the kernhist(9) changes. Bump the size.
XXX The amount of increase may be excessive - anyone with more detailed
XXX knowledge please feel free to further adjust the value
appropriately.
--
Misssed one cast of pointer --> uintptr_t in previous kernhist(9) commit
--
And yet another one. :(
--
Use correct mark-up for NetBSD version.
--
More improvements in grammar and readability.
--
Remove a stray '"' (obvious typo) and add a couple of casts that are
probably needed.
--
And replace an instance of "%p" conversion with "%#jx"
--
Whitespace fix. Give Bl tag table a width. Fix Xr.
 1.126.2.1  25-Jun-2018  pgoyette Sync with HEAD
 1.127.2.1  13-Apr-2020  martin Mostly merge changes from HEAD upto 20200411
 1.133.2.2  29-Feb-2020  ad Sync with head.
 1.133.2.1  17-Jan-2020  ad Sync with head.
 1.151.2.2  03-Apr-2021  thorpej Sync with HEAD.
 1.151.2.1  14-Dec-2020  thorpej Sync w/ HEAD.

RSS XML Feed