Home | History | Annotate | Download | only in uvm
History log of /src/sys/uvm/uvm_pglist.c
RevisionDateAuthorComments
 1.92  14-Jan-2024  tnn fix DEBUG build
 1.91  13-Jan-2024  tnn uvm: change type of uvm_physseg.start_hint from u_int to u_long

Avoids assertion failure in uvm_pglistalloc_s_ps() with large paddrs.
PR kern/57683.
 1.90  21-Dec-2021  skrll branches: 1.90.4;
Update uvm_pglistalloc_[cs]_ps to return EINVAL if [low, high] doesn't
match any memory.

Useful for bus_dmamem_alloc where a tag might not cover any memory.
This will be used in an update to ehci.

"looks good" from chuq@
 1.89  20-Dec-2021  skrll Slight code re-structure and wrap a long line. Interestingly this gives
the same binary before and after.
 1.88  26-Mar-2021  chs in uvm_pglistalloc_contig_aggressive(), avoid looking forward past
the end of the target range of the physseg.
fixes PR 56074.
 1.87  24-Mar-2021  skrll Trailing whitespace
 1.86  07-Oct-2020  chs branches: 1.86.2; 1.86.4;
Add a new, more aggressive allocator for uvm_pglistalloc() to allocate
contiguous physical pages, and try this new allocator if the existing
one fails. The existing contig allocator only tries to allocate pages
that are already free, which works fine shortly after boot but rarely
works after the system has been up for a while. The new allocator uses
the pagedaemon to evict pages from memory in the hope that this will
free up a range of pages that satisfies the constraits of the request.
This should help with things like plugging in a USB device, which often
fails for some USB controllers because they can't get contigous memory.
 1.85  14-Jun-2020  ad Remove PG_ZERO. It worked brilliantly on x86 machines from the mid-90s but
having spent an age experimenting with it over the last 6 months on various
machines and with different use cases it's always either break-even or a
slight net loss for me.
 1.84  11-Jun-2020  ad Counter tweaks:

- Don't need to count anonpages+filepages any more; clean+unknown+dirty for
each kind of page can be summed to get the totals.

- Track the number of free pages with a counter so that it's one less thing
for the allocator to do, which opens up further options there.

- Remove cpu_count_sync_one(). It has no users and doesn't save a whole lot.
For the cheap option, give cpu_count_sync() a boolean parameter indicating
that a cached value is okay, and rate limit the updates for cached values
to hz.
 1.83  11-Jun-2020  ad uvm_availmem(): give it a boolean argument to specify whether a recent
cached value will do, or if the very latest total must be fetched. It can
be called thousands of times a second and fetching the totals impacts not
only the calling LWP but other CPUs doing unrelated activity in the VM
system.
 1.82  23-May-2020  ad uvm_pglistfree(): just use uvm_pagefree().
 1.81  01-Mar-2020  ad uvm_pglistalloc() / uvm_pglistfree() musn't be called from interrupt
context. Assert it.
 1.80  20-Feb-2020  rin Make this compile again with PGALLOC_VERBOSE.
 1.79  31-Dec-2019  ad branches: 1.79.2;
Rename uvm_free() -> uvm_availmem().
 1.78  27-Dec-2019  ad Redo the page allocator to perform better, especially on multi-core and
multi-socket systems. Proposed on tech-kern. While here:

- add rudimentary NUMA support - needs more work.
- remove now unused "listq" from vm_page.
 1.77  21-Dec-2019  ad Detangle the pagedaemon from uvm_fpageqlock:

- Have a single lock (uvmpd_lock) to protect pagedaemon state that was
previously covered by uvmpd_pool_drain_lock plus uvm_fpageqlock.
- Don't require any locks be held when calling uvm_kick_pdaemon().
- Use uvm_free().
 1.76  21-Dec-2019  ad - Rename VM_PGCOLOR_BUCKET() to VM_PGCOLOR(). I want to reuse "bucket" for
something else soon and TBH it matches what this macro does better.

- Add inlines to set/get locator values in the unused lower bits of
pg->phys_addr. Begin by using it to cache the freelist index, because
computing it is expensive and that shows up during profiling. Discussed
on tech-kern.
 1.75  21-Dec-2019  ad uvmexp.free -> uvm_free()
 1.74  16-Dec-2019  ad - Extend the per-CPU counters matt@ did to include all of the hot counters
in UVM, excluding uvmexp.free, which needs special treatment and will be
done with a separate commit. Cuts system time for a build by 20-25% on
a 48 CPU machine w/DIAGNOSTIC.

- Avoid 64-bit integer divide on every fault (for rnd_add_uint32).
 1.73  13-Dec-2019  ad Break the global uvm_pageqlock into a per-page identity lock and a private
lock for use of the pagedaemon policy code. Discussed on tech-kern.

PR kern/54209: NetBSD 8 large memory performance extremely low
PR kern/54210: NetBSD-8 processes presumably not exiting
PR kern/54727: writing a large file causes unreasonable system behaviour
 1.72  13-Nov-2018  mrg branches: 1.72.4;
only warn once per call to uvm_pglistalloc_simple() if waiting.
 1.71  03-Sep-2018  riastradh Rename min/max -> uimin/uimax for better honesty.

These functions are defined on unsigned int. The generic name
min/max should not silently truncate to 32 bits on 64-bit systems.
This is purely a name change -- no functional change intended.

HOWEVER! Some subsystems have

#define min(a, b) ((a) < (b) ? (a) : (b))
#define max(a, b) ((a) > (b) ? (a) : (b))

even though our standard name for that is MIN/MAX. Although these
may invite multiple evaluation bugs, these do _not_ cause integer
truncation.

To avoid `fixing' these cases, I first changed the name in libkern,
and then compile-tested every file where min/max occurred in order to
confirm that it failed -- and thus confirm that nothing shadowed
min/max -- before changing it.

I have left a handful of bootloaders that are too annoying to
compile-test, and some dead code:

cobalt ews4800mips hp300 hppa ia64 luna68k vax
acorn32/if_ie.c (not included in any kernels)
macppc/if_gm.c (superseded by gem(4))

It should be easy to fix the fallout once identified -- this way of
doing things fails safe, and the goal here, after all, is to _avoid_
silent integer truncations, not introduce them.

Maybe one day we can reintroduce min/max as type-generic things that
never silently truncate. But we should avoid doing that for a while,
so that existing code has a chance to be detected by the compiler for
conversion to uimin/uimax without changing the semantics until we can
properly audit it all. (Who knows, maybe in some cases integer
truncation is actually intended!)
 1.70  23-Dec-2016  skrll branches: 1.70.14; 1.70.16;
PRIxPHYSMEM -> PRIxPHYSSEG to fix the build
 1.69  23-Dec-2016  skrll Whitespace
 1.68  23-Dec-2016  cherry "Make NetBSD great again!"

Introduce uvm_hotplug(9) to the kernel.

Many thanks, in no particular order to:

TNF, for funding the project.

Chuck Silvers - for multiple API reviews and feedback.
Nick Hudson - for testing on multiple architectures and bugfix patches.
Everyone who helped with boot testing.

KeK (http://www.kek.org.in) for hosting the primary developers.
 1.67  26-Oct-2014  christos branches: 1.67.2; 1.67.4;
Define UVMDEBUG for expensive debugging operations. Idea from chuq.
 1.66  05-Sep-2014  matt Don't use C++ try keyword as a variable name.
 1.65  19-May-2014  riastradh Back out previous silliness -- on failure no pages are allocated.
 1.64  19-May-2014  riastradh Don't leak memory on failure in uvm_pglistalloc_contig.

Free pages like uvm_pglistalloc_simple does.

Discovered by code inspection.
 1.63  15-Sep-2013  martin branches: 1.63.2;
Mark potentialy unused variables
 1.62  27-Sep-2011  jym branches: 1.62.2; 1.62.12; 1.62.16;
Modify *ASSERTMSG() so they are now used as variadic macros. The main goal
is to provide routines that do as KASSERT(9) says: append a message
to the panic format string when the assertion triggers, with optional
arguments.

Fix call sites to reflect the new definition.

Discussed on tech-kern@. See
http://mail-index.netbsd.org/tech-kern/2011/09/07/msg011427.html
 1.61  23-Apr-2011  rmind Replace "malloc" in comments, remove unnecessary header inclusions.
 1.60  26-Jan-2011  enami Introducing inner loop prevent us from exiting from the original loop.
 1.59  25-Jan-2011  matt When starting the second pass, don't continue the for loop but instead
just test try exceeding limit.
 1.58  24-Jan-2011  matt Use the (new) KDASSERTMSG
 1.57  24-Jan-2011  matt Fix start_hint in "simple" alloc (fencepost error).
When restarting the loop, make sure end is not above current limit.
Do a quick test to see if the physseg is within the range of desired addresses.
 1.56  23-Jan-2011  he DEBUG does not imply DIAGNOSTIC; make sure we have a non-null
KASSERTMSG implementation (DIAGNOSTIC) so that the variable inside
the DEBUG section gets used.
 1.55  22-Jan-2011  matt Fix the corruption of ps->start_hint.
 1.54  21-Jan-2011  matt Cleanup/add some asserts. no functional change.
 1.53  21-Jan-2011  cegger buildfix: use PRIxPADDR for type paddr_t
 1.52  18-Jan-2011  matt branches: 1.52.2;
Improve the efficiency of searching for a contiguous set of free pages.
 1.51  25-Nov-2010  uebayasi branches: 1.51.2;
Revert vm_physseg allocation changes. A report says that it causes
panics when used with mplayer in heavy load.
 1.50  18-Nov-2010  cegger build fix: vm_physmem_index is only used with DEBUG.
Fix build when DIAGNOSTIC is enabled but not DEBUG
 1.49  18-Nov-2010  uebayasi Optimize DIAGNOSTIC check code.
 1.48  18-Nov-2010  uebayasi Fix DIAGNOSTIC physseg find check.
 1.47  14-Nov-2010  uebayasi Be a little more friendly to dynamic physical segment registration.

Maintain an array of pointer to struct vm_physseg, instead of struct
array. So that VM subsystem can take its pointer safely. Pointer
to this struct will replace raw paddr_t usage in the future.

Dynamic removal is not supported yet.

Only MD data structure changes, no kernel bump needed.

Tested on i386, amd64, powerpc/ibm40x, arm11.
 1.46  17-Jun-2010  mrg disable some DEBUG code uvm_pglist_add() that has severe performance
problems with large mappings. i've seen my system hang for a total
of 45 seconds when radeondrm is opened by X11, and it is the checks
in this function that take so long.
 1.45  10-Mar-2009  nonaka branches: 1.45.2; 1.45.4;
remove "#define PGALLOC_VERBOSE".
 1.44  09-Mar-2009  reinoud For this physical address printing use uintmax_t since on Xen PAE this length
(64) is not the same as the base architecture (32).
 1.43  09-Mar-2009  nonaka fix compile failure when PGALLOC_VERBOSE is defined.
 1.42  04-Jun-2008  ad branches: 1.42.6; 1.42.12; 1.42.16;
- vm_page: put listq, pageq into a union alongside a LIST_ENTRY, so we can
use both types of list.

- Make page coloring and idle zero state per-CPU.

- Maintain per-CPU page freelists. When freeing, put pages onto the local
CPU's lists and the global lists. When allocating, prefer to take pages
from the local CPU. If none are available take from the global list as
done now. Proposed on tech-kern@.
 1.41  02-Jun-2008  ad UVM_PAGEZERO_TARGET -> UVM_PAGEZERO_LOWAT
 1.40  28-Apr-2008  martin branches: 1.40.2;
Remove clause 3 and 4 from TNF licenses
 1.39  27-Feb-2008  ad branches: 1.39.2; 1.39.4;
Assert uvm_fpageqlock is held in a few more places.
 1.38  21-Jul-2007  ad branches: 1.38.6; 1.38.22; 1.38.26; 1.38.28;
Merge unobtrusive locking changes from the vmlocking branch.
 1.37  21-Feb-2007  thorpej branches: 1.37.4; 1.37.12;
Replace the Mach-derived boolean_t type with the C99 bool type. A
future commit will replace use of TRUE and FALSE with true and false.
 1.36  15-Sep-2006  yamt branches: 1.36.6;
merge yamt-pdpolicy branch.
- separate page replacement policy from the rest of kernel
- implement an alternative replacement policy
 1.35  14-May-2006  christos branches: 1.35.8;
XXX: GCC uninitialized.
 1.34  11-Dec-2005  christos branches: 1.34.4; 1.34.6; 1.34.8; 1.34.12;
merge ktrace-lwp.
 1.33  27-Jun-2005  thorpej branches: 1.33.2;
Use ANSI function decls.
 1.32  17-Sep-2004  yamt make free page queue filo rather than fifo.
data in pages freed more recently are more likely on cpu cache.
 1.31  24-Mar-2004  junyoung Drop trailing spaces.
 1.30  03-Nov-2003  yamt add a DEBUG check if freed PG_ZERO pages are really zero-filled.
 1.29  01-Nov-2003  yamt in uvm_pagefree and friends, if freed pages have been marked by
PG_ZERO flag, put them to PGFL_ZEROS queue rather than default one
so that we can re-use zero-filled pages efficiently.
 1.28  26-Aug-2003  yamt use VM_PAGE_TO_PHYS macro instead of using phys_addr directly.
 1.27  02-Aug-2003  drochner sync comments with reality
 1.26  10-Mar-2003  thorpej branches: 1.26.2;
Make PGALLOC_VERBOSE compile where size_t != int.
 1.25  02-Nov-2002  perry /*CONTCOND*/ while (0)'ed macros
 1.24  27-Jun-2002  drochner Big cleanup and speed improvements to pglist_alloc code:
-pass vm_physseg* instead of physseg index, and PFN (int) instead
of physical address (could be done even more)
-simplify detection of boundary crossing and behave more intelligently
in this case
-take stuff out of the inner loops, or put into "#ifdef DEBUG"
(because we move along physsegs we don't need to check that the
pages are physically contigous)
-make the "simple" and "contigous" branches look more uniform; at
least the outer loops might coalesce one day
 1.23  20-Jun-2002  enami Shift by PAGE_SHIFT instead of dividing by PAGE_SIZE.
 1.22  18-Jun-2002  drochner Make the DMA memory allocators (uvm_pglistalloc())
obey the preferences expressed by freelist assignment,
to avoid wasting valuable "low memory" to devices which
don't really need it.
comments:
-I'm not sure searching the physsegs within a freelist
beginning with the biggest is the right thing. This is
what the "memory steal" code in uvm_page.c does, so
keep it consistent.
-There seems to be some confusion whether the upper
address limit passed is inclusive or not. Stays on
the save side, possibly leaving one page out.
-The boundary/pagemask check can be simplified, also some
arguments passed are only used for diagnostic checks.
-Integration with UVM_PAGE_TRKOWN???
 1.21  02-Jun-2002  drochner move initialization of the "struct pglist" returned by uvm_pglistalloc()
from the calling code into uvm_pglistalloc() itself for consistency
and easier error handling
 1.20  29-May-2002  drochner Add another allocator to uvm_pglistalloc() which is used in the case where
no alignment / boundary / nsegs restrictions apply.
This one doesn't insist in a contigous range, and it honours the "waitok"
flag, thus succeeds in situations which were hopeless with the existing one.

(A solution which searches for a minimum number of contiguous ranges using
some best-fit or so algorithm would be expensive to implement; I believe the
"either-or" done here does reflect the current use by bus_dma quite well.)

Now agp memory allocation is robust for me. (tested on i810)
 1.19  10-Nov-2001  lukem branches: 1.19.4; 1.19.8;
add RCSIDs, and in some cases, slightly cleanup #include order
 1.18  15-Sep-2001  chs branches: 1.18.2;
a whole bunch of changes to improve performance and robustness under load:

- remove special treatment of pager_map mappings in pmaps. this is
required now, since I've removed the globals that expose the address range.
pager_map now uses pmap_kenter_pa() instead of pmap_enter(), so there's
no longer any need to special-case it.
- eliminate struct uvm_vnode by moving its fields into struct vnode.
- rewrite the pageout path. the pager is now responsible for handling the
high-level requests instead of only getting control after a bunch of work
has already been done on its behalf. this will allow us to UBCify LFS,
which needs tighter control over its pages than other filesystems do.
writing a page to disk no longer requires making it read-only, which
allows us to write wired pages without causing all kinds of havoc.
- use a new PG_PAGEOUT flag to indicate that a page should be freed
on behalf of the pagedaemon when it's unlocked. this flag is very similar
to PG_RELEASED, but unlike PG_RELEASED, PG_PAGEOUT can be cleared if the
pageout fails due to eg. an indirect-block buffer being locked.
this allows us to remove the "version" field from struct vm_page,
and together with shrinking "loan_count" from 32 bits to 16,
struct vm_page is now 4 bytes smaller.
- no longer use PG_RELEASED for swap-backed pages. if the page is busy
because it's being paged out, we can't release the swap slot to be
reallocated until that write is complete, but unlike with vnodes we
don't keep a count of in-progress writes so there's no good way to
know when the write is done. instead, when we need to free a busy
swap-backed page, just sleep until we can get it busy ourselves.
- implement a fast-path for extending writes which allows us to avoid
zeroing new pages. this substantially reduces cpu usage.
- encapsulate the data used by the genfs code in a struct genfs_node,
which must be the first element of the filesystem-specific vnode data
for filesystems which use genfs_{get,put}pages().
- eliminate many of the UVM pagerops, since they aren't needed anymore
now that the pager "put" operation is a higher-level operation.
- enhance the genfs code to allow NFS to use the genfs_{get,put}pages
instead of a modified copy.
- clean up struct vnode by removing all the fields that used to be used by
the vfs_cluster.c code (which we don't use anymore with UBC).
- remove kmem_object and mb_object since they were useless.
instead of allocating pages to these objects, we now just allocate
pages with no object. such pages are mapped in the kernel until they
are freed, so we can use the mapping to find the page to free it.
this allows us to remove splvm() protection in several places.

The sum of all these changes improves write throughput on my
decstation 5000/200 to within 1% of the rate of NetBSD 1.5
and reduces the elapsed time for "make release" of a NetBSD 1.5
source tree on my 128MB pc to 10% less than a 1.5 kernel took.
 1.17  27-Jun-2001  thorpej branches: 1.17.2; 1.17.4;
Macro'ize the code that checks the free and inactive thresholds and
wakes the pagedaemon.
 1.16  26-May-2001  chs replace vm_page_t with struct vm_page *.
 1.15  25-May-2001  chs remove trailing whitespace.
 1.14  29-Apr-2001  thorpej Implement page coloring, using a round-robin bucket selection
algorithm (Solaris calls this "Bin Hopping").

This implementation currently relies on MD code to define a
constant defining the number of buckets. This will change
reasonably soon (MD code will be able to dynamically size
the bucket array).
 1.13  18-Feb-2001  chs branches: 1.13.2;
clean up DIAGNOSTIC checks, use KASSERT().
 1.12  25-Nov-2000  chs lots of cleanup:
use queue.h macros and KASSERT().
address amap offsets in pages instead of bytes.
make amap_ref() and amap_unref() take an amap, offset and length
instead of a vm_map_entry_t.
improve whitespace and comments.
 1.11  27-Jun-2000  mrg remove include of <vm/vm.h>
 1.10  20-May-2000  thorpej Clean up a comment.
 1.9  24-Apr-2000  thorpej Changes necessary to implement pre-zero'ing of pages in the idle loop:
- Make page free lists have two actual queues: known-zero pages and
pages with unknown contents.
- Implement uvm_pageidlezero(). This function attempts to zero up to
the target number of pages until the target has been reached (currently
target is `all free pages') or until whichqs becomes non-zero (indicating
that a process is ready to run).
- Define a new hook for the pmap module for pre-zero'ing pages. This is
used to zero the pages using uncached access. This allows us to zero
as many pages as we want without polluting the cache.

In order to use this feature, each platform must add the appropropriate
glue in their idle loop.
 1.8  22-Jul-1999  thorpej branches: 1.8.2;
Garbage collect thread_sleep()/thread_wakeup() left over from the old
Mach VM code. Also nuke iprintf(), which was no longer used anywhere.

Add proclist locking where appropriate.
 1.7  24-May-1999  thorpej - Change uvm_{lock,unlock}_fpageq() to return/take the previous interrupt
level directly, instead of making the caller wrap the calls in
splimp()/splx().
- Add a comment documenting that interrupts that cause memory allocation
must be blocked while the free page queue is locked.

Since interrupts must be blocked while this lock is asserted, tying them
together like this helps to prevent mistakes.
 1.6  13-Aug-1998  eeh branches: 1.6.2; 1.6.8;
Merge paddr_t changes into the main branch.
 1.5  08-Jul-1998  thorpej branches: 1.5.2;
Add support for multiple memory free lists. There is at least one
default free list, and 0 - N additional free list, in order of descending
priority.

A new page allocation function, uvm_pagealloc_strat(), has been added,
providing three page allocation strategies:

- normal: high -> low priority free list walk, taking the
page off the first free list that has one.

- only: attempt to allocate a page only from the specified free
list, failing if that free list has none available.

- fallback: if `only' fails, fall back on `normal'.

uvm_pagealloc(...) is provided for normal use (and is a synonym for
uvm_pagealloc_strat(..., UVM_PGA_STRAT_NORMAL, 0); the free list argument
is ignored for the `normal' case).

uvm_page_physload() now specified which free list the pages will be
loaded onto. This means that some platforms which have multiple physical
memory segments may define additional vm_physsegs if they wish to break
individual physical segments into differing priorities.

Machine-dependent code must define _at least_ the following constants
in <machine/vmparam.h>:

VM_NFREELIST: the number of free lists the system will have

VM_FREELIST_DEFAULT: the default freelist (should always be 0,
but is defined in machdep code so that it's with all of the
other free list-related constants).

Additional free list names may be defined by machine-dependent code, but
they will only be used by machine-dependent code (e.g. for loading the
vm_physsegs).
 1.4  05-May-1998  kleink Remove inclusions of syscall (and syscall argument) related header files;
we don't need them here.
 1.3  09-Mar-1998  mrg KNF.
 1.2  06-Feb-1998  thorpej RCS ID police.
 1.1  05-Feb-1998  mrg branches: 1.1.1;
Initial revision
 1.1.1.1  05-Feb-1998  mrg initial import of the new virtual memory system, UVM, into -current.

UVM was written by chuck cranor <chuck@maria.wustl.edu>, with some
minor portions derived from the old Mach code. i provided some help
getting swap and paging working, and other bug fixes/ideas. chuck
silvers <chuq@chuq.com> also provided some other fixes.

this is the UVM kernel code portion.


this will be KNF'd shortly. :-)
 1.5.2.1  30-Jul-1998  eeh Split vm_offset_t and vm_size_t into paddr_t, psize_t, vaddr_t, and vsize_t.
 1.6.8.2  21-Jun-1999  thorpej Sync w/ -current.
 1.6.8.1  07-Jun-1999  chs merge everything from chs-ubc branch.
 1.6.2.1  25-Feb-1999  chs in uvm_pglistalloc(), treat pages being paged out as "free"
when deciding whether to wakeup the pagedaemon.
also, don't unlock the free page queue until we've done the wakeup.
 1.8.2.3  12-Mar-2001  bouyer Sync with HEAD.
 1.8.2.2  08-Dec-2000  bouyer Sync with HEAD.
 1.8.2.1  20-Nov-2000  bouyer Update thorpej_scsipi to -current as of a month ago
 1.13.2.7  11-Nov-2002  nathanw Catch up to -current
 1.13.2.6  01-Aug-2002  nathanw Catch up to -current.
 1.13.2.5  20-Jun-2002  nathanw Catch up to -current.
 1.13.2.4  14-Nov-2001  nathanw Catch up to -current.
 1.13.2.3  21-Sep-2001  nathanw Catch up to -current.
 1.13.2.2  24-Aug-2001  nathanw Catch up with -current.
 1.13.2.1  21-Jun-2001  nathanw Catch up to -current.
 1.17.4.1  01-Oct-2001  fvdl Catch up with -current.
 1.17.2.3  06-Sep-2002  jdolecek sync kqueue branch with HEAD
 1.17.2.2  23-Jun-2002  jdolecek catch up with -current on kqueue branch
 1.17.2.1  10-Jan-2002  thorpej Sync kqueue branch with -current.
 1.18.2.1  12-Nov-2001  thorpej Sync the thorpej-mips-cache branch with -current.
 1.19.8.3  15-Jul-2002  gehenna catch up with -current.
 1.19.8.2  20-Jun-2002  gehenna catch up with -current.
 1.19.8.1  30-May-2002  gehenna Catch up with -current.
 1.19.4.1  12-Mar-2002  thorpej Convert the fpageqlock to a spin mutex at IPL_VM and rename it
to fpageq_mutex.
 1.26.2.4  10-Nov-2005  skrll Sync with HEAD. Here we go again...
 1.26.2.3  21-Sep-2004  skrll Fix the sync with head I botched.
 1.26.2.2  18-Sep-2004  skrll Sync with HEAD.
 1.26.2.1  03-Aug-2004  skrll Sync with HEAD
 1.33.2.5  17-Mar-2008  yamt sync with head.
 1.33.2.4  03-Sep-2007  yamt sync with head.
 1.33.2.3  26-Feb-2007  yamt sync with head.
 1.33.2.2  30-Dec-2006  yamt sync with head.
 1.33.2.1  21-Jun-2006  yamt sync with head.
 1.34.12.1  24-May-2006  tron Merge 2006-05-24 NetBSD-current into the "peter-altq" branch.
 1.34.8.3  15-Sep-2006  yamt make UVM_KICK_PDAEMON() a real function and stop including
uvm_pdpolicy.h from uvm.h. this also fixes build of pmap(1).
 1.34.8.2  24-May-2006  yamt sync with head.
 1.34.8.1  05-Mar-2006  yamt separate page replacement policy from the rest of kernel.
 1.34.6.1  01-Jun-2006  kardel Sync with head.
 1.34.4.1  09-Sep-2006  rpaulo sync with head
 1.35.8.1  18-Nov-2006  ad Sync with head.
 1.36.6.1  27-Feb-2007  yamt - sync with head.
- move sched_changepri back to kern_synch.c as it doesn't know PPQ anymore.
 1.37.12.1  15-Aug-2007  skrll Sync with HEAD.
 1.37.4.2  20-Aug-2007  ad Sync with HEAD.
 1.37.4.1  13-Mar-2007  ad Pull in the initial set of changes for the vmlocking branch.
 1.38.28.2  21-Jul-2007  ad Merge unobtrusive locking changes from the vmlocking branch.
 1.38.28.1  21-Jul-2007  ad file uvm_pglist.c was added on branch matt-mips64 on 2007-07-21 19:21:56 +0000
 1.38.26.3  05-Jun-2008  mjf Sync with HEAD.

Also fix build.
 1.38.26.2  02-Jun-2008  mjf Sync with HEAD.
 1.38.26.1  03-Apr-2008  mjf Sync with HEAD.
 1.38.22.1  24-Mar-2008  keiichi sync with head.
 1.38.6.1  23-Mar-2008  matt sync with HEAD
 1.39.4.3  11-Aug-2010  yamt sync with head.
 1.39.4.2  04-May-2009  yamt sync with head.
 1.39.4.1  16-May-2008  yamt sync with head.
 1.39.2.3  17-Jun-2008  yamt sync with head.
 1.39.2.2  04-Jun-2008  yamt sync with head
 1.39.2.1  18-May-2008  yamt sync with head.
 1.40.2.1  23-Jun-2008  wrstuden Sync w/ -current. 34 merge conflicts to follow.
 1.42.16.14  15-Feb-2014  matt Adapt to K{,D}ASSERTMSG changes
 1.42.16.13  29-Feb-2012  matt Improve UVM_PAGE_TRKOWN.
Add more asserts to uvm_page.
 1.42.16.12  14-Feb-2012  matt Add more KASSERTs (more! more! more!).
When returning page to the free pool, make sure to dequeue the pages before
hand or free page queue corruption will happen.
 1.42.16.11  09-Feb-2012  matt Major changes to uvm.
Support multiple collections (groups) of free pages and run the page
reclaimation algorithm on each group independently.
 1.42.16.10  03-Jun-2011  matt Restore $NetBSD$
 1.42.16.9  03-Jun-2011  matt Rework page free lists to be sorted by color first rather than free_list.
Kept per color PGFL_* counter in each page free list.
Minor cleanups.
 1.42.16.8  27-May-2011  matt Fix a bug where limit could be greater avail_end. Now if that happens, we
just bail. Use KDASSERTMSG so panics are more informative.
 1.42.16.7  25-May-2011  matt Make uvm_map recognize UVM_FLAG_COLORMATCH which tells uvm_map that the
'align' argument specifies the starting color of the KVA range to be returned.

When calling uvm_km_alloc with UVM_KMF_VAONLY, also specify the starting
color of the kva range returned (UMV_KMF_COLORMATCH) and pass those to
uvm_map.

In uvm_pglistalloc, make sure the pages being returned have sequentially
advancing colors (so they can be mapped in a contiguous address range).
Add a few missing UVM_FLAG_COLORMATCH flags to uvm_pagealloc calls.

Make the socket and pipe loan color-safe.

Make the mips pmap enforce strict page color (color(VA) == color(PA)).
 1.42.16.6  01-Jun-2010  matt Fix bad initialization spotted by Manuel Bouyer.
 1.42.16.5  23-Jan-2010  matt Use roundup2 instead of roundup when doing alignment rounding since all
alignments must be a power of 2. (thanks to rmind for suggesting it).
 1.42.16.4  23-Jan-2010  matt Add a start_hint to vm_physseg so when allocating pages, we can skip
forward over pages that are probably still allocated.
 1.42.16.3  22-Jan-2010  matt Remove some optimizations since they actually don't do the right thing.
We never want to test the starting page first since it doesn't really give
use any good information that we can use for the next pass.
 1.42.16.2  22-Jan-2010  snj Fix a couple comment typos.
 1.42.16.1  22-Jan-2010  matt Rework the algorithm to allocate contiguous pages to be much much faster.
(read the comments if you want to know how it's done).
 1.42.12.1  13-May-2009  jym Sync with HEAD.

Commit is split, to avoid a "too many arguments" protocol error.
 1.42.6.1  28-Apr-2009  skrll Sync with HEAD.
 1.45.4.3  31-May-2011  rmind sync with head
 1.45.4.2  05-Mar-2011  rmind sync with head
 1.45.4.1  03-Jul-2010  rmind sync with head
 1.45.2.5  21-Nov-2010  uebayasi Sync with HEAD.
 1.45.2.4  12-Nov-2010  uebayasi Fix debug code.
 1.45.2.3  17-Aug-2010  uebayasi Sync with HEAD.
 1.45.2.2  28-Apr-2010  uebayasi Always use struct vm_physseg *vm_physmem_ptrs[] in MD code.
 1.45.2.1  09-Feb-2010  uebayasi vm_nphysseg -> vm_nphysmem
 1.51.2.1  06-Jun-2011  jruoho Sync with HEAD.
 1.52.2.1  08-Feb-2011  bouyer Sync with HEAD
 1.62.16.1  18-May-2014  rmind sync with head
 1.62.12.2  03-Dec-2017  jdolecek update from HEAD
 1.62.12.1  20-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.62.2.2  22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.62.2.1  06-Nov-2011  yamt remove pg->listq and uobj->memq
 1.63.2.1  10-Aug-2014  tls Rebase.
 1.67.4.1  07-Jan-2017  pgoyette Sync with HEAD. (Note that most of these changes are simply $NetBSD$
tag issues.)
 1.67.2.1  05-Feb-2017  skrll Sync with HEAD
 1.70.16.2  08-Apr-2020  martin Merge changes from current as of 20200406
 1.70.16.1  10-Jun-2019  christos Sync with HEAD
 1.70.14.2  26-Nov-2018  pgoyette Sync with HEAD, resolve a couple of conflicts
 1.70.14.1  06-Sep-2018  pgoyette Sync with HEAD

Resolve a couple of conflicts (result of the uimin/uimax changes)
 1.72.4.1  27-Feb-2020  martin Pull up following revision(s) (requested by rin in ticket #732):

sys/uvm/uvm_pglist.c: revision 1.80

Make this compile again with PGALLOC_VERBOSE.
 1.79.2.1  29-Feb-2020  ad Sync with head.
 1.86.4.1  03-Apr-2021  thorpej Sync with HEAD.
 1.86.2.1  03-Apr-2021  thorpej Sync with HEAD.
 1.90.4.1  15-Jan-2024  martin Pull up following revision(s) (requested by tnn in ticket #554):

sys/uvm/uvm_physseg.c: revision 1.20
sys/uvm/uvm_pglist.c: revision 1.91
sys/uvm/uvm_pglist.c: revision 1.92
sys/uvm/uvm_physseg.h: revision 1.9

uvm: change type of uvm_physseg.start_hint from u_int to u_long
Avoids assertion failure in uvm_pglistalloc_s_ps() with large paddrs.
PR kern/57683.

fix DEBUG build

RSS XML Feed