Fix 32bit UVMHIST builds

More debug.

Fix types in pmap_page_clear_attributes so that the top bits of
the u_long mdpg_attrs aren't dropped giving atomic_cas_ulong no
chance of completing if any of the top bits is set.

Update pmap_page_set_attributes for consistency.

An ATF test run completed for me with this fix.

port-riscv/58006: ATF tests no longer complete on riscv-riscv64

Default pmap_stealdebug to false

Change the PMAP_STEAL_MEMORY debug output from aprint_debug.

The new printfs are conditional on pmap_stealdebug and the DEBUG compile
option. The former defaults to true, but can be changed at a boot -d ddb
prompt.

ci_data.cpu_kcpuset -> ci_kcpuset

NFCI.

branches: 1.74.2;
Provide MI PMAP support on AARCH64

KNF

MI PMAP EFI_RUNTIME support

No need to hold the pmap_tlb_miss_lock when calling pmap_segtab_destroy

Rename pm_count to pm_refcnt

MI PMAP hardware page table walker support.

This is based on code given to me by Matt Thomas a long time ago with
many updates and bugs fixes from me.

Correct the pmap_kstart_segtab entry in pmap_kern_segtab

whitespace - remove spaces before tabs

A simplification and some minor whitespace

Introduce PMAP_PV_TRACK_ONLY_STUBS option, by which only empty stubs for
global functions in pmap_pvt.h are provided, instead of real support for
PV tracking.

Necessary for powerpc: Only one sub-arch (oea) has PV tracking support.
Others (booke/ibm4xx) do not at the moment (probably never for ibm4xx),
but __HAVE_PMAP_PV_TRACK is necessary, so that modules can be shared by
all of sub-archs.

sys: Use membar_release/acquire around reference drop.

This just goes through my recent reference count membar audit and
changes membar_exit to membar_release and membar_enter to
membar_acquire -- this should make everything cheaper on most CPUs
without hurting correctness, because membar_acquire is generally
cheaper than membar_enter.

sys: Membar audit around reference count releases.

If two threads are using an object that is freed when the reference
count goes to zero, we need to ensure that all memory operations
related to the object happen before freeing the object.

Using an atomic_dec_uint_nv(&refcnt) == 0 ensures that only one
thread takes responsibility for freeing, but it's not enough to
ensure that the other thread's memory operations happen before the
freeing.

Consider:

Thread A Thread B
obj->foo = 42; obj->baz = 73;
mumble(&obj->bar); grumble(&obj->quux);
/* membar_exit(); */ /* membar_exit(); */
atomic_dec -- not last atomic_dec -- last
/* membar_enter(); */
KASSERT(invariant(obj->foo,
obj->bar));
free_stuff(obj);

The memory barriers ensure that

obj->foo = 42;
mumble(&obj->bar);

in thread A happens before

KASSERT(invariant(obj->foo, obj->bar));
free_stuff(obj);

in thread B. Without them, this ordering is not guaranteed.

So in general it is necessary to do

membar_exit();
if (atomic_dec_uint_nv(&obj->refcnt) != 0)
return;
membar_enter();

to release a reference, for the `last one out hit the lights' style
of reference counting. (This is in contrast to the style where one
thread blocks new references and then waits under a lock for existing
ones to drain with a condvar -- no membar needed thanks to mutex(9).)

I searched for atomic_dec to find all these. Obviously we ought to
have a better abstraction for this because there's so much copypasta.
This is a stop-gap measure to fix actual bugs until we have that. It
would be nice if an abstraction could gracefully handle the different
styles of reference counting in use -- some years ago I drafted an
API for this, but making it cover everything got a little out of hand
(particularly with struct vnode::v_usecount) and I ended up setting
it aside to work on psref/localcount instead for better scalability.

I got bored of adding #ifdef __HAVE_ATOMIC_AS_MEMBAR everywhere, so I
only put it on things that look performance-critical on 5sec review.
We should really adopt membar_enter_preatomic/membar_exit_postatomic
or something (except they are applicable only to atomic r/m/w, not to
atomic_load/store_*, making the naming annoying) and get rid of all
the ifdefs.

remove KERNHIST_INIT_STATIC(). it stradles the line between usable
early in boot and broken early in boot by requiring a partly static
structure with another structure that must be present by the time
any uses are performed. theoretically platform code could allocate
a chunk while seting up memory and assign it here, giving a dynamic
sizing for the entry list, but the reality is that all users have
a statically allocated entry list as well.

the existing KERNHIST_LINK_STATIC() is used in conjunction with
KERNHIST_INITIALIZER() instead.

this stops a NULL pointer deref when the _LOG() macro is called
before the storage is linked in, which happens with GCC 10 on OCTEON
with UVMHIST enabled, crashing in very early kernel init.

branches: 1.61.2;
Support pmap_growkernel and KASAN shadow mapping of the new KVA.

Neither mips nor ppc booke actually use pmap_growkernel (at present).

Thanks to rin@ for testing a similar patch on ppc booke.

s/pfi_faultpte/&p/ for consistency with arm / other uses of ptep

Don't use %jx for 0 or 1 - just use %jd in UVMHIST_LOG format.

Support __HAVE_PMAP_PV_TRACK in sys/uvm/pmap based pmaps (aka common pmap)

branches: 1.57.2;
%#jx vs %jx consistency in UVMHIST_LOG

Whitespace

move pmap segtab history into a new history of only 1000 entries,
but will overflow much slower than the main pmap history.

move various debug info into kernhist. make pte array checker
into an array and use it in pmap_segtab_release() and
pmap_pte_reserve(). move check before MD callback(), incase it
wants to change ptes for some reason (they're passed in, but
this callback is currently always NULL.)

clean up some history logs to reduce the number of lines required.

Remove trailing \n from UVMHIST_LOG() format strings.

More UVMHIST_LOG. Remove some commented output printfs.

Fix a comment

Provide a pmap_segtab_deactivate for symmetry with pmap_segtab_activate
and use it in pmap_deactivate

Call pmap_md_xtab_{,de}activate from pmap_segtab_{,de}activate to be used
for PMAP_HWPAGEWALKER and any caches ops that might be required.

Provide empty (for now) pmap_md_xtab_{,de}activate functions on the
platforms that use sys/uvm/pmap

Always call pmap_segtab_activate in pmap_activate. pmap_segtab_activate
does the right thing if called with non-curlwp.

Use UVMHIST_CALLARGS

branches: 1.48.2;
pmap_remove_all(): Return a boolean value to indicate the behaviour. If
true, all mappings have been removed, the pmap is totally cleared out, and
UVM can then avoid doing the work to call pmap_remove() for each map entry.
If false, either nothing has been done, or some helpful arch-specific voodoo
has taken place.

pmap_tlb_miss_lock needs to be globally visible.

With DEBUG defined, it's possible to execute a TLB-vs-segmap consistency
check from a (soft) interrupt handler. But if a platform does not otherwise
require the pmap_tlb_miss_lock, then where will be a brief window of
inconsistency that, while harmless, will still fire an assertion in the
consistency check.

Fix this with the following changes:
1- Refactor the pmap_tlb_miss_lock into MI code and rename it from
pmap_tlb_miss_lock_{enter,exit}() to pmap_tlb_miss_lock_{enter,exit}().
MD code can still define the "md" hooks as necessary, and if so, will
override the common implementation.
2- Provde a pmap_bootstrap_common() function to perform common pmap bootstrap
operations, namely initializing the pmap_tlb_miss_lock if it's needed.
If MD code overrides the implementation, it's responsible for initializing
its own lock.
3- Call pmap_bootstrap_common() from the mips, powerpc booke, and riscv
pmap_bootstrap() routines. (This required adding one for riscv.)
4- Switch powerpc booke to the common pmap_tlb_miss_lock.
5- Enable pmap_tlb_miss_lock if DEBUG is defined, even if it's not otherwise
required.

PR port-mips/55062 (Failed assertion in pmap_md_tlb_check_entry())

Remove duplicate #includes

Define and use VM_PAGEMD_PVLIST_EMPTY_P

Remove KASSERT(!VM_PAGEMD_PVLIST_LOCKED_P(mdpg)) - can only assert that it
is owned

Provide and use PV_ISKENTER_P. NFCI.

Make a comment generic and not MIPS specific

branches: 1.40.2; 1.40.6;
Remove unneeded casts to (uintptr_t). This is already taken care of in
the xxxHIST_LOG() macros.

No need to pull-up to -8 - the extra cast really won't hurt anything.

And replace an instance of "%p" conversion with "%#jx"

Remove a stray '"' (obvious typo) and add a couple of casts that are
probably needed.

Update the kernhist(9) kernel history code to address issues identified
in PR kern/52639, as well as some general cleaning-up...

(As proposed on tech-kern@ with additional changes and enhancements.)

Details of changes:

* All history arguments are now stored as uintmax_t values[1], both in
the kernel and in the structures used for exporting the history data
to userland via sysctl(9). This avoids problems on some architectures
where passing a 64-bit (or larger) value to printf(3) can cause it to
process the value as multiple arguments. (This can be particularly
problematic when printf()'s format string is not a literal, since in
that case the compiler cannot know how large each argument should be.)

* Update the data structures used for exporting kernel history data to
include a version number as well as the length of history arguments.

* All [2] existing users of kernhist(9) have had their format strings
updated. Each format specifier now includes an explicit length
modifier 'j' to refer to numeric values of the size of uintmax_t.

* All [2] existing users of kernhist(9) have had their format strings
updated to replace uses of "%p" with "%#jx", and the pointer
arguments are now cast to (uintptr_t) before being subsequently cast
to (uintmax_t). This is needed to avoid compiler warnings about
casting "pointer to integer of a different size."

* All [2] existing users of kernhist(9) have had instances of "%s" or
"%c" format strings replaced with numeric formats; several instances
of mis-match between format string and argument list have been fixed.

* vmstat(1) has been modified to handle the new size of arguments in the
history data as exported by sysctl(9).

* vmstat(1) now provides a warning message if the history requested with
the -u option does not exist (previously, this condition was silently
ignored, with only a single blank line being printed).

* vmstat(1) now checks the version and argument length included in the
data exported via sysctl(9) and exits if they do not match the values
with which vmstat was built.

* The kernhist(9) man-page has been updated to note the additional
requirements imposed on the format strings, along with several other
minor changes and enhancements.

[1] It would have been possible to use an explicit length (for example,
uint64_t) for the history arguments. But that would require another
"rototill" of all the users in the future when we add support for an
architecture that supports a larger size. Also, the printf(3) format
specifiers for explicitly-sized values, such as "%"PRIu64, are much
more verbose (and less aesthetically appealing, IMHO) than simply
using "%ju".

[2] I've tried very hard to find "all [the] existing users of kernhist(9)"
but it is possible that I've missed some of them. I would be glad to
update any stragglers that anyone identifies.

There's no need to call pmap_tlb_invalidate_addr if pmap_remove_all was
called and PMAP_DEFERRED_ACTIVATE is set.

Use pte_set

branches: 1.34.2;
Sprinkle some KASSERTs

Fix a comment

Remove unused LNAME macro

Fix a UVMHIST_LOG after the "%s" removal

branches: 1.30.2;
Improve a comment

Trailing whitespace

avoid using %s in UVMHIST.

branches: 1.27.2;
PHYSMEM -> PHYSSEG to fix build

"Make NetBSD great again!"

Introduce uvm_hotplug(9) to the kernel.

Many thanks, in no particular order to:

TNF, for funding the project.

Chuck Silvers - for multiple API reviews and feedback.
Nick Hudson - for testing on multiple architectures and bugfix patches.
Everyone who helped with boot testing.

KeK (http://www.kek.org.in) for hosting the primary developers.

extend the pmap_activate/pmap_deactivate UVMHIST logs to include the
pid, lid, and either l_name or p_comm.

Move some code before pmap_enter_pv in pmap_enter so that when we are
re-mapping a VA to a new PA the old mapping is removed first. This means
the cache alias code need to do less work and works better with the last
va tracking.

Increment resident_count if we're remapping onto new PA as
pmap_remove -> pmap_pte_remote will decrement it

When removing a page, make sure to clear its execness regardless of whether
the page is clean or dirty. This fixes the problem of execpages leaking
into the freepage lists.

put a variable under the #ifdef it's only used in.

Don't track kenter_pa/kremove PVs unless we are worrying about cache aliasing.

Only include `static inline pmap_asid_check()` if it might be used.

Should fix HEAD-llvm evbppc autobuild.

branches: 1.18.2;
Spell PMAP_TLB_NEED_SHOOTDOWN correctly

Trailing whitespace

Fix build by removing accidential duplicate line.

Changes so that MIPS can use the common pmap.
Change/augment the virtual cache alias callbacks.

KNF. Remove extra spaces. No functional change.

Remove unnecessary #include for sys/shm.h - there's nothing here that needs
anything from there.

Add virtual_start to pmap_limits. This allows MD to steal address space
before pmap_bootstrap.

Disable pmap_md_tlb_check_entry, when MP.

Avoid race condition between PTE update and TLB miss walk.

Use PMAP_TLB_MAX instead of MAXCPUS.

fix build failure when UVMHIST is defined.

fix compile failure.

pmap->pm_active and pmap->pm_onproc must be destroyed.

Initialize pmap->pm_active and pmap->pm_onproc.
Avoid "panic: kernel diagnostic assertion "!pmap_tlb_intersecting_onproc_p(pm, ti)" failed: file "/usr/src/sys/uvm/pmap/pmap_tlb.c", line 762".

branches: 1.4.6;
Mark a potentially unused variable

In the non-MP case, just initialize onproc to NULL.

Make this kcpuset_t instead of the private __cpuset_t
Add improvements for single TLB implementation (PPC, ARM).

branches: 1.1.2; 1.1.4; 1.1.10;
move from common/pmap/tlb -> uvm/pmap

Sprinkle #ifdef _KERNEL

branches: 1.27.2;
Fix types in pmap_page_clear_attributes so that the top bits of
the u_long mdpg_attrs aren't dropped giving atomic_cas_ulong no
chance of completing if any of the top bits is set.

Update pmap_page_set_attributes for consistency.

An ATF test run completed for me with this fix.

port-riscv/58006: ATF tests no longer complete on riscv-riscv64

branches: 1.26.2;
_KERNEL_OPT protection

Provide MI PMAP support on AARCH64

Rename pm_count to pm_refcnt

Fix the crash(1) build for mips platforms

MI PMAP hardware page table walker support.

This is based on code given to me by Matt Thomas a long time ago with
many updates and bugs fixes from me.

Introduce PMAP_PV_TRACK_ONLY_STUBS option, by which only empty stubs for
global functions in pmap_pvt.h are provided, instead of real support for
PV tracking.

Necessary for powerpc: Only one sub-arch (oea) has PV tracking support.
Others (booke/ibm4xx) do not at the moment (probably never for ibm4xx),
but __HAVE_PMAP_PV_TRACK is necessary, so that modules can be shared by
all of sub-archs.

Support pmap_growkernel and KASAN shadow mapping of the new KVA.

Neither mips nor ppc booke actually use pmap_growkernel (at present).

Thanks to rin@ for testing a similar patch on ppc booke.

Remove variable in function declaration argument

Support __HAVE_PMAP_PV_TRACK in sys/uvm/pmap based pmaps (aka common pmap)

branches: 1.17.2;
move pmap segtab history into a new history of only 1000 entries,
but will overflow much slower than the main pmap history.

move various debug info into kernhist. make pte array checker
into an array and use it in pmap_segtab_release() and
pmap_pte_reserve(). move check before MD callback(), incase it
wants to change ptes for some reason (they're passed in, but
this callback is currently always NULL.)

clean up some history logs to reduce the number of lines required.

Provide a pmap_segtab_deactivate for symmetry with pmap_segtab_activate
and use it in pmap_deactivate

Call pmap_md_xtab_{,de}activate from pmap_segtab_{,de}activate to be used
for PMAP_HWPAGEWALKER and any caches ops that might be required.

Provide empty (for now) pmap_md_xtab_{,de}activate functions on the
platforms that use sys/uvm/pmap

Comment updates

Fix build for ports using uvm/pmap; pmap_remove_all() returns a boolean.

With DEBUG defined, it's possible to execute a TLB-vs-segmap consistency
check from a (soft) interrupt handler. But if a platform does not otherwise
require the pmap_tlb_miss_lock, then where will be a brief window of
inconsistency that, while harmless, will still fire an assertion in the
consistency check.

Fix this with the following changes:
1- Refactor the pmap_tlb_miss_lock into MI code and rename it from
pmap_tlb_miss_lock_{enter,exit}() to pmap_tlb_miss_lock_{enter,exit}().
MD code can still define the "md" hooks as necessary, and if so, will
override the common implementation.
2- Provde a pmap_bootstrap_common() function to perform common pmap bootstrap
operations, namely initializing the pmap_tlb_miss_lock if it's needed.
If MD code overrides the implementation, it's responsible for initializing
its own lock.
3- Call pmap_bootstrap_common() from the mips, powerpc booke, and riscv
pmap_bootstrap() routines. (This required adding one for riscv.)
4- Switch powerpc booke to the common pmap_tlb_miss_lock.
5- Enable pmap_tlb_miss_lock if DEBUG is defined, even if it's not otherwise
required.

PR port-mips/55062 (Failed assertion in pmap_md_tlb_check_entry())

Misc changes in RISC-V. Start changing the memory layout, too.

Usee __BIT()

Trailing whitespace

branches: 1.9.4; 1.9.8;
Trailing whitespace

Multiple inclusion protection define consistency

Changes so that MIPS can use the common pmap.
Change/augment the virtual cache alias callbacks.

KNF. Remove extra spaces. No functional change.

Add virtual_start to pmap_limits. This allows MD to steal address space
before pmap_bootstrap.

branches: 1.4.6;
Merge riastradh-drm2 to HEAD.

Make this kcpuset_t instead of the private __cpuset_t
Add improvements for single TLB implementation (PPC, ARM).

branches: 1.2.2;
Split tlb related stuff into pmap_tlb.h so that can be used for ASID mgmt
for non-soft TLB pmaps.

branches: 1.1.2; 1.1.4;
move from common/pmap/tlb -> uvm/pmap

Correct a type.

Provide a shared pmap_devmap implementation and convert all pmap_devmap
arrays to use DEVMAP_ENTRY{,_END}

Sprinkle #ifdef _KERNEL

branches: 1.1.6;
Provide a shared pmap_devmap implementation and convert all pmap_devmap
arrays to use DEVMAP_ENTRY{,_END}

Oops, correct misleading #endif comment.

It seems I need a cup of coffee...

Improve wording a bit in a comment for the previous.

For PMAP_PV_TRACK_ONLY_STUBS, comment out pmap_pv_{,un}track().

If modules call these functions, the result should be an
inconsistent state.

Such modules require real PV-tracking support, anyway.

The best we can do should be to make two symbols undefined, and
prevent these modules from loaded.

Introduce PMAP_PV_TRACK_ONLY_STUBS option, by which only empty stubs for
global functions in pmap_pvt.h are provided, instead of real support for
PV tracking.

Necessary for powerpc: Only one sub-arch (oea) has PV tracking support.
Others (booke/ibm4xx) do not at the moment (probably never for ibm4xx),
but __HAVE_PMAP_PV_TRACK is necessary, so that modules can be shared by
all of sub-archs.

need <sys/param.h> for COHERENCY_UNIT

Minor KNF along the way.

branches: 1.10.8;
Use C99-ism to reduce ifdefs. Pointed out by christos@.

pmap_pv_track(): use PMAP_PAGE_INIT() otherwise the x86 pmap pukes.

Revert previous (include of sys/param.h) - the headers requiring this
have been fixed.

Add mising sys/param.h include (for COHERENCY_UNIT, now needed in uvm headers)

KNF

Convert pmap_pvt to atomic_load/store.

sys/atomic.h for membar_*

branches: 1.3.16; 1.3.20;
Use IPL_NONE for pserialized lock. Assert sleepable. (OOPS.)

branches: 1.2.2;
Remove #if 0 / #endif includes

Split out the pmap_pv_track stuff for use by others.

Discussed with riastradh@

pmap_pvt.h: Fix bogus include.

branches: 1.2.4;
Multiple inclusion protection define consistency

branches: 1.1.2;
Split out the pmap_pv_track stuff for use by others.

Discussed with riastradh@

KASSERT -> KASSERTMSG

Fix build when KERNHIST defined, but not UVMHIST

Rename pmap_md_pdetab_destroy to pmap_md_pdetab_fini to match
pmap_md_pdetab_init.

Call pmap_md_pdetab_fini from pmap_segtab_destroy.

In pmap_pte_reserve ensure we're atomically swapping out an invalid entry
otherwise concurrent updates might both think they've updated the entry.

MI PMAP hardware page table walker support.

This is based on code given to me by Matt Thomas a long time ago with
many updates and bugs fixes from me.

Rename pmap_segtab_t *stp to stb for consistency with a future
pmap_pdetab_t *ptb. pmap_pdetab_t *ptp would be far too confusing.

NFC. Same code before and after.

Consistently use %#jx instead of 0x%jx or just %jx in UVMHIST_LOG formats

branches: 1.26.2;
%#jx vs %jx consistency in UVMHIST_LOG

Whitespace

Cast pointer arguments of UVMHIST_CALLARGS() into uintptr_t.

Appease GCC9 -Wpointer-to-int-cast on ILP32 environments.

Remove pte_zero_p and simply check against 0.

Remove the #if defined(__mips_n64) && PAGE_SIZE == 8192 and make the
check MI - all PTs are PAGE_SIZE aligned

Trailing whitespace

fix hpcmips and evbppc builds (wrong type in panic()).

move pmap segtab history into a new history of only 1000 entries,
but will overflow much slower than the main pmap history.

move various debug info into kernhist. make pte array checker
into an array and use it in pmap_segtab_release() and
pmap_pte_reserve(). move check before MD callback(), incase it
wants to change ptes for some reason (they're passed in, but
this callback is currently always NULL.)

clean up some history logs to reduce the number of lines required.

Fix small tyop in a comment.

Improve a panic message ever so slightly

add pmaphist calls around seg_tab[] manipulation. hopefully will
help find what causes this:

panic: pmap_segtab_alloc: pm_segtab.seg_tab[1010] != 0 (0x980000004eeb6068): from free list

Provide a pmap_segtab_deactivate for symmetry with pmap_segtab_activate
and use it in pmap_deactivate

Call pmap_md_xtab_{,de}activate from pmap_segtab_{,de}activate to be used
for PMAP_HWPAGEWALKER and any caches ops that might be required.

Provide empty (for now) pmap_md_xtab_{,de}activate functions on the
platforms that use sys/uvm/pmap

0x%p --> %p for non-external codes.

branches: 1.13.2;
KNF

Use pageq.list instead of listq.list.

Whitespace

Use "segmap" for uvm_wait message in pmap_segtab_alloc

s/pte/ptep/ in pmap_pte_process for consistency with other code. NFCI.

Whitespace

s/ the the / the /

branches: 1.6.8; 1.6.12;
KASSERT -> KASSERTMSG

Trailing whitespace

branches: 1.4.6;
fix the start index generation in pmap_segtab_release() to
ensure it fits in the actual array. fixes N64 binaries from
triggering later panic. move the panic check itself into a
common function that is called from a couple of new places too.

branches: 1.3.2;
Changes so that MIPS can use the common pmap.
Change/augment the virtual cache alias callbacks.

Use PMAP_MAP_POOLPAGE instead of POOL_PHYSTOV since we use PMAP_UNMAP_POOLPAGE.
Use PMAP_ALLOC_POOLPAGE instead of pmap_md_alloc_poolpage.
Cleanup some panic messages.

branches: 1.1.2; 1.1.14; 1.1.16; 1.1.18; 1.1.20;
move from common/pmap/tlb -> uvm/pmap

Trailing whitespace

branches: 1.4.6;
KNF

branches: 1.3.16; 1.3.20;
Changes so that MIPS can use the common pmap.
Change/augment the virtual cache alias callbacks.

branches: 1.2.8;
Split tlb related stuff into pmap_tlb.h so that can be used for ASID mgmt
for non-soft TLB pmaps.

branches: 1.1.2; 1.1.4;
move from common/pmap/tlb -> uvm/pmap

branches: 1.1.4; 1.1.18;
Changes so that MIPS can use the common pmap.
Change/augment the virtual cache alias callbacks.

Don't log anything when !DIAGNOSTIC in pmap_tlb_pai_check as it's a no-op.

Appease KASSERTs for zero ASID CPUs (I mean harts)

Support CPUs that might not have ASIDs in the common pmap.

Improve debug

Fix compile for non-MULTIPROCESSOR and PMAP_TLB_MAX > 1 builds

Fixup UVMHIST builds

KASSERT(kpreempt_disabled()) before accessing curcpu()

Spaces to TABs. NFCI.

Fix UVMHIST build

MI PMAP hardware page table walker support.

This is based on code given to me by Matt Thomas a long time ago with
many updates and bugs fixes from me.

Add a KASSERT to check that tlb_asid_t is a large enough type.

Rmmove an incorrect KASSERT.

fix KASSERTMSG issue

Remove duplicate KASSERT

TAB police.

When adjusting the max ASID count, check if ti->ti_asid_max == 0 as
well. This defaults to 0 for the non-PMAP_TLB_NUM_PIDS case, so would
skip the updated test.

Fix for port-pmax/56466 (which affects all MIPS).

ok srkll@

Fix a logic botch to actually apply the ASID limit returned by
pmap_md_tlb_asid_max.

Pass the pmap in tlb_set_asid for the benefit of aarch64.

comment whitespace

Always expose pmap_tlb_update_addr now that all current PMAP_HWPAGEWALKERs
(arm) users provide the required functions.

Revert previous

Expose pmap_tlb_update_addr to the PMAP_HWPAGEWALKER platforms

branches: 1.41.6;
Whitespace

Whitespace - line continutation alignment

KNF. Add some whitespace to the TLBINV_MAP macro and tlb_invalidate_op
enum.

Unwrap short line KASSERT

Fix inverted logic test in pmap_tlb_shootdown_process for if the victim
is onproc.

s/pmaphist/maphist/ for now

More UVMHIST_LOG. Remove some commented output printfs.

Don't kcpuset_clone every pmap_tlb_shootdown_bystanders. Instead allocate
a kcpuset_t per cpu_info and use that.

Fix UVMHIST bulid

Use UVMHIST_CALLARGS

Make a comment less MIPS specific

branches: 1.30.6;
KNF

Fix a UVMHIST_LOG format

branches: 1.28.4;
fix the DIAGNOSTIC function pmap_tlb_asid_count() to not expect
that TLBINFO_ASID_INUSE_P() returns just 0 or 1; the underlying
__BITMAP_ISSET() actually returns the matching bit nowadays, which
caused miscounting

fixes PR kern/53054 by Sevan Janiyan

adjust KASSERT() triggered in PR port-cobalt/53054 to provide more info

KERNEL_PID is > 0 on powerpc/ibm4xx, need to mask all bits <0,
KERNEL_PID> to avoid triggering KASSERT() checking allocated asid
is bigger than KERNEL_PID; adjust also TLBINFO_ASID_INITIAL_FREE()
accordingly

discussed with Nick

convert to use actual __BITMAP_*() macros from <sys/bitops.h>, and make
it possible to override the ASID bitmap length; default to 256 ASIDs as before

XXX NFCI; compile tested only on evbpcc and evbmips, unfortunately didn't
find any combination of port using the MI pmap_tlb.c and working in QEMU

a bit of DRY - add macro for initial free ASID count

make it possible to not use the icache evcnts

branches: 1.22.2;
Update the kernhist(9) kernel history code to address issues identified
in PR kern/52639, as well as some general cleaning-up...

(As proposed on tech-kern@ with additional changes and enhancements.)

Details of changes:

* All history arguments are now stored as uintmax_t values[1], both in
the kernel and in the structures used for exporting the history data
to userland via sysctl(9). This avoids problems on some architectures
where passing a 64-bit (or larger) value to printf(3) can cause it to
process the value as multiple arguments. (This can be particularly
problematic when printf()'s format string is not a literal, since in
that case the compiler cannot know how large each argument should be.)

* Update the data structures used for exporting kernel history data to
include a version number as well as the length of history arguments.

* All [2] existing users of kernhist(9) have had their format strings
updated. Each format specifier now includes an explicit length
modifier 'j' to refer to numeric values of the size of uintmax_t.

* All [2] existing users of kernhist(9) have had their format strings
updated to replace uses of "%p" with "%#jx", and the pointer
arguments are now cast to (uintptr_t) before being subsequently cast
to (uintmax_t). This is needed to avoid compiler warnings about
casting "pointer to integer of a different size."

* All [2] existing users of kernhist(9) have had instances of "%s" or
"%c" format strings replaced with numeric formats; several instances
of mis-match between format string and argument list have been fixed.

* vmstat(1) has been modified to handle the new size of arguments in the
history data as exported by sysctl(9).

* vmstat(1) now provides a warning message if the history requested with
the -u option does not exist (previously, this condition was silently
ignored, with only a single blank line being printed).

* vmstat(1) now checks the version and argument length included in the
data exported via sysctl(9) and exits if they do not match the values
with which vmstat was built.

* The kernhist(9) man-page has been updated to note the additional
requirements imposed on the format strings, along with several other
minor changes and enhancements.

[1] It would have been possible to use an explicit length (for example,
uint64_t) for the history arguments. But that would require another
"rototill" of all the users in the future when we add support for an
architecture that supports a larger size. Also, the printf(3) format
specifiers for explicitly-sized values, such as "%"PRIu64, are much
more verbose (and less aesthetically appealing, IMHO) than simply
using "%ju".

[2] I've tried very hard to find "all [the] existing users of kernhist(9)"
but it is possible that I've missed some of them. I would be glad to
update any stragglers that anyone identifies.

branches: 1.21.2;
Remove incorrect __diagused

Use the define name PMAP_HWPAGEWALKER and not PMAP_TLB_HWPAGEWALKER

PR/51540: Henning Petersen: replace , with ;

Lock the tlbinfo if it wasn't when doing a pmap_tlb_pai_check

branches: 1.17.2;
Use KERNEL_PID instead of 0

Fix some comments.

Trailing whitespace

Fix typo for build check

Changes so that MIPS can use the common pmap.
Change/augment the virtual cache alias callbacks.

Don't call kcpuset_intersecting_p and then kcpuset_ffs_intersecting since
that the last will tell use what we need to know.

pmap_tlb_intersecting_active_p is not used in some combinations of
platform options as seen by recent ARM changes.

branches: 1.10.2;
s/0/KERNEL_PID/ for correctness

Minor comment update.

branches: 1.8.4;
Change cpu_tlb_info definition based on PMAP_TLB_MAX instead of MULTIPROCESSOR

Compare ASIDs, not pmaps.

Make this compile on booke again.

Allow this to handle H/W tlbs. Some ARM allow for a cheap way to flush all
entries using an ASID from the TLB. Add support for taking advantage of it.
Most ARMs don't have an easy way to find out what's in the TLB so make
record_asids can just say all ASIDs are in use. Fix some off by 1 errors.

Merge riastradh-drm2 to HEAD.

Make this kcpuset_t instead of the private __cpuset_t
Add improvements for single TLB implementation (PPC, ARM).

branches: 1.2.2;
Split tlb related stuff into pmap_tlb.h so that can be used for ASID mgmt
for non-soft TLB pmaps.

branches: 1.1.2; 1.1.4;
move from common/pmap/tlb -> uvm/pmap

Support CPUs that might not have ASIDs in the common pmap.

MI PMAP hardware page table walker support.

This is based on code given to me by Matt Thomas a long time ago with
many updates and bugs fixes from me.

KNF. Add some whitespace to the TLBINV_MAP macro and tlb_invalidate_op
enum.

Provide a TLBINFO_OWNED

convert to use actual __BITMAP_*() macros from <sys/bitops.h>, and make
it possible to override the ASID bitmap length; default to 256 ASIDs as before

XXX NFCI; compile tested only on evbpcc and evbmips, unfortunately didn't
find any combination of port using the MI pmap_tlb.c and working in QEMU

make it possible to not use the icache evcnts

Multiple inclusion protection define consistency

Whitespace

Changes so that MIPS can use the common pmap.
Change/augment the virtual cache alias callbacks.

include <sys/evcnt.h>

Use PMAP_TLB_MAX instead of MAXCPUS.

branches: 1.6.4; 1.6.8; 1.6.10;
Change cpu_tlb_info definition based on PMAP_TLB_MAX instead of MULTIPROCESSOR

Allow this to handle H/W tlbs. Some ARM allow for a cheap way to flush all
entries using an ASID from the TLB. Add support for taking advantage of it.
Most ARMs don't have an easy way to find out what's in the TLB so make
record_asids can just say all ASIDs are in use. Fix some off by 1 errors.

Merge riastradh-drm2 to HEAD.

branches: 1.3.2;
If not MULTIPROCESSOR, just make cpu_tlb_info(ci) return &pmap_tlb0_info

Make this kcpuset_t instead of the private __cpuset_t
Add improvements for single TLB implementation (PPC, ARM).

branches: 1.1.2;
Split tlb related stuff into pmap_tlb.h so that can be used for ASID mgmt
for non-soft TLB pmaps.

Pass the pmap in tlb_set_asid for the benefit of aarch64.

branches: 1.4.4;
Multiple inclusion protection define consistency

Changes so that MIPS can use the common pmap.
Change/augment the virtual cache alias callbacks.

Update multiple inclusion macro

branches: 1.1.2; 1.1.16;
move from common/pmap/tlb -> uvm/pmap

Support __HAVE_PMAP_PV_TRACK in sys/uvm/pmap based pmaps (aka common pmap)

branches: 1.16.8;
pg->phys_addr -> VM_PAGE_TO_PHYS().

Define and use VM_PAGEMD_PVLIST_EMPTY_P

Whitespace

Re-order _P() macros to match bit definitions. NFCI

Provide and use PV_ISKENTER_P. NFCI.

use __nothing

Once more short line to unwrap

Unwrap short lines. NFCI.

branches: 1.8.2;
s/static inline/static __inline/g for consistency.

branches: 1.7.4; 1.7.6;
Use __BIT(0) for PV_KENTER. NFC.

Whitespace - comment alignment.

Multiple inclusion protection define consistency

Use __BIT(). No functional change.

Changes so that MIPS can use the common pmap.
Change/augment the virtual cache alias callbacks.

branches: 1.2.6;
use _KERNEL_OPT around #include

branches: 1.1.2; 1.1.4;
move from common/pmap/tlb -> uvm/pmap